git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/8] refs: cleanup errno sideband ref related functions
@ 2021-04-29 15:32 Han-Wen Nienhuys via GitGitGadget
  2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                   ` (9 more replies)
  0 siblings, 10 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys

v3

 * remove errno as an implicit communication mechanism from refs support
   completely.

Han-Wen Nienhuys (8):
  refs: remove EINVAL specification from the errno sideband in
    read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: make errno output explicit for refs_resolve_ref_unsafe
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: clear errno return in refs_resolve_ref_unsafe()
  refs: stop setting EINVAL and ELOOP in symref resolution
  refs: explicitly propagate errno from refs_read_raw_ref

 refs.c                | 46 ++++++++++++++++++++--------------
 refs.h                |  1 +
 refs/debug.c          |  4 +--
 refs/files-backend.c  | 58 +++++++++++++++++++------------------------
 refs/packed-backend.c | 16 ++++++------
 refs/refs-internal.h  | 31 +++++++++++++++--------
 6 files changed, 86 insertions(+), 70 deletions(-)


base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v1
Pull-Request: https://github.com/git/git/pull/1012
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-04-30  2:38   ` Junio C Hamano
  2021-06-03  2:19   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
                   ` (8 subsequent siblings)
  9 siblings, 2 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
references.

The files ref backend does use EINVAL so parse_loose_ref_contents() can
communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
read in files_read_raw_ref(), but the files backend does not call into
refs_read_raw_ref(), so its EINVAL sideband error is unused.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs/refs-internal.h | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936d..29728a339fed 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,10 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
+ * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
+ * broken; set REF_ISBROKEN in type, and return -1. If there is another error
+ * reading the ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-04-30  3:10   ` Junio C Hamano
  2021-06-03  2:33   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                   ` (7 subsequent siblings)
  9 siblings, 2 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

Errno is a global variable written by almost all system calls, and therefore it
is hard to reason about its state. It's also useless for user-visible errors, as
it leaves no place to report the offending file and/or syscall.

For the copy/rename support, calls to lock_ref_oid_basic() in this file are
followed by:

* lock_ref_oid_basic (copy/rename rollback error path)

* write_ref_to_lockfile (both in the rollback path and the success path of
  copy/rename)

These calls do not inspect the incoming errno. As they perform I/O, they can
clobber errno. For this reason, callers cannot reliably observe the errno that
lock_ref_oid_basic() generated, so it is unsound for programmatic use.

For files_create_symref() and files_reflog_expire(), grepping over callers
showed no callers inspecting errno.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs/files-backend.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 119972ee16f8..c9511da1d387 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		last_errno = errno;
+		int last_errno = errno;
 		if (last_errno != ENOTDIR ||
 		    !refs_verify_refname_available(&refs->base, refname,
 						   extras, skip, err))
@@ -981,20 +978,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +999,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-04-29 15:32 ` [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-04-30  3:34   ` Junio C Hamano
  2021-06-03  2:37   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
                   ` (6 subsequent siblings)
  9 siblings, 2 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c                |  7 +++++--
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 24 ++++++++++++------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 16 +++++++++-------
 5 files changed, 32 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 261fd82beb98..43e2ad6b612a 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
+	int result, failure;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					     type, &failure);
+	errno = failure;
+	return result;
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 922e64fa6ad9..887dbb14be6e 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,14 +238,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
 
 	oidcpy(oid, &null_oid);
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index c9511da1d387..efe493ca1425 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	if (failure_errno)
+		*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno = 0;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db60..a457f18e93c8 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 29728a339fed..ac8a14086724 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,10 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
- * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
- * broken; set REF_ISBROKEN in type, and return -1. If there is another error
- * reading the ref, set errno appropriately and return -1.
+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
+ * and return -1. If the ref exists but is neither a symbolic ref nor an object
+ * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files
+ * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If
+ * there is another error reading the ref, set failure_errno appropriately and
+ * return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -634,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (2 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-06-03  2:51   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 5/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
                   ` (5 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c               | 12 ++++++++++++
 refs.h               |  1 +
 refs/files-backend.c | 19 ++++++++++---------
 refs/refs-internal.h |  8 ++++++++
 4 files changed, 31 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index 43e2ad6b612a..e8ce88040b3e 100644
--- a/refs.c
+++ b/refs.c
@@ -1780,6 +1780,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
+{
+	const char *result = refs_resolve_ref_unsafe(refs, refname,
+						     resolve_flags, oid, flags);
+	*failure_errno = errno;
+	return result;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
diff --git a/refs.h b/refs.h
index 48970dfc7e0f..ede405ac3874 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index efe493ca1425..3ab09871db5e 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
+		if (resolve_errno != ENOTDIR ||
+		    !refs_verify_refname_available(&refs->base, refname, extras,
+						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index ac8a14086724..b9f713b5acd6 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type);
 
+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
+ * failure. */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
+
 /*
  * Write an error to `err` and return a nonzero value iff the same
  * refname appears multiple times in `refnames`. `refnames` must be
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 5/8] refs: add failure_errno to refs_read_raw_ref() signature
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (3 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-04-29 15:32 ` [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This makes the errno output of refs_read_raw_ref explicit.
lock_raw_ref() now explicitly reads the errno output of refs_read_raw_ref.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c                | 26 ++++++++++++--------------
 refs/files-backend.c  |  8 ++++----
 refs/packed-backend.c | 10 ++++++----
 refs/refs-internal.h  |  6 +++---
 4 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/refs.c b/refs.c
index e8ce88040b3e..08f69e2a16f6 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,20 +1671,17 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
-	int result, failure;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure);
-	errno = failure;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, failure_errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
@@ -1725,9 +1722,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
+		int read_failure = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1739,9 +1737,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (read_failure != ENOENT && read_failure != EISDIR &&
+			    read_failure != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -2253,7 +2250,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, NULL)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3ab09871db5e..b6dc1b36c752 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (lstat(path, &st) < 0) {
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c8..03353ce48869 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		*failure_errno = ENOENT;
+		if (failure_errno)
+			*failure_errno = ENOENT;
 		return -1;
 	}
 
@@ -1347,6 +1348,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1359,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure) ||
+		    failure != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index b9f713b5acd6..6a0840e22772 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
  * failure. */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (4 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 5/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-06-03  2:53   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution Han-Wen Nienhuys via GitGitGadget
                   ` (3 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This is done in a separate commit, to pinpoint the precise cause should there be
regressions in error reporting.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 08f69e2a16f6..6e746cb01f24 100644
--- a/refs.c
+++ b/refs.c
@@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 }
 
 /* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
+						 const char *refname,
+						 int resolve_flags,
+						 struct object_id *oid,
+						 int *flags)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
@@ -1777,14 +1778,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	const char *result = refs_resolve_ref_unsafe_errno(
+		refs, refname, resolve_flags, oid, flags);
+	errno = 0;
+	return result;
+}
+
 const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
 					       int *flags, int *failure_errno)
 {
-	const char *result = refs_resolve_ref_unsafe(refs, refname,
-						     resolve_flags, oid, flags);
+	const char *result = refs_resolve_ref_unsafe_errno(
+		refs, refname, resolve_flags, oid, flags);
 	*failure_errno = errno;
 	return result;
 }
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (5 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-06-03  2:55   ` Jonathan Tan
  2021-04-29 15:32 ` [PATCH 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
                   ` (2 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

The only caller of refs_resolve_ref_unsafe_with_errno() is in
refs/files-backend.c, and it only cares about EISDIR and ENOTDIR.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/refs.c b/refs.c
index 6e746cb01f24..597e4d1f18f9 100644
--- a/refs.c
+++ b/refs.c
@@ -1706,7 +1706,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
 			return NULL;
 		}
 
@@ -1766,7 +1765,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
 				return NULL;
 			}
 
@@ -1774,7 +1772,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
 	return NULL;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (6 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution Han-Wen Nienhuys via GitGitGadget
@ 2021-04-29 15:32 ` Han-Wen Nienhuys via GitGitGadget
  2021-06-03  2:13 ` [PATCH 0/8] refs: cleanup errno sideband ref related functions Jonathan Tan
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
  9 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-04-29 15:32 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

The function refs_resolve_ref_unsafe_with_errno should produce an errno output.
Rather than taking the value from the errno (which might contain garbage
beforehand), explicitly propagate the failure_errno coming out of
refs_read_raw_ref().

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs.c | 33 +++++++++++----------------------
 1 file changed, 11 insertions(+), 22 deletions(-)

diff --git a/refs.c b/refs.c
index 597e4d1f18f9..a25d18873c56 100644
--- a/refs.c
+++ b/refs.c
@@ -1684,12 +1684,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
-						 const char *refname,
-						 int resolve_flags,
-						 struct object_id *oid,
-						 int *flags)
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
@@ -1702,6 +1701,7 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
 		flags = &unused_flags;
 
 	*flags = 0;
+	*failure_errno = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
@@ -1728,6 +1728,8 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
 				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
+			*failure_errno = read_failure;
+
 			/* In reading mode, refs must eventually resolve */
 			if (resolve_flags & RESOLVE_REF_READING)
 				return NULL;
@@ -1779,22 +1781,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
 				    int resolve_flags, struct object_id *oid,
 				    int *flags)
 {
-	const char *result = refs_resolve_ref_unsafe_errno(
-		refs, refname, resolve_flags, oid, flags);
-	errno = 0;
-	return result;
-}
-
-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
-					       const char *refname,
-					       int resolve_flags,
-					       struct object_id *oid,
-					       int *flags, int *failure_errno)
-{
-	const char *result = refs_resolve_ref_unsafe_errno(
-		refs, refname, resolve_flags, oid, flags);
-	*failure_errno = errno;
-	return result;
+	int ignore;
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore);
 }
 
 /* backend functions */
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
  2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-04-30  2:38   ` Junio C Hamano
  2021-05-19 12:25     ` Han-Wen Nienhuys
  2021-06-03  2:19   ` Jonathan Tan
  1 sibling, 1 reply; 148+ messages in thread
From: Junio C Hamano @ 2021-04-30  2:38 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget; +Cc: git, Han-Wen Nienhuys, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
> references.

We often use a pattern (which is common) like this:

	if (some_func_in_ref_API(...) < 0) {
		if (errno == ENOENT || errno == EISDIR)
			... it is OK for the file to be missing ...
		else
			... error ...
	}

If a piece of code currently sets EINVAL to errno manually when
signalling a failure by returning a negative value to communicate
with such a caller, we wouldn't see EINVAL mentioned, so such a grep
alone would not help us guarantee the correctness of an update to
stop assignment of EINVAL at all.  The callers must be vetted more
carefully than "we are happy that nobody explicitly mentions EINVAL".

> The files ref backend does use EINVAL so parse_loose_ref_contents() can
> communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
> read in files_read_raw_ref(), but the files backend does not call into
> refs_read_raw_ref(), so its EINVAL sideband error is unused.

This paragraph is confusing.  It says EINVAL is used to signal
lock_raw_ref(), and it says EINVAL is not used by the same files
backend.  Which is correct?  If one part of the backend uses it, and
other parts don't, wouldn't the backend as a whole still use it?

Having said that, ...

> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
> - * and return -1. If the ref exists but is neither a symbolic ref nor
> - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
> - * EINVAL, and return -1. If there is another error reading the ref,
> - * set errno appropriately and return -1.

... the mention (and requirement) of EINVAL seems redundant, as it
sounds sufficient for the caller to inspect 'type' to see if it is
REF_ISBOKEN.  So it may be OK for the code that gives REF_ISBROKEN
to type *not* to set errno to EINVAL, as long as it won't leave it
as ENOENT (meaning, an unrelated system call failed earlier may have
set errno to ENOENT, and after having dealt with such an error, the
control may have reached to the codepath we are interested in
here---errno must be cleared to some value other than ENOENT, and
assigning EINVAL is as good as any).

That is because there is a codeflow like this:

	if (files_read_raw_ref(...)) {
		if (errno == ENOENT) {
			... do various things ...
		} else if (errno == EISDIR) {
			... do different and various things ...
		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
			... deal with broken ref ...
		}
		 ...
	}

where errno is looked at.

> + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
> + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
> + * broken; set REF_ISBROKEN in type, and return -1. If there is another error
> + * reading the ref, set errno appropriately and return -1.

So, this is not sufficient to let caller correctly and safely handle
errors.  "set REF_ISBROKEN in type, set errno to something other
than ENOENT or EISDIR, and then return -1" is necessary, I would
think.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-04-29 15:32 ` [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-04-30  3:10   ` Junio C Hamano
  2021-05-19 12:29     ` Han-Wen Nienhuys
  2021-06-03  2:33   ` Jonathan Tan
  1 sibling, 1 reply; 148+ messages in thread
From: Junio C Hamano @ 2021-04-30  3:10 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget; +Cc: git, Han-Wen Nienhuys, Han-Wen Nienhuys

I 100% agree with you that errno is cumbersome to use and carries
far less information than we want (we do not learn what operation
failed on what path) over a long distance.  It only is useful when
the callchain still knows what path was operated on.

But...

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> For the copy/rename support, calls to lock_ref_oid_basic() in this file are
> followed by:
>
> * lock_ref_oid_basic (copy/rename rollback error path)
>
> * write_ref_to_lockfile (both in the rollback path and the success path of
>   copy/rename)
>
> These calls do not inspect the incoming errno. As they perform I/O, they can
> clobber errno. For this reason, callers cannot reliably observe the errno that
> lock_ref_oid_basic() generated, so it is unsound for programmatic use.

In the latter part of the above, "callers" refers to the callers of
"the copy/rename support" (aka files_copy_or_rename_ref())?

Then I am not sure why "callers cannot reliably observe the errno
that lock_ref_oid_basic() generated" is a problem.  They will see
the errno from the last system call that failed, if they care.  So
their performing I/O is perfectly acceptable, too.

Hence, I am not sure what change the above justifies, if any.

If we can show that no caller of files_copy_or_rename_ref() uses
errno, it is a clear indication that lock_ref_oid_basic() is saving
and restoring errno for no good reason.  I think that is what was
done for the other two callers below.

So I traced what happens after the copy-rename thing gets called.

refs_rename_ref(), rename_ref(), refs_copy_existing_ref() and
copy_existing_ref() (all in refs.c) should be the only callers of
the function.  All callers in builtin/branch.c and builtin/remote.c
of these functions (by the way, refs_X() variants do not seem to be
called from anywhere---are they over-engineered?) just die() when
they signal a failure by returning non-zero, so I think it is safe
and much easier to understand to justify this change like so:

    refs/files-backend.c::lock_ref_oid_basic() tries hard to signal
    how it failed to its callers using errno.  The three callers of
    this file-scope static function are 

    * files_copy_or_rename_ref()
    * files_create_symref()
    * files_reflog_expire()

    None of them looks at errno after seeing a negative return from
    lock_ref_oid_basic() to make any decision, and no caller of
    these three functions looks at errno after they signal a failure
    by returning a negative value.

> For files_create_symref() and files_reflog_expire(), grepping over callers
> showed no callers inspecting errno.

Yes, this is a lot more relevant justification to allow these two
functions, and functions that are called _only_ _by_ these two
functions, stop worrying about errno.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-04-29 15:32 ` [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-04-30  3:34   ` Junio C Hamano
  2021-04-30  6:02     ` Junio C Hamano
  2021-06-03  2:37   ` Jonathan Tan
  1 sibling, 1 reply; 148+ messages in thread
From: Junio C Hamano @ 2021-04-30  3:34 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget; +Cc: git, Han-Wen Nienhuys, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> read_raw_ref_fn needs to supply a credible errno for a number of cases. These
> are primarily:
>
> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
> resulting error codes to create/remove directories as needed.
>
> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
> returning the last successfully resolved symref. This is necessary so
> read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
> du-jour), and we know on which branch to create the first commit.
>
> Make this information flow explicit by adding a failure_errno to the signature
> of read_raw_ref. All errnos from the files backend are still propagated
> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
> relevant.

I like the general direction to move away from errno, which may make
sense only in the context of files backend (e.g. ENOENT would be
left in errno, only because the original "loose ref" implementation
used one file per ref, when a ref we wanted to see did not exist)
and having other backends use the same errno would not make much
sense, as it is not the goal to make other backends like reftable
emulate files backend.

I wonder if in the ideal world, we'd rather want to define our own
enum, not <errno.h>, that is specific to failure modes of ref API
functions and signal failures by returning these values (and the
enum includes 0 as a value to signal success, all other errors are
negative values).

What I am really getting at is if we need an extra "failure"
out-parameter-pointer in the internal API.  I do not mind if it
helps the transition to the ideal world, but I offhand do not see if
we need result and failure as separate variables.

> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> ---
>  refs.c                |  7 +++++--
>  refs/debug.c          |  4 ++--
>  refs/files-backend.c  | 24 ++++++++++++------------
>  refs/packed-backend.c |  8 ++++----
>  refs/refs-internal.h  | 16 +++++++++-------
>  5 files changed, 32 insertions(+), 27 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 261fd82beb98..43e2ad6b612a 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>  		      const char *refname, struct object_id *oid,
>  		      struct strbuf *referent, unsigned int *type)
>  {
> +	int result, failure;
>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
>  					      type);
>  	}
>  
> -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> -					   type);
> +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> +					     type, &failure);
> +	errno = failure;
> +	return result;
>  }
>  
>  /* This function needs to return a meaningful errno on failure */
> diff --git a/refs/debug.c b/refs/debug.c
> index 922e64fa6ad9..887dbb14be6e 100644
> --- a/refs/debug.c
> +++ b/refs/debug.c
> @@ -238,14 +238,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
>  
>  static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  			      struct object_id *oid, struct strbuf *referent,
> -			      unsigned int *type)
> +			      unsigned int *type, int *failure_errno)
>  {
>  	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
>  	int res = 0;
>  
>  	oidcpy(oid, &null_oid);
>  	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
> -					    type);
> +					    type, failure_errno);
>  
>  	if (res == 0) {
>  		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index c9511da1d387..efe493ca1425 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
>  	return refs->loose;
>  }
>  
> -static int files_read_raw_ref(struct ref_store *ref_store,
> -			      const char *refname, struct object_id *oid,
> -			      struct strbuf *referent, unsigned int *type)
> +static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
> +			      struct object_id *oid, struct strbuf *referent,
> +			      unsigned int *type, int *failure_errno)
>  {
>  	struct files_ref_store *refs =
>  		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
> @@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
>  	struct stat st;
>  	int fd;
>  	int ret = -1;
> -	int save_errno;
>  	int remaining_retries = 3;
>  
>  	*type = 0;
> @@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,
>  	ret = parse_loose_ref_contents(buf, oid, referent, type);
>  
>  out:
> -	save_errno = errno;
> +	if (failure_errno)
> +		*failure_errno = errno;
>  	strbuf_release(&sb_path);
>  	strbuf_release(&sb_contents);
> -	errno = save_errno;
>  	return ret;
>  }
>  
> @@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
>  	struct strbuf ref_file = STRBUF_INIT;
>  	int attempts_remaining = 3;
>  	int ret = TRANSACTION_GENERIC_ERROR;
> +	int failure_errno = 0;
>  
>  	assert(err);
>  	files_assert_main_repository(refs, "lock_raw_ref");
> @@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
>  	 * fear that its value will change.
>  	 */
>  
> -	if (files_read_raw_ref(&refs->base, refname,
> -			       &lock->old_oid, referent, type)) {
> -		if (errno == ENOENT) {
> +	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
> +			       type, &failure_errno)) {
> +		if (failure_errno == ENOENT) {
>  			if (mustexist) {
>  				/* Garden variety missing reference. */
>  				strbuf_addf(err, "unable to resolve reference '%s'",
> @@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
>  				 *   reference named "refs/foo/bar/baz".
>  				 */
>  			}
> -		} else if (errno == EISDIR) {
> +		} else if (failure_errno == EISDIR) {
>  			/*
>  			 * There is a directory in the way. It might have
>  			 * contained references that have been deleted. If
> @@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
>  					goto error_return;
>  				}
>  			}
> -		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
> +		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
>  			strbuf_addf(err, "unable to resolve reference '%s': "
>  				    "reference broken", refname);
>  			goto error_return;
>  		} else {
>  			strbuf_addf(err, "unable to resolve reference '%s': %s",
> -				    refname, strerror(errno));
> +				    refname, strerror(failure_errno));
>  			goto error_return;
>  		}
>  
> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index dfecdbc1db60..a457f18e93c8 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
>  	return refs->snapshot;
>  }
>  
> -static int packed_read_raw_ref(struct ref_store *ref_store,
> -			       const char *refname, struct object_id *oid,
> -			       struct strbuf *referent, unsigned int *type)
> +static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
> +			       struct object_id *oid, struct strbuf *referent,
> +			       unsigned int *type, int *failure_errno)
>  {
>  	struct packed_ref_store *refs =
>  		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
> @@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
>  
>  	if (!rec) {
>  		/* refname is not a packed reference. */
> -		errno = ENOENT;
> +		*failure_errno = ENOENT;
>  		return -1;
>  	}
>  
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 29728a339fed..ac8a14086724 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -617,10 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
>   * properly-formatted or even safe reference name. NEITHER INPUT NOR
>   * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
>   *
> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
> - * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
> - * broken; set REF_ISBROKEN in type, and return -1. If there is another error
> - * reading the ref, set errno appropriately and return -1.
> + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
> + * and return -1. If the ref exists but is neither a symbolic ref nor an object
> + * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files
> + * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If
> + * there is another error reading the ref, set failure_errno appropriately and
> + * return -1.
>   *
>   * Backend-specific flags might be set in type as well, regardless of
>   * outcome.
> @@ -634,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
>   * - in all other cases, referent will be untouched, and therefore
>   *   refname will still be valid and unchanged.
>   */
> -typedef int read_raw_ref_fn(struct ref_store *ref_store,
> -			    const char *refname, struct object_id *oid,
> -			    struct strbuf *referent, unsigned int *type);
> +typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
> +			    struct object_id *oid, struct strbuf *referent,
> +			    unsigned int *type, int *failure_errno);
>  
>  struct ref_storage_be {
>  	struct ref_storage_be *next;

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-04-30  3:34   ` Junio C Hamano
@ 2021-04-30  6:02     ` Junio C Hamano
  2021-05-19 12:33       ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Junio C Hamano @ 2021-04-30  6:02 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget; +Cc: git, Han-Wen Nienhuys, Han-Wen Nienhuys

Junio C Hamano <gitster@pobox.com> writes:

> I like the general direction to move away from errno, which may make
> sense only in the context of files backend (e.g. ENOENT would be
> left in errno, only because the original "loose ref" implementation
> used one file per ref, when a ref we wanted to see did not exist)
> and having other backends use the same errno would not make much
> sense, as it is not the goal to make other backends like reftable
> emulate files backend.
>
> I wonder if in the ideal world, we'd rather want to define our own
> enum, not <errno.h>, that is specific to failure modes of ref API
> functions and signal failures by returning these values (and the
> enum includes 0 as a value to signal success, all other errors are
> negative values).
>
> What I am really getting at is if we need an extra "failure"
> out-parameter-pointer in the internal API.  I do not mind if it
> helps the transition to the ideal world, but I offhand do not see if
> we need result and failure as separate variables.

In any case, the change in the function signature helped me catch
that this series invalidates what has been queued on hn/reftable
without updating that topic to align with the new way to handle the
errno (it would have become a silent semantic failure if we instead
encoded the "failure" in the return value).

The following is what I am tentatively using for tonight's
integration when merging this and hn/reftable together to the 'seen'
branch.

Thanks.


 refs/reftable-backend.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git c/refs/reftable-backend.c w/refs/reftable-backend.c
index 1aff21adb4..4385d2d6f5 100644
--- c/refs/reftable-backend.c
+++ w/refs/reftable-backend.c
@@ -52,7 +52,8 @@ static struct reftable_stack *stack_for(struct git_reftable_ref_store *store,
 static int git_reftable_read_raw_ref(struct ref_store *ref_store,
 				     const char *refname, struct object_id *oid,
 				     struct strbuf *referent,
-				     unsigned int *type);
+				     unsigned int *type,
+				     int *failure_errno);
 
 static void clear_reftable_log_record(struct reftable_log_record *log)
 {
@@ -376,7 +377,8 @@ static int fixup_symrefs(struct ref_store *ref_store,
 						&old_oid, &referent,
 						/* mutate input, like
 						   files-backend.c */
-						&update->type);
+						&update->type,
+						&errno);
 		if (err < 0 && errno == ENOENT &&
 		    is_null_oid(&update->old_oid)) {
 			err = 0;
@@ -1538,7 +1540,7 @@ static int reftable_error_to_errno(int err)
 static int git_reftable_read_raw_ref(struct ref_store *ref_store,
 				     const char *refname, struct object_id *oid,
 				     struct strbuf *referent,
-				     unsigned int *type)
+				     unsigned int *type, int *failure_errno)
 {
 	struct git_reftable_ref_store *refs =
 		(struct git_reftable_ref_store *)ref_store;
@@ -1560,12 +1562,12 @@ static int git_reftable_read_raw_ref(struct ref_store *ref_store,
 
 	err = reftable_stack_read_ref(stack, refname, &ref);
 	if (err > 0) {
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		err = -1;
 		goto done;
 	}
 	if (err < 0) {
-		errno = reftable_error_to_errno(err);
+		*failure_errno = reftable_error_to_errno(err);
 		err = -1;
 		goto done;
 	}
@@ -1578,7 +1580,7 @@ static int git_reftable_read_raw_ref(struct ref_store *ref_store,
 		oidread(oid, reftable_ref_record_val1(&ref));
 	} else {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		err = -1;
 	}
 done:

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
  2021-04-30  2:38   ` Junio C Hamano
@ 2021-05-19 12:25     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-05-19 12:25 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Fri, Apr 30, 2021 at 4:38 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Han-Wen Nienhuys <hanwen@google.com>
> >
> > A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
> > references.
>
> We often use a pattern (which is common) like this:
>
>         if (some_func_in_ref_API(...) < 0) {
>                 if (errno == ENOENT || errno == EISDIR)
>                         ... it is OK for the file to be missing ...
>                 else
>                         ... error ...
>         }
>
> If a piece of code currently sets EINVAL to errno manually when
> signalling a failure by returning a negative value to communicate
> with such a caller, we wouldn't see EINVAL mentioned, so such a grep
> alone would not help us guarantee the correctness of an update to
> stop assignment of EINVAL at all.  The callers must be vetted more
> carefully than "we are happy that nobody explicitly mentions EINVAL".

Sure. I looked at the callers, and documented further what I looked
for. But how far should one go? It's a global variable, so
transitively, almost all of the code could be observing the EINVAL
value under very specific circumstances. But it would also be a
terrible, fragile coding style and use of undocumented behavior.

> > The files ref backend does use EINVAL so parse_loose_ref_contents() can
> > communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
> > read in files_read_raw_ref(), but the files backend does not call into
> > refs_read_raw_ref(), so its EINVAL sideband error is unused.
>
> This paragraph is confusing.  It says EINVAL is used to signal
> lock_raw_ref(), and it says EINVAL is not used by the same files
> backend.  Which is correct?  If one part of the backend uses it, and
> other parts don't, wouldn't the backend as a whole still use it?

I tried to clarify the message. files-backend.c makes assumptions
about the errno return for files_read_raw_ref, but it's not making
assumptions about the abstracted API in refs.h

> That is because there is a codeflow like this:
>
>         if (files_read_raw_ref(...)) {
>                 if (errno == ENOENT) {
>                         ... do various things ...
>                 } else if (errno == EISDIR) {
>                         ... do different and various things ...
>                 } else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
>                         ... deal with broken ref ...
>                 }
>                  ...
>         }

as mentioned above, this isn't calling refs_read_raw_ref, so it's not
affected by this patch.

> > + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
> > + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
> > + * broken; set REF_ISBROKEN in type, and return -1. If there is another error
> > + * reading the ref, set errno appropriately and return -1.
>
> So, this is not sufficient to let caller correctly and safely handle
> errors.  "set REF_ISBROKEN in type, set errno to something other
> than ENOENT or EISDIR, and then return -1" is necessary, I would
> think.

I tweaked the comment. Note that the function has only a handful of
callers (and only one caller where this behavior is relevant), and
it's changed in a follow-on patch in this series. Is it worth the
effort to wordsmith this further?

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-04-30  3:10   ` Junio C Hamano
@ 2021-05-19 12:29     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-05-19 12:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Fri, Apr 30, 2021 at 5:10 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> I 100% agree with you that errno is cumbersome to use and carries
> far less information than we want (we do not learn what operation
> failed on what path) over a long distance.  It only is useful when
> the callchain still knows what path was operated on.
>
> But...
>
> "Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > For the copy/rename support, calls to lock_ref_oid_basic() in this file are
> > followed by:
> >
> > * lock_ref_oid_basic (copy/rename rollback error path)
> >
> > * write_ref_to_lockfile (both in the rollback path and the success path of
> >   copy/rename)
> >
> > These calls do not inspect the incoming errno. As they perform I/O, they can
> > clobber errno. For this reason, callers cannot reliably observe the errno that
> > lock_ref_oid_basic() generated, so it is unsound for programmatic use.
>
> In the latter part of the above, "callers" refers to the callers of
> "the copy/rename support" (aka files_copy_or_rename_ref())?
>
> Then I am not sure why "callers cannot reliably observe the errno
> that lock_ref_oid_basic() generated" is a problem.  They will see
> the errno from the last system call that failed, if they care.  So
> their performing I/O is perfectly acceptable, too.
>
> Hence, I am not sure what change the above justifies, if any.
>
> If we can show that no caller of files_copy_or_rename_ref() uses
> errno, it is a clear indication that lock_ref_oid_basic() is saving
> and restoring errno for no good reason.  I think that is what was
> done for the other two callers below.
>
> So I traced what happens after the copy-rename thing gets called.
>
> refs_rename_ref(), rename_ref(), refs_copy_existing_ref() and
> copy_existing_ref() (all in refs.c) should be the only callers of
> the function.  All callers in builtin/branch.c and builtin/remote.c
> of these functions (by the way, refs_X() variants do not seem to be
> called from anywhere---are they over-engineered?) just die() when
> they signal a failure by returning non-zero, so I think it is safe
> and much easier to understand to justify this change like so:
>
>     refs/files-backend.c::lock_ref_oid_basic() tries hard to signal
>     how it failed to its callers using errno.  The three callers of
>     this file-scope static function are
>
>     * files_copy_or_rename_ref()
>     * files_create_symref()
>     * files_reflog_expire()
>
>     None of them looks at errno after seeing a negative return from
>     lock_ref_oid_basic() to make any decision, and no caller of
>     these three functions looks at errno after they signal a failure
>     by returning a negative value.

I stole your message here; hope that's OK. My original message tries
to convey that if you do

/* should return errno */
int a() { .. }

int b() {
   result = a();
   maybe_do_IO();
   return result;
}

then callers of b() can't reason about the errno result of a(),
because they can't know if an error code was generated by
maybe_do_IO() or a(). This means that the errno result of a() is
useless.  (This is assuming that b() doesn't inspect errno, which I
failed to mention.)

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-04-30  6:02     ` Junio C Hamano
@ 2021-05-19 12:33       ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-05-19 12:33 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Fri, Apr 30, 2021 at 8:02 AM Junio C Hamano <gitster@pobox.com> wrote:
> > I like the general direction to move away from errno, which may make
> > sense only in the context of files backend (e.g. ENOENT would be
> > left in errno, only because the original "loose ref" implementation
> > used one file per ref, when a ref we wanted to see did not exist)
> > and having other backends use the same errno would not make much
> > sense, as it is not the goal to make other backends like reftable
> > emulate files backend.
> >
> > I wonder if in the ideal world, we'd rather want to define our own
> > enum, not <errno.h>, that is specific to failure modes of ref API
> > functions and signal failures by returning these values (and the
> > enum includes 0 as a value to signal success, all other errors are
> > negative values).

I think it would be healthy to have a set of canonical error codes in
general,  but I don't know if this will help here. The files and
packed backend want to communicate about very specific conditions
(ENOENT, EISDIR, ENOTDIR), which seem too low-level to put in a
generic error code.

> In any case, the change in the function signature helped me catch
> that this series invalidates what has been queued on hn/reftable
> without updating that topic to align with the new way to handle the
> errno (it would have become a silent semantic failure if we instead
> encoded the "failure" in the return value).
>
> The following is what I am tentatively using for tonight's
> integration when merging this and hn/reftable together to the 'seen'
> branch.

That sounds right. How should I post the next update to hn/reftable ?
(based on which branch?)

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 0/8] refs: cleanup errno sideband ref related functions
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (7 preceding siblings ...)
  2021-04-29 15:32 ` [PATCH 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
@ 2021-06-03  2:13 ` Jonathan Tan
  2021-06-09 11:29   ` Han-Wen Nienhuys
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
  9 siblings, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:13 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, Jonathan Tan

> v3
> 
>  * remove errno as an implicit communication mechanism from refs support
>    completely.

Looking at all the patches, it seems that this patch set is about
functions in Git code that set errno themselves to indicate the category
of failure encountered, and instead of setting errno, we want them to
transmit that information through an out param instead. I notice that
the cover letter talks about "cleanup" and "remove errno", but that
could have been explained in greater detail, I think.

As for whether it is a good idea overall, it could be said that errno is
idiomatic in C and writing "if (myfunc()) { store_errno = errno; ..." is
not much more difficult than "int store_errno; if (myfunc(&store_errno))
{ ...". But presumably Han-Wen thinks the opposite, since he wrote this
patch set, and I'll defer to his opinion on this since he's working on
the ref code and I'm not. Besides that, having the out param is more
explicit (which might be better) and permits chaining of such functions
(e.g. "if (myfunc(&store_errno) || myotherfunc(&store_errno))"). I'll
review this patch set as if this is the approach we want to take.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
  2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-04-30  2:38   ` Junio C Hamano
@ 2021-06-03  2:19   ` Jonathan Tan
  2021-06-09 11:28     ` Han-Wen Nienhuys
  1 sibling, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:19 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> From: Han-Wen Nienhuys <hanwen@google.com>
> 
> A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
> references.
> 
> The files ref backend does use EINVAL so parse_loose_ref_contents() can
> communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
> read in files_read_raw_ref(), but the files backend does not call into
> refs_read_raw_ref(), so its EINVAL sideband error is unused.

Does this mean that there is some code that sets EINVAL, but no code
uses it? If yes, that seems to mean that we can't remove EINVAL from the
documentation, since some code still sets it.

> As the errno sideband is unintuitive and error-prone, remove EINVAL
> value, as a step towards getting rid of the errno sideband altogether.

How is removing one possible value a step towards getting rid of the
errno sideband? I would have thought that we would be working towards
transmitting all existing values to the new sideband (the out param).
Unless we are planning to get rid of all values in the sideband - in
which case, this should be described in the commit message.

[snip]

> ---
>  refs/refs-internal.h | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 467f4b3c936d..29728a339fed 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -617,11 +617,10 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
>   * properly-formatted or even safe reference name. NEITHER INPUT NOR
>   * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
>   *
> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
> - * and return -1. If the ref exists but is neither a symbolic ref nor
> - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
> - * EINVAL, and return -1. If there is another error reading the ref,
> - * set errno appropriately and return -1.
> + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
> + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
> + * broken; set REF_ISBROKEN in type, and return -1. If there is another error
> + * reading the ref, set errno appropriately and return -1.

The rewrapping was unnecessary and makes it hard to see what changed.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-04-29 15:32 ` [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
  2021-04-30  3:10   ` Junio C Hamano
@ 2021-06-03  2:33   ` Jonathan Tan
  2021-06-10 10:02     ` Han-Wen Nienhuys
  1 sibling, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:33 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> From: Han-Wen Nienhuys <hanwen@google.com>
> 
> Errno is a global variable written by almost all system calls, and therefore it
> is hard to reason about its state. It's also useless for user-visible errors, as
> it leaves no place to report the offending file and/or syscall.

I don't think this paragraph is useful.

> For the copy/rename support, calls to lock_ref_oid_basic() in this file are
> followed by:
> 
> * lock_ref_oid_basic (copy/rename rollback error path)
> 
> * write_ref_to_lockfile (both in the rollback path and the success path of
>   copy/rename)
> 
> These calls do not inspect the incoming errno. As they perform I/O, they can
> clobber errno. For this reason, callers cannot reliably observe the errno that
> lock_ref_oid_basic() generated, so it is unsound for programmatic use.
> 
> For files_create_symref() and files_reflog_expire(), grepping over callers
> showed no callers inspecting errno.

This is probably written more clearly as follows:

 No call to the static function lock_ref_oid_basic() is immediately
 followed by an errno check, so stopping setting errno is safe. But as a
 sanity check, lock_ref_oid_basic() is used in these functions:
 - files_copy_or_rename_ref() - here, calls are followed by error() (which
   performs I/O) or write_ref_to_lockfile() (which calls parse_object() which
   may perform I/O)
 - files_create_symref() - here, calls are followed by error() or
   create_symref_locked() (which performs I/O and does not inspect
   errno)
 - files_reflog_expire() - here, calls are followed by error() or
   refs_reflog_exists() (which calls a function in a vtable that is not
   documented to use and/or preserve errno)

 So it is safe to stop setting errno in lock_ref_oid_basic().

The diff itself looks good.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-04-29 15:32 ` [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-04-30  3:34   ` Junio C Hamano
@ 2021-06-03  2:37   ` Jonathan Tan
  2021-06-10 10:05     ` Han-Wen Nienhuys
  1 sibling, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:37 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> From: Han-Wen Nienhuys <hanwen@google.com>
> 
> read_raw_ref_fn needs to supply a credible errno for a number of cases. These
> are primarily:
> 
> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
> resulting error codes to create/remove directories as needed.
> 
> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
> returning the last successfully resolved symref. This is necessary so
> read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
> du-jour), and we know on which branch to create the first commit.
> 
> Make this information flow explicit by adding a failure_errno to the signature
> of read_raw_ref. All errnos from the files backend are still propagated
> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
> relevant.

Looks good.

> @@ -1675,13 +1675,16 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>  		      const char *refname, struct object_id *oid,
>  		      struct strbuf *referent, unsigned int *type)
>  {
> +	int result, failure;
>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
>  					      type);
>  	}
>  
> -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> -					   type);
> +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> +					     type, &failure);
> +	errno = failure;
> +	return result;
>  }

Should we initialize "failure" to 0 here? As a reader, I would assume
that "failure" has the semantics of errno here, which upon success, may
or may not be set.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-04-29 15:32 ` [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-06-03  2:51   ` Jonathan Tan
  2021-06-10 11:27     ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:51 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> From: Han-Wen Nienhuys <hanwen@google.com>
> 
> This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
> contract for the errno output explicit. The implementation still relies on
> the global errno variable to ensure no side effects.

Looking at the next few patches, I think the plan is:

 - Introduce a new function refs_resolve_ref_unsafe_with_errno() which
   returns the error information in an out param instead of errno. Right
   now, it wraps refs_resolve_ref_unsafe().
 - Migrate all callers that require the error information to
   refs_resolve_ref_unsafe_with_errno(), and leave all callers that do
   not require the error information alone (that is, using
   refs_resolve_ref_unsafe()).
 - To ensure that all callers using refs_resolve_ref_unsafe() really do
   not use errno, set it to 0. (This is allowed by the errno contract -
   successes can set errno to whatever they want.)

If this is the plan, it should be written in the commit message.

As it is, this patch handles the first and maybe part of the second
step, and patch 5 handles the rest of the second step? I think the
patches should be more clearly divided.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()
  2021-04-29 15:32 ` [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
@ 2021-06-03  2:53   ` Jonathan Tan
  2021-06-10 11:45     ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:53 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> @@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  }
>  
>  /* This function needs to return a meaningful errno on failure */
> -const char *refs_resolve_ref_unsafe(struct ref_store *refs,
> -				    const char *refname,
> -				    int resolve_flags,
> -				    struct object_id *oid, int *flags)
> +static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
> +						 const char *refname,
> +						 int resolve_flags,
> +						 struct object_id *oid,
> +						 int *flags)

So a third function (refs_resolve_ref_unsafe_errno() - not to be
confused with refs_resolve_ref_unsafe_with_errno(), which has an extra
"with")? Couldn't we just swap the other 2 functions directly instead of
going through this intermediary?

> +const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
> +				    int resolve_flags, struct object_id *oid,
> +				    int *flags)
> +{
> +	const char *result = refs_resolve_ref_unsafe_errno(
> +		refs, refname, resolve_flags, oid, flags);
> +	errno = 0;
> +	return result;
> +}

This is the errno = 0 part that I was talking about in my review of patch 4.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution
  2021-04-29 15:32 ` [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution Han-Wen Nienhuys via GitGitGadget
@ 2021-06-03  2:55   ` Jonathan Tan
  2021-06-10 11:58     ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Jonathan Tan @ 2021-06-03  2:55 UTC (permalink / raw)
  To: gitgitgadget; +Cc: git, hanwenn, hanwen, Jonathan Tan

> From: Han-Wen Nienhuys <hanwen@google.com>
> 
> The only caller of refs_resolve_ref_unsafe_with_errno() is in
> refs/files-backend.c, and it only cares about EISDIR and ENOTDIR.
> 
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> ---
>  refs.c | 3 ---
>  1 file changed, 3 deletions(-)
> 
> diff --git a/refs.c b/refs.c
> index 6e746cb01f24..597e4d1f18f9 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1706,7 +1706,6 @@ static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
>  	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
>  		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
>  		    !refname_is_safe(refname)) {
> -			errno = EINVAL;
>  			return NULL;
>  		}
>  

I don't think this is related to avoiding errno and conveying error
information through an out param. But besides that, as it is, I'm not
sure that this is correct. Even if EINVAL is not checked by the caller,
setting errno to EINVAL here means avoiding exposing a potential
EISDIR/ENOTDIR that a preceding call set. Same comment for the other
instances.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
  2021-06-03  2:19   ` Jonathan Tan
@ 2021-06-09 11:28     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-09 11:28 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:19 AM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> > From: Han-Wen Nienhuys <hanwen@google.com>
> >
> > A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
> > references.
> >
> > The files ref backend does use EINVAL so parse_loose_ref_contents() can
> > communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
> > read in files_read_raw_ref(), but the files backend does not call into
> > refs_read_raw_ref(), so its EINVAL sideband error is unused.
>
> Does this mean that there is some code that sets EINVAL, but no code
> uses it? If yes, that seems to mean that we can't remove EINVAL from the
> documentation, since some code still sets it.

It means that an alternate ref backend doesn't have to return EINVAL
to work properly.

added to commit message.

> > As the errno sideband is unintuitive and error-prone, remove EINVAL
> > value, as a step towards getting rid of the errno sideband altogether.
>
> How is removing one possible value a step towards getting rid of the
> errno sideband? I would have thought that we would be working towards
> transmitting all existing values to the new sideband (the out param).
> Unless we are planning to get rid of all values in the sideband - in
> which case, this should be described in the commit message.

It means that we don't have to spend braincycles in further commits of
this series reasoning about EINVAL.

The existing functions potentially transmit all kinds of errors, eg.
EIO if there was a problem with the media, or ENOTCONN if the git repo
happens to be on a FUSE filesystem that crashed. These are not
relevant to the ref backend, so we don't treat them specially either.

> > - * set errno appropriately and return -1.
> > + * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
> > + * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
> > + * broken; set REF_ISBROKEN in type, and return -1. If there is another error
> > + * reading the ref, set errno appropriately and return -1.
>
> The rewrapping was unnecessary and makes it hard to see what changed.

Fixed.  I suppose .clang-format settings should add some configuration
about the line size for wrapping comments.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 0/8] refs: cleanup errno sideband ref related functions
  2021-06-03  2:13 ` [PATCH 0/8] refs: cleanup errno sideband ref related functions Jonathan Tan
@ 2021-06-09 11:29   ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-09 11:29 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:14 AM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> > v3
> >
> >  * remove errno as an implicit communication mechanism from refs support
> >    completely.
>
> Looking at all the patches, it seems that this patch set is about
> functions in Git code that set errno themselves to indicate the category
> of failure encountered, and instead of setting errno, we want them to
> transmit that information through an out param instead. I notice that
> the cover letter talks about "cleanup" and "remove errno", but that
> could have been explained in greater detail, I think.

What is the right place to document these considerations? The data in
the cover letter doesn't end up in the project history, so it seems
like the wrong place.

> As for whether it is a good idea overall, it could be said that errno is
> idiomatic in C and writing "if (myfunc()) { store_errno = errno; ..." is
> not much more difficult than "int store_errno; if (myfunc(&store_errno))
> { ...". But presumably Han-Wen thinks the opposite, since he wrote this
> patch set, and I'll defer to his opinion on this since he's working on
> the ref code and I'm not. Besides that, having the out param is more
> explicit (which might be better) and permits chaining of such functions
> (e.g. "if (myfunc(&store_errno) || myotherfunc(&store_errno))"). I'll
> review this patch set as if this is the approach we want to take.


it's certainly idiomatic and easy for writing. The problem is that it
makes reading code much more difficult: since all of the code has
access to errno, it's hard to tell which code depends on certain
writes to errno. One signal of this are the extensive discussions in
this series about how to argue (in the commit message) that a certain
code change is safe.


-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-06-03  2:33   ` Jonathan Tan
@ 2021-06-10 10:02     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-10 10:02 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:33 AM Jonathan Tan <jonathantanmy@google.com> wrote:
> > Errno is a global variable written by almost all system calls, and therefore it
> > is hard to reason about its state. It's also useless for user-visible errors, as
> > it leaves no place to report the offending file and/or syscall.
>
> I don't think this paragraph is useful.

Dropped.

> This is probably written more clearly as follows:
>
>  No call to the static function lock_ref_oid_basic() is immediately
>  followed by an errno check, so stopping setting errno is safe. But as a
>  sanity check, lock_ref_oid_basic() is used in these functions:
>  - files_copy_or_rename_ref() - here, calls are followed by error() (which
>    performs I/O) or write_ref_to_lockfile() (which calls parse_object() which
>    may perform I/O)
>  - files_create_symref() - here, calls are followed by error() or
>    create_symref_locked() (which performs I/O and does not inspect
>    errno)
>  - files_reflog_expire() - here, calls are followed by error() or
>    refs_reflog_exists() (which calls a function in a vtable that is not
>    documented to use and/or preserve errno)
>
>  So it is safe to stop setting errno in lock_ref_oid_basic().

I used some of the above text.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-06-03  2:37   ` Jonathan Tan
@ 2021-06-10 10:05     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-10 10:05 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:37 AM Jonathan Tan <jonathantanmy@google.com> wrote:
> Should we initialize "failure" to 0 here? As a reader, I would assume
> that "failure" has the semantics of errno here, which upon success, may
> or may not be set.

Done.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-06-03  2:51   ` Jonathan Tan
@ 2021-06-10 11:27     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-10 11:27 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:51 AM Jonathan Tan <jonathantanmy@google.com> wrote:
> > This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
> > contract for the errno output explicit. The implementation still relies on
> > the global errno variable to ensure no side effects.
>
> Looking at the next few patches, I think the plan is:
>
>  - Introduce a new function refs_resolve_ref_unsafe_with_errno() which
>    returns the error information in an out param instead of errno. Right
>    now, it wraps refs_resolve_ref_unsafe().
>  - Migrate all callers that require the error information to
>    refs_resolve_ref_unsafe_with_errno(), and leave all callers that do
>    not require the error information alone (that is, using
>    refs_resolve_ref_unsafe()).
>  - To ensure that all callers using refs_resolve_ref_unsafe() really do
>    not use errno, set it to 0. (This is allowed by the errno contract -
>    successes can set errno to whatever they want.)
>
> If this is the plan, it should be written in the commit message.

I added more description to the commit message.

> As it is, this patch handles the first and maybe part of the second
> step, and patch 5 handles the rest of the second step? I think the
> patches should be more clearly divided.

Done.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe()
  2021-06-03  2:53   ` Jonathan Tan
@ 2021-06-10 11:45     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-10 11:45 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:53 AM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> > @@ -1685,10 +1685,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
> >  }
> >
> >  /* This function needs to return a meaningful errno on failure */
> > -const char *refs_resolve_ref_unsafe(struct ref_store *refs,
> > -                                 const char *refname,
> > -                                 int resolve_flags,
> > -                                 struct object_id *oid, int *flags)
> > +static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
> > +                                              const char *refname,
> > +                                              int resolve_flags,
> > +                                              struct object_id *oid,
> > +                                              int *flags)
>
> So a third function (refs_resolve_ref_unsafe_errno() - not to be
> confused with refs_resolve_ref_unsafe_with_errno(), which has an extra
> "with")? Couldn't we just swap the other 2 functions directly instead of
> going through this intermediary?

I've clarified the name. I've done it this way, because it keeps the
diff small. Swapping the functions would require code changes that I
thought would be more work to review.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution
  2021-06-03  2:55   ` Jonathan Tan
@ 2021-06-10 11:58     ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-10 11:58 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Han-Wen Nienhuys via GitGitGadget, git, Han-Wen Nienhuys

On Thu, Jun 3, 2021 at 4:55 AM Jonathan Tan <jonathantanmy@google.com> wrote:
> I don't think this is related to avoiding errno and conveying error
> information through an out param. But besides that, as it is, I'm not
> sure that this is correct. Even if EINVAL is not checked by the caller,
> setting errno to EINVAL here means avoiding exposing a potential
> EISDIR/ENOTDIR that a preceding call set. Same comment for the other
> instances.

You are right, but it's probably moot, because the follow-up commit
stops using errno (making it impossible for EISDIR/ENOTDIR to
haphazardly appear), but I dropped this commit from the series.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v2 0/8] refs: cleanup errno sideband ref related functions
  2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                   ` (8 preceding siblings ...)
  2021-06-03  2:13 ` [PATCH 0/8] refs: cleanup errno sideband ref related functions Jonathan Tan
@ 2021-06-10 12:57 ` Han-Wen Nienhuys via GitGitGadget
  2021-06-10 12:57   ` [PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                     ` (9 more replies)
  9 siblings, 10 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys

v4

 * commit msg tweaks in response to Jun.

Han-Wen Nienhuys (8):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: make errno output explicit for refs_resolve_ref_unsafe
  refs: use refs_resolve_ref_unsafe_with_errno() where needed
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: clear errno return in refs_resolve_ref_unsafe()
  refs: explicitly propagate errno from refs_read_raw_ref

 refs.c                | 51 +++++++++++++++++++++++--------------
 refs/debug.c          |  4 +--
 refs/files-backend.c  | 58 +++++++++++++++++++------------------------
 refs/packed-backend.c | 16 ++++++------
 refs/refs-internal.h  | 31 +++++++++++++++--------
 5 files changed, 90 insertions(+), 70 deletions(-)


base-commit: ebf3c04b262aa27fbb97f8a0156c2347fecafafb
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v2
Pull-Request: https://github.com/git/git/pull/1012

Range-diff vs v1:

 1:  7e8181e77d40 ! 1:  f9b92e62b598 refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
     @@ Metadata
      Author: Han-Wen Nienhuys <hanwen@google.com>
      
       ## Commit message ##
     -    refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn
     +    refs: remove EINVAL errno output from specification of read_raw_ref_fn
      
     -    A grep for EINVAL */*c reveals that no code inspects EINVAL after reading
     -    references.
     +    This commit does not change code; it documents the fact that an alternate ref
     +    backend does not need to return EINVAL from read_raw_ref_fn to function
     +    properly.
      
     -    The files ref backend does use EINVAL so parse_loose_ref_contents() can
     -    communicate to lock_raw_ref() about garbage following the hex SHA1, or a short
     -    read in files_read_raw_ref(), but the files backend does not call into
     -    refs_read_raw_ref(), so its EINVAL sideband error is unused.
     +    This is correct, because refs_read_raw_ref is only called from;
     +
     +    * resolve_ref_unsafe(), which does not care for the EINVAL errno result.
     +
     +    * refs_verify_refname_available(), which does not inspect errno.
     +
     +    * files-backend.c, where errno is overwritten on failure.
     +
     +    * packed-backend.c (is_packed_transaction_needed), which calls it for the
     +      packed ref backend, which never emits EINVAL.
     +
     +    A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
     +    reading references. In addition, the refs.h file does not mention errno at all.
     +
     +    A grep over resolve_ref_unsafe() turned up the following callers that inspect
     +    errno:
     +
     +    * sequencer.c::print_commit_summary, which uses it for die_errno
     +
     +    * lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.
     +
     +    The files ref backend does use EINVAL. The files backend does not call into
     +    the generic API (refs_read_raw), but into the files-specific function
     +    (files_read_raw_ref), which we are not changing in this commit.
      
          As the errno sideband is unintuitive and error-prone, remove EINVAL
          value, as a step towards getting rid of the errno sideband altogether.
     @@ Commit message
          Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs/refs-internal.h ##
      @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,
     -  * properly-formatted or even safe reference name. NEITHER INPUT NOR
     -  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
        *
     -- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
     -- * and return -1. If the ref exists but is neither a symbolic ref nor
     +  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
     +  * and return -1. If the ref exists but is neither a symbolic ref nor
      - * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
      - * EINVAL, and return -1. If there is another error reading the ref,
      - * set errno appropriately and return -1.
     -+ * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
     -+ * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
     -+ * broken; set REF_ISBROKEN in type, and return -1. If there is another error
     -+ * reading the ref, set errno appropriately and return -1.
     ++ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
     ++ * (errno should not be ENOENT) If there is another error reading the
     ++ * ref, set errno appropriately and return -1.
        *
        * Backend-specific flags might be set in type as well, regardless of
        * outcome.
 2:  b2c72097e5e8 ! 2:  cbe09a48036c refs/files-backend: stop setting errno from lock_ref_oid_basic
     @@ Metadata
       ## Commit message ##
          refs/files-backend: stop setting errno from lock_ref_oid_basic
      
     -    Errno is a global variable written by almost all system calls, and therefore it
     -    is hard to reason about its state. It's also useless for user-visible errors, as
     -    it leaves no place to report the offending file and/or syscall.
     +    refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
     +    to its callers using errno.
      
     -    For the copy/rename support, calls to lock_ref_oid_basic() in this file are
     -    followed by:
     +    It is safe to stop setting errno here, because the callers of this
     +    file-scope static function are
      
     -    * lock_ref_oid_basic (copy/rename rollback error path)
     +    * files_copy_or_rename_ref()
     +    * files_create_symref()
     +    * files_reflog_expire()
      
     -    * write_ref_to_lockfile (both in the rollback path and the success path of
     -      copy/rename)
     +    None of them looks at errno after seeing a negative return from
     +    lock_ref_oid_basic() to make any decision, and no caller of these three
     +    functions looks at errno after they signal a failure by returning a
     +    negative value. In particular,
      
     -    These calls do not inspect the incoming errno. As they perform I/O, they can
     -    clobber errno. For this reason, callers cannot reliably observe the errno that
     -    lock_ref_oid_basic() generated, so it is unsound for programmatic use.
     +    * files_copy_or_rename_ref() - here, calls are followed by error()
     +    (which performs I/O) or write_ref_to_lockfile() (which calls
     +    parse_object() which may perform I/O)
      
     -    For files_create_symref() and files_reflog_expire(), grepping over callers
     -    showed no callers inspecting errno.
     +    * files_create_symref() - here, calls are followed by error() or
     +    create_symref_locked() (which performs I/O and does not inspect
     +    errno)
     +
     +    * files_reflog_expire() - here, calls are followed by error() or
     +    refs_reflog_exists() (which calls a function in a vtable that is not
     +    documented to use and/or preserve errno)
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
 3:  ebd7b8380bf7 ! 3:  3e2831e59c8e refs: make errno output explicit for read_raw_ref_fn
     @@ Commit message
          relevant.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
       
      -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
      -					   type);
     ++	failure = 0;
      +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
      +					     type, &failure);
      +	errno = failure;
     @@ refs/debug.c: debug_ref_iterator_begin(struct ref_store *ref_store, const char *
       {
       	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
       	int res = 0;
     - 
     - 	oidcpy(oid, &null_oid);
     +@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
     + 	oidcpy(oid, null_oid());
     + 	errno = 0;
       	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
      -					    type);
      +					    type, failure_errno);
     @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,
        * properly-formatted or even safe reference name. NEITHER INPUT NOR
        * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
        *
     -- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT and return
     -- * -1. If the ref exists but is neither a symbolic ref nor an object ID, it is
     -- * broken; set REF_ISBROKEN in type, and return -1. If there is another error
     -- * reading the ref, set errno appropriately and return -1.
     +- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
     +- * and return -1. If the ref exists but is neither a symbolic ref nor
     +- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
     +- * (errno should not be ENOENT) If there is another error reading the
     +- * ref, set errno appropriately and return -1.
      + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
      + * and return -1. If the ref exists but is neither a symbolic ref nor an object
     -+ * ID, it is broken; set REF_ISBROKEN in type, and return -1. For the files
     -+ * backend, EISDIR and ENOTDIR may be set if the ref name is a directory. If
     -+ * there is another error reading the ref, set failure_errno appropriately and
     -+ * return -1.
     ++ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno
     ++ * should not be ENOENT). The files backend may return EISDIR (if the ref name
     ++ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is
     ++ * another error reading the ref, set failure_errno appropriately and return -1.
        *
        * Backend-specific flags might be set in type as well, regardless of
        * outcome.
 -:  ------------ > 4:  11b2184044d7 refs: make errno output explicit for refs_resolve_ref_unsafe
 4:  dd3eceade4fc ! 5:  005ee8e6fb2a refs: make errno output explicit for refs_resolve_ref_unsafe
     @@ Metadata
      Author: Han-Wen Nienhuys <hanwen@google.com>
      
       ## Commit message ##
     -    refs: make errno output explicit for refs_resolve_ref_unsafe
     +    refs: use refs_resolve_ref_unsafe_with_errno() where needed
      
     -    This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
     -    contract for the errno output explicit. The implementation still relies on
     -    the global errno variable to ensure no side effects.
     +    lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
     +    that needs error information to make logic decisions.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -
     - ## refs.c ##
     -@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     - 	return NULL;
     - }
     - 
     -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
     -+					       const char *refname,
     -+					       int resolve_flags,
     -+					       struct object_id *oid,
     -+					       int *flags, int *failure_errno)
     -+{
     -+	const char *result = refs_resolve_ref_unsafe(refs, refname,
     -+						     resolve_flags, oid, flags);
     -+	*failure_errno = errno;
     -+	return result;
     -+}
     -+
     - /* backend functions */
     - int refs_init_db(struct strbuf *err)
     - {
     -
     - ## refs.h ##
     -@@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     - 				    int resolve_flags,
     - 				    struct object_id *oid,
     - 				    int *flags);
     -+
     - const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
     - 			       struct object_id *oid, int *flags);
     - 
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
       	int mustexist = (old_oid && !is_null_oid(old_oid));
       	int resolve_flags = RESOLVE_REF_NO_RECURSE;
       	int resolved;
     -+	int resolve_errno;
     ++	int resolve_errno = 0;
       
       	files_assert_main_repository(refs, "lock_ref_oid_basic");
       	assert(err);
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
       
       		goto error_return;
       	}
     -
     - ## refs/refs-internal.h ##
     -@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store,
     - 		      const char *refname, struct object_id *oid,
     - 		      struct strbuf *referent, unsigned int *type);
     - 
     -+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
     -+ * failure. */
     -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
     -+					       const char *refname,
     -+					       int resolve_flags,
     -+					       struct object_id *oid,
     -+					       int *flags, int *failure_errno);
     -+
     - /*
     -  * Write an error to `err` and return a nonzero value iff the same
     -  * refname appears multiple times in `refnames`. `refnames` must be
 5:  039fc4be4b90 ! 6:  2b346caf1aed refs: add failure_errno to refs_read_raw_ref() signature
     @@ Commit message
          refs: add failure_errno to refs_read_raw_ref() signature
      
          This makes the errno output of refs_read_raw_ref explicit.
     -    lock_raw_ref() now explicitly reads the errno output of refs_read_raw_ref.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs.c ##
      @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
     @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
      +		      unsigned int *type, int *failure_errno)
       {
      -	int result, failure;
     ++	if (failure_errno)
     ++		*failure_errno = 0;
       	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
       		return refs_read_special_head(ref_store, refname, oid, referent,
       					      type);
       	}
       
     +-	failure = 0;
      -	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
      -					     type, &failure);
      -	errno = failure;
 6:  1bb350ea5d21 ! 7:  d86516219689 refs: clear errno return in refs_resolve_ref_unsafe()
     @@ Commit message
          This is done in a separate commit, to pinpoint the precise cause should there be
          regressions in error reporting.
      
     +    This is implemented by renaming the existing logic to a static function
     +    refs_resolve_unsafe_implicit_errno(), minimizing the code diff.
     +
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
      -				    const char *refname,
      -				    int resolve_flags,
      -				    struct object_id *oid, int *flags)
     -+static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
     -+						 const char *refname,
     -+						 int resolve_flags,
     -+						 struct object_id *oid,
     -+						 int *flags)
     ++static const char *
     ++refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
     ++				       const char *refname, int resolve_flags,
     ++				       struct object_id *oid, int *flags)
       {
       	static struct strbuf sb_refname = STRBUF_INIT;
       	struct object_id unused_oid;
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
      +				    int resolve_flags, struct object_id *oid,
      +				    int *flags)
      +{
     -+	const char *result = refs_resolve_ref_unsafe_errno(
     ++	const char *result = refs_resolve_ref_unsafe_implicit_errno(
      +		refs, refname, resolve_flags, oid, flags);
      +	errno = 0;
      +	return result;
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
       {
      -	const char *result = refs_resolve_ref_unsafe(refs, refname,
      -						     resolve_flags, oid, flags);
     -+	const char *result = refs_resolve_ref_unsafe_errno(
     ++	const char *result = refs_resolve_ref_unsafe_implicit_errno(
      +		refs, refname, resolve_flags, oid, flags);
       	*failure_errno = errno;
       	return result;
 7:  95d64d73353d < -:  ------------ refs: stop setting EINVAL and ELOOP in symref resolution
 8:  9e161eeb5f6b ! 8:  2a9ebe43deac refs: explicitly propagate errno from refs_read_raw_ref
     @@ Commit message
          refs_read_raw_ref().
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     +    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
       }
       
      -/* This function needs to return a meaningful errno on failure */
     --static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
     --						 const char *refname,
     --						 int resolve_flags,
     --						 struct object_id *oid,
     --						 int *flags)
     +-static const char *
     +-refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
     +-				       const char *refname, int resolve_flags,
     +-				       struct object_id *oid, int *flags)
      +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
      +					       const char *refname,
      +					       int resolve_flags,
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
       {
       	static struct strbuf sb_refname = STRBUF_INIT;
       	struct object_id unused_oid;
     -@@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
     +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
       		flags = &unused_flags;
       
       	*flags = 0;
     @@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
       
       	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
       		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
     -@@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
     + 		    !refname_is_safe(refname)) {
     +-			errno = EINVAL;
     ++			*failure_errno = EINVAL;
     + 			return NULL;
     + 		}
     + 
     +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
       				      &read_flags, &read_failure)) {
       			*flags |= read_flags;
       
     @@ refs.c: static const char *refs_resolve_ref_unsafe_errno(struct ref_store *refs,
       			/* In reading mode, refs must eventually resolve */
       			if (resolve_flags & RESOLVE_REF_READING)
       				return NULL;
     +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
     + 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
     + 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
     + 			    !refname_is_safe(refname)) {
     +-				errno = EINVAL;
     ++				*failure_errno = EINVAL;
     + 				return NULL;
     + 			}
     + 
     +@@ refs.c: refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
     + 		}
     + 	}
     + 
     +-	errno = ELOOP;
     ++	*failure_errno = ELOOP;
     + 	return NULL;
     + }
     + 
      @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
       				    int resolve_flags, struct object_id *oid,
       				    int *flags)
       {
     --	const char *result = refs_resolve_ref_unsafe_errno(
     +-	const char *result = refs_resolve_ref_unsafe_implicit_errno(
      -		refs, refname, resolve_flags, oid, flags);
      -	errno = 0;
      -	return result;
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *
      -					       struct object_id *oid,
      -					       int *flags, int *failure_errno)
      -{
     --	const char *result = refs_resolve_ref_unsafe_errno(
     +-	const char *result = refs_resolve_ref_unsafe_implicit_errno(
      -		refs, refname, resolve_flags, oid, flags);
      -	*failure_errno = errno;
      -	return result;

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-06-10 12:57   ` [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
                     ` (8 subsequent siblings)
  9 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936d..f4445e329045 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
  2021-06-10 12:57   ` [PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 11:13     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                     ` (7 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs/files-backend.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 677b7e4cdd2d..6aa0f5c41dd3 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		last_errno = errno;
+		int last_errno = errno;
 		if (last_errno != ENOTDIR ||
 		    !refs_verify_refname_available(&refs->base, refname,
 						   extras, skip, err))
@@ -981,20 +978,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +999,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
  2021-06-10 12:57   ` [PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-06-10 12:57   ` [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 11:34     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
                     ` (6 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs.c                |  8 ++++++--
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 24 ++++++++++++------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 17 +++++++++--------
 5 files changed, 33 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235ea6..bebe3f584da7 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
+	int result, failure;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+	failure = 0;
+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					     type, &failure);
+	errno = failure;
+	return result;
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc341..f12413a9bc0f 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 6aa0f5c41dd3..8f969c8f711f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	if (failure_errno)
+		*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +540,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno = 0;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -629,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db60..a457f18e93c8 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e329045..904b2a9e51ae 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,12 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
+ * and return -1. If the ref exists but is neither a symbolic ref nor an object
+ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno
+ * should not be ENOENT). The files backend may return EISDIR (if the ref name
+ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is
+ * another error reading the ref, set failure_errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +636,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (2 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 11:56     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed Han-Wen Nienhuys via GitGitGadget
                     ` (5 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

In a follow-on commits, we will

* migrate callers that need this error information

* clear errno in refs_resolve_ref_unsafe() to make sure these other callers
aren't using the error output accidentally.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs.c               | 12 ++++++++++++
 refs/refs-internal.h |  8 ++++++++
 2 files changed, 20 insertions(+)

diff --git a/refs.c b/refs.c
index bebe3f584da7..64e2d55adcfb 100644
--- a/refs.c
+++ b/refs.c
@@ -1781,6 +1781,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
+{
+	const char *result = refs_resolve_ref_unsafe(refs, refname,
+						     resolve_flags, oid, flags);
+	*failure_errno = errno;
+	return result;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 904b2a9e51ae..eb97023658f8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type);
 
+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
+ * failure. */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
+
 /*
  * Write an error to `err` and return a nonzero value iff the same
  * refname appears multiple times in `refnames`. `refnames` must be
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (3 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 11:58     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
                     ` (4 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs/files-backend.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8f969c8f711f..5a430aabf623 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
+		if (resolve_errno != ENOTDIR ||
+		    !refs_verify_refname_available(&refs->base, refname, extras,
+						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (4 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 12:06     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
                     ` (3 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This makes the errno output of refs_read_raw_ref explicit.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs.c                | 29 ++++++++++++++---------------
 refs/files-backend.c  |  8 ++++----
 refs/packed-backend.c | 10 ++++++----
 refs/refs-internal.h  |  6 +++---
 4 files changed, 27 insertions(+), 26 deletions(-)

diff --git a/refs.c b/refs.c
index 64e2d55adcfb..ed2dde1c0c6d 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,21 +1671,19 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
-	int result, failure;
+	if (failure_errno)
+		*failure_errno = 0;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	failure = 0;
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure);
-	errno = failure;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, failure_errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
@@ -1726,9 +1724,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
+		int read_failure = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1740,9 +1739,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (read_failure != ENOENT && read_failure != EISDIR &&
+			    read_failure != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -2254,7 +2252,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, NULL)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 5a430aabf623..01c9bd0dbf04 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (lstat(path, &st) < 0) {
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c8..03353ce48869 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		*failure_errno = ENOENT;
+		if (failure_errno)
+			*failure_errno = ENOENT;
 		return -1;
 	}
 
@@ -1347,6 +1348,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1359,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure) ||
+		    failure != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index eb97023658f8..c65d26580ce8 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
  * failure. */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe()
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (5 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 12:19     ` Ævar Arnfjörð Bjarmason
  2021-06-10 12:57   ` [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
                     ` (2 subsequent siblings)
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This is done in a separate commit, to pinpoint the precise cause should there be
regressions in error reporting.

This is implemented by renaming the existing logic to a static function
refs_resolve_unsafe_implicit_errno(), minimizing the code diff.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index ed2dde1c0c6d..191cbf5a330f 100644
--- a/refs.c
+++ b/refs.c
@@ -1687,10 +1687,10 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 }
 
 /* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *
+refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
+				       const char *refname, int resolve_flags,
+				       struct object_id *oid, int *flags)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
@@ -1779,14 +1779,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	const char *result = refs_resolve_ref_unsafe_implicit_errno(
+		refs, refname, resolve_flags, oid, flags);
+	errno = 0;
+	return result;
+}
+
 const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
 					       int *flags, int *failure_errno)
 {
-	const char *result = refs_resolve_ref_unsafe(refs, refname,
-						     resolve_flags, oid, flags);
+	const char *result = refs_resolve_ref_unsafe_implicit_errno(
+		refs, refname, resolve_flags, oid, flags);
 	*failure_errno = errno;
 	return result;
 }
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (6 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
@ 2021-06-10 12:57   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-01 12:26     ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:10   ` [PATCH v2 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
  9 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

The function refs_resolve_ref_unsafe_with_errno should produce an errno output.
Rather than taking the value from the errno (which might contain garbage
beforehand), explicitly propagate the failure_errno coming out of
refs_read_raw_ref().

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs.c | 38 ++++++++++++++------------------------
 1 file changed, 14 insertions(+), 24 deletions(-)

diff --git a/refs.c b/refs.c
index 191cbf5a330f..92c4796078bb 100644
--- a/refs.c
+++ b/refs.c
@@ -1686,11 +1686,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-static const char *
-refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
-				       const char *refname, int resolve_flags,
-				       struct object_id *oid, int *flags)
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
@@ -1703,11 +1703,12 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
 		flags = &unused_flags;
 
 	*flags = 0;
+	*failure_errno = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1730,6 +1731,8 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
 				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
+			*failure_errno = read_failure;
+
 			/* In reading mode, refs must eventually resolve */
 			if (resolve_flags & RESOLVE_REF_READING)
 				return NULL;
@@ -1767,7 +1770,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1775,7 +1778,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
@@ -1783,22 +1786,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
 				    int resolve_flags, struct object_id *oid,
 				    int *flags)
 {
-	const char *result = refs_resolve_ref_unsafe_implicit_errno(
-		refs, refname, resolve_flags, oid, flags);
-	errno = 0;
-	return result;
-}
-
-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
-					       const char *refname,
-					       int resolve_flags,
-					       struct object_id *oid,
-					       int *flags, int *failure_errno)
-{
-	const char *result = refs_resolve_ref_unsafe_implicit_errno(
-		refs, refname, resolve_flags, oid, flags);
-	*failure_errno = errno;
-	return result;
+	int ignore;
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore);
 }
 
 /* backend functions */
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 0/8] refs: cleanup errno sideband ref related functions
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (7 preceding siblings ...)
  2021-06-10 12:57   ` [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
@ 2021-06-14 10:10   ` Han-Wen Nienhuys
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
  9 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-06-14 10:10 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Junio C Hamano

On Thu, Jun 10, 2021 at 2:57 PM Han-Wen Nienhuys via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> v4
>
>  * commit msg tweaks in response to Jun.

I forgot to update the cover letter, but this includes the updates in
response to review by Jonathan Tan.

I think this topic could graduate?

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-06-10 12:57   ` [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 11:13     ` Ævar Arnfjörð Bjarmason
  2021-07-05 14:16       ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 11:13 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
> to its callers using errno.
>
> It is safe to stop setting errno here, because the callers of this
> file-scope static function are
>
> * files_copy_or_rename_ref()
> * files_create_symref()
> * files_reflog_expire()
>
> None of them looks at errno after seeing a negative return from
> lock_ref_oid_basic() to make any decision, and no caller of these three
> functions looks at errno after they signal a failure by returning a
> negative value. In particular,
>
> * files_copy_or_rename_ref() - here, calls are followed by error()
> (which performs I/O) or write_ref_to_lockfile() (which calls
> parse_object() which may perform I/O)
>
> * files_create_symref() - here, calls are followed by error() or
> create_symref_locked() (which performs I/O and does not inspect
> errno)
>
> * files_reflog_expire() - here, calls are followed by error() or
> refs_reflog_exists() (which calls a function in a vtable that is not
> documented to use and/or preserve errno)
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs/files-backend.c | 9 +--------
>  1 file changed, 1 insertion(+), 8 deletions(-)

This all looks good and well justified after the last commit (where we
even mentioned refs_verify_refname_available() explicitly), but...

> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 677b7e4cdd2d..6aa0f5c41dd3 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
>  
>  /*
>   * Locks a ref returning the lock on success and NULL on failure.
> - * On failure errno is set to something meaningful.
>   */
>  static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  					   const char *refname,
> @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  {
>  	struct strbuf ref_file = STRBUF_INIT;
>  	struct ref_lock *lock;
> -	int last_errno = 0;
>  	int mustexist = (old_oid && !is_null_oid(old_oid));
>  	int resolve_flags = RESOLVE_REF_NO_RECURSE;
>  	int resolved;
> @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  		 * to remain.
>  		 */
>  		if (remove_empty_directories(&ref_file)) {
> -			last_errno = errno;
>  			if (!refs_verify_refname_available(
>  					    &refs->base,
>  					    refname, extras, skip, err))
> @@ -962,7 +959,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  						     &lock->old_oid, type);
>  	}
>  	if (!resolved) {
> -		last_errno = errno;
> +		int last_errno = errno;
>  		if (last_errno != ENOTDIR ||
>  		    !refs_verify_refname_available(&refs->base, refname,
>  						   extras, skip, err))

...this particular change gives me some pause, because all the rest is
about squirreling away our own errno for our own caller (which it turns
out, we didn't need).

But in this case we're only guarding against
refs_verify_refname_available() possibly clobbering the errno we just
got on !resolved in refs_verify_refname_available().

So instead I'd expect either this on top:
	
	diff --git a/refs/files-backend.c b/refs/files-backend.c
	index 6aa0f5c41dd..28aa4932529 100644
	--- a/refs/files-backend.c
	+++ b/refs/files-backend.c
	@@ -959,12 +959,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
	 						     &lock->old_oid, type);
	 	}
	 	if (!resolved) {
	-		int last_errno = errno;
	-		if (last_errno != ENOTDIR ||
	+		if (errno != ENOTDIR ||
	 		    !refs_verify_refname_available(&refs->base, refname,
	 						   extras, skip, err))
	 			strbuf_addf(err, "unable to resolve reference '%s': %s",
	-				    refname, strerror(last_errno));
	+				    refname, strerror(errno));
	 
	 		goto error_return;
	 	}

Or, if we are actually worried about the errno being reset as we report
it:
	
	diff --git a/refs/files-backend.c b/refs/files-backend.c
	index 6aa0f5c41dd..8ee6af61f1a 100644
	--- a/refs/files-backend.c
	+++ b/refs/files-backend.c
	@@ -960,11 +960,20 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
	 	}
	 	if (!resolved) {
	 		int last_errno = errno;
	+		errno = 0;
	 		if (last_errno != ENOTDIR ||
	 		    !refs_verify_refname_available(&refs->base, refname,
	-						   extras, skip, err))
	-			strbuf_addf(err, "unable to resolve reference '%s': %s",
	-				    refname, strerror(last_errno));
	+						   extras, skip, err)) {
	+			if (errno)
	+				strbuf_addf(err, "unable to resolve reference '%s': '%s', "
	+					    "also got '%s when reporting the error!",
	+					    refname, strerror(last_errno),
	+					    strerror(errno));
	+			else
	+				strbuf_addf(err, "unable to resolve reference '%s': %s",
	+					    refname, strerror(errno));
	+
	+		}
	 
	 		goto error_return;
	 	}

I think in the end it doesn't matter much, we hit our primary error, so
we're only potentially losing another error on our way out the door.

It's more about clarity, the "last_errno" pattern signals "I'm about to
call something that'll reset the errno I care about", but it's not clear
if that's actually the case here, or if this is just leftover
boilerplate.

In any case running the tests with my second hunk with just a:

	if (errno)
		BUG("lost it?");
	else
		...

Passes all our tests. I don't think it should be the scope of this
series to give this code 100% test coverage, but (looking ahead) there's
no mention of /test/ anywhere in the commit messages/comments.

I think even if we keep your "last_errno" as-is here it would be useful
to at least say something like:

	/*
	 * Just paranoia, we probably won't lose errno in
	 * refs_verify_refname_available().
	 */
	int last_errno = errno;
	...

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-06-10 12:57   ` [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 11:34     ` Ævar Arnfjörð Bjarmason
  2021-07-05 14:34       ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 11:34 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> read_raw_ref_fn needs to supply a credible errno for a number of cases. These
> are primarily:
>
> 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
> resulting error codes to create/remove directories as needed.
>
> 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
> returning the last successfully resolved symref. This is necessary so
> read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
> du-jour), and we know on which branch to create the first commit.
>
> Make this information flow explicit by adding a failure_errno to the signature
> of read_raw_ref. All errnos from the files backend are still propagated
> unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
> relevant.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs.c                |  8 ++++++--
>  refs/debug.c          |  4 ++--
>  refs/files-backend.c  | 24 ++++++++++++------------
>  refs/packed-backend.c |  8 ++++----
>  refs/refs-internal.h  | 17 +++++++++--------
>  5 files changed, 33 insertions(+), 28 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 8c9490235ea6..bebe3f584da7 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>  		      const char *refname, struct object_id *oid,
>  		      struct strbuf *referent, unsigned int *type)
>  {
> +	int result, failure;

Style nit: Different variables should be on different lines, except
where they're the same, so "int i, j", not "int ret, i". Perhaps "ret"
and "errno" are similar enough, but I'd split them, just my 0.02.

Also nit: s/failure/failure_errno/, just like the function signature.

>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
>  					      type);
>  	}
>  
> -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> -					   type);
> +	failure = 0;
> +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> +					     type, &failure);
> +	errno = failure;
> +	return result;
>  }

Is there some subtlety here where this isn't equivalent to the more
simple/straightforward:

	diff --git a/refs.c b/refs.c
	index bebe3f584da..49ab7555de9 100644
	--- a/refs.c
	+++ b/refs.c
	@@ -1675,17 +1675,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,
	 		      const char *refname, struct object_id *oid,
	 		      struct strbuf *referent, unsigned int *type)
	 {
	-	int result, failure;
	 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
	 		return refs_read_special_head(ref_store, refname, oid, referent,
	 					      type);
	 	}
	 
	-	failure = 0;
	-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
	-					     type, &failure);
	-	errno = failure;
	-	return result;
	+	errno = 0;
	+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
	+					     type, &errno);
	 }
	 
	 /* This function needs to return a meaningful errno on failure */

In that case adding the "failure_errno" to the signature won't be needed
either, but I'm assuming we're heading towards some reftable end-state
where that makes more sense.

I can only imagine a case where we think files_read_raw_ref() would
encounter a new errno after it assigned to *failure_errno, which is just
a couple of strbuf_release() calls.

if that is a case we're worried about then like in my comment on 2/8
shouldn't we be explicitly checking for such a lost/different errno?
I.e. something like (should probably be less fatal than BUG(...):
		
	diff --git a/refs.c b/refs.c
	index bebe3f584da..9584ddae392 100644
	--- a/refs.c
	+++ b/refs.c
	@@ -1675,16 +1675,22 @@ int refs_read_raw_ref(struct ref_store *ref_store,
	 		      const char *refname, struct object_id *oid,
	 		      struct strbuf *referent, unsigned int *type)
	 {
	-	int result, failure;
	+	int result, failure_errno = 0;
	+
	 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
	 		return refs_read_special_head(ref_store, refname, oid, referent,
	 					      type);
	 	}
	 
	-	failure = 0;
	+	errno = 0;
	 	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
	-					     type, &failure);
	-	errno = failure;
	+					     type, &failure_errno);
	+	if (errno)
	+		BUG("Got another errno from read_raw_ref()?: %s, failure errno: %s",
	+		    strerror(errno),
	+		    strerror(failure_errno));
	+	else
	+		errno = failure_errno;
	 	return result;
	 }
	 
	diff --git a/refs/files-backend.c b/refs/files-backend.c
	index 8f969c8f711..57cfdf738da 100644
	--- a/refs/files-backend.c
	+++ b/refs/files-backend.c
	@@ -458,8 +458,10 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
	 	ret = parse_loose_ref_contents(buf, oid, referent, type);
	 
	 out:
	-	if (failure_errno)
	+	if (failure_errno) {
	 		*failure_errno = errno;
	+		errno = 0;
	+	}
	 	strbuf_release(&sb_path);
	 	strbuf_release(&sb_contents);
	 	return ret;
	diff --git a/refs/packed-backend.c b/refs/packed-backend.c
	index a457f18e93c..4bcb4777f0f 100644
	--- a/refs/packed-backend.c
	+++ b/refs/packed-backend.c
	@@ -740,6 +740,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
	 	if (!rec) {
	 		/* refname is not a packed reference. */
	 		*failure_errno = ENOENT;
	+		errno = 0;
	 		return -1;
	 	}

Which also passes all our tests, and the packed_read_raw_ref() suggests
how you may have come up with the current pattern.
> [...]
> - * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
> - * and return -1. If the ref exists but is neither a symbolic ref nor
> - * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
> - * (errno should not be ENOENT) If there is another error reading the
> - * ref, set errno appropriately and return -1.
> + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
> + * and return -1. If the ref exists but is neither a symbolic ref nor an object
> + * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno
> + * should not be ENOENT). The files backend may return EISDIR (if the ref name
> + * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is
> + * another error reading the ref, set failure_errno appropriately and return -1.
>   *

This documentation is a bit confusing in light of the above comments. I
assume that "set failure_errno appropriately" here means that it will do
it, but it's really worded like it's suggesting that you should do it,
and does the "another error" suggest that a caller may need to deal with
an arbitrary errno, not just the ENOENT|EISDIR|ENOTDIR?

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-06-10 12:57   ` [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 11:56     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 11:56 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
> contract for the errno output explicit. The implementation still relies on
> the global errno variable to ensure no side effects of this refactoring.
>
> In a follow-on commits, we will
>
> * migrate callers that need this error information
>
> * clear errno in refs_resolve_ref_unsafe() to make sure these other callers
> aren't using the error output accidentally.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs.c               | 12 ++++++++++++
>  refs/refs-internal.h |  8 ++++++++
>  2 files changed, 20 insertions(+)
>
> diff --git a/refs.c b/refs.c
> index bebe3f584da7..64e2d55adcfb 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1781,6 +1781,18 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
>  	return NULL;
>  }
>  
> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
> +					       const char *refname,
> +					       int resolve_flags,
> +					       struct object_id *oid,
> +					       int *flags, int *failure_errno)
> +{
> +	const char *result = refs_resolve_ref_unsafe(refs, refname,
> +						     resolve_flags, oid, flags);
> +	*failure_errno = errno;
> +	return result;
> +}
> +
>  /* backend functions */
>  int refs_init_db(struct strbuf *err)
>  {
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 904b2a9e51ae..eb97023658f8 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>  		      const char *refname, struct object_id *oid,
>  		      struct strbuf *referent, unsigned int *type);
>  
> +/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
> + * failure. */
> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
> +					       const char *refname,
> +					       int resolve_flags,
> +					       struct object_id *oid,
> +					       int *flags, int *failure_errno);
> +
>  /*
>   * Write an error to `err` and return a nonzero value iff the same
>   * refname appears multiple times in `refnames`. `refnames` must be

Okey, this might be eligable for comments similar to what I've had so
far, but no callers yet. Let's read on to those commits, right now I'm
thinking "this looks small enough that it should be squashed into that
future commit"....

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed
  2021-06-10 12:57   ` [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 11:58     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 11:58 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
> that needs error information to make logic decisions.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs/files-backend.c | 19 ++++++++++---------
>  1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 8f969c8f711f..5a430aabf623 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -924,6 +924,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  	int mustexist = (old_oid && !is_null_oid(old_oid));
>  	int resolve_flags = RESOLVE_REF_NO_RECURSE;
>  	int resolved;
> +	int resolve_errno = 0;
>  
>  	files_assert_main_repository(refs, "lock_ref_oid_basic");
>  	assert(err);
> @@ -936,10 +937,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
>  
>  	files_ref_path(refs, &ref_file, refname);
> -	resolved = !!refs_resolve_ref_unsafe(&refs->base,
> -					     refname, resolve_flags,
> -					     &lock->old_oid, type);
> -	if (!resolved && errno == EISDIR) {
> +	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
> +							resolve_flags,
> +							&lock->old_oid, type,
> +							&resolve_errno);
> +	if (!resolved && resolve_errno == EISDIR) {
>  		/*
>  		 * we are trying to lock foo but we used to
>  		 * have foo/bar which now does not exist;
> @@ -959,12 +961,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  						     &lock->old_oid, type);
>  	}
>  	if (!resolved) {
> -		int last_errno = errno;
> -		if (last_errno != ENOTDIR ||
> -		    !refs_verify_refname_available(&refs->base, refname,
> -						   extras, skip, err))
> +		if (resolve_errno != ENOTDIR ||
> +		    !refs_verify_refname_available(&refs->base, refname, extras,
> +						   skip, err))
>  			strbuf_addf(err, "unable to resolve reference '%s': %s",
> -				    refname, strerror(last_errno));
> +				    refname, strerror(resolve_errno));
>  
>  		goto error_return;
>  	}

So, having read 4/8 and this I wonder why
refs_resolve_ref_unsafe_with_errno() is needed at all. It's just a
wrapper that sets errno to a variable you give it, but we could just
document that tha caller should check errno.

So far I haven't seen anything that suggests the below diff-on-top
wouldn't be OK (and all tests pass with it). It steps on the toes of
some of my earlier suggestions, but I'm doing these one-at-a-time.

In any case the comment I adjusted seems like something you should
adjust to. It looks like a TODO for having the sort of function you've
just implemented in refs_resolve_ref_unsafe_with_errno().

	diff --git a/refs.c b/refs.c
	index 64e2d55adcf..a07d852fcdc 100644
	--- a/refs.c
	+++ b/refs.c
	@@ -1688,7 +1688,10 @@ int refs_read_raw_ref(struct ref_store *ref_store,
	 	return result;
	 }
	 
	-/* This function needs to return a meaningful errno on failure */
	+/*
	+ * This function clears errno at the beginning. If it fails the errno
	+ * will be meaningful.
	+ */
	 const char *refs_resolve_ref_unsafe(struct ref_store *refs,
	 				    const char *refname,
	 				    int resolve_flags,
	@@ -1698,6 +1701,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
	 	struct object_id unused_oid;
	 	int unused_flags;
	 	int symref_count;
	+	errno = 0;
	 
	 	if (!oid)
	 		oid = &unused_oid;
	@@ -1781,18 +1785,6 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
	 	return NULL;
	 }
	 
	-const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
	-					       const char *refname,
	-					       int resolve_flags,
	-					       struct object_id *oid,
	-					       int *flags, int *failure_errno)
	-{
	-	const char *result = refs_resolve_ref_unsafe(refs, refname,
	-						     resolve_flags, oid, flags);
	-	*failure_errno = errno;
	-	return result;
	-}
	-
	 /* backend functions */
	 int refs_init_db(struct strbuf *err)
	 {
	diff --git a/refs/files-backend.c b/refs/files-backend.c
	index 5a430aabf62..5a400e55cbf 100644
	--- a/refs/files-backend.c
	+++ b/refs/files-backend.c
	@@ -924,7 +924,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
	 	int mustexist = (old_oid && !is_null_oid(old_oid));
	 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
	 	int resolved;
	-	int resolve_errno = 0;
	 
	 	files_assert_main_repository(refs, "lock_ref_oid_basic");
	 	assert(err);
	@@ -937,11 +936,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
	 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
	 
	 	files_ref_path(refs, &ref_file, refname);
	-	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
	-							resolve_flags,
	-							&lock->old_oid, type,
	-							&resolve_errno);
	-	if (!resolved && resolve_errno == EISDIR) {
	+	resolved = !!refs_resolve_ref_unsafe(&refs->base, refname,
	+					     resolve_flags,
	+					     &lock->old_oid, type);
	+	if (!resolved && errno == EISDIR) {
	 		/*
	 		 * we are trying to lock foo but we used to
	 		 * have foo/bar which now does not exist;
	@@ -961,11 +959,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
	 						     &lock->old_oid, type);
	 	}
	 	if (!resolved) {
	-		if (resolve_errno != ENOTDIR ||
	+		if (errno != ENOTDIR ||
	 		    !refs_verify_refname_available(&refs->base, refname, extras,
	 						   skip, err))
	 			strbuf_addf(err, "unable to resolve reference '%s': %s",
	-				    refname, strerror(resolve_errno));
	+				    refname, strerror(errno));
	 
	 		goto error_return;
	 	}
	

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature
  2021-06-10 12:57   ` [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 12:06     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 12:06 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> This makes the errno output of refs_read_raw_ref explicit.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs.c                | 29 ++++++++++++++---------------
>  refs/files-backend.c  |  8 ++++----
>  refs/packed-backend.c | 10 ++++++----
>  refs/refs-internal.h  |  6 +++---
>  4 files changed, 27 insertions(+), 26 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 64e2d55adcfb..ed2dde1c0c6d 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1671,21 +1671,19 @@ static int refs_read_special_head(struct ref_store *ref_store,
>  	return result;
>  }
>  
> -int refs_read_raw_ref(struct ref_store *ref_store,
> -		      const char *refname, struct object_id *oid,
> -		      struct strbuf *referent, unsigned int *type)
> +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
> +		      struct object_id *oid, struct strbuf *referent,
> +		      unsigned int *type, int *failure_errno)
>  {
> -	int result, failure;
> +	if (failure_errno)
> +		*failure_errno = 0;
>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
>  					      type);
>  	}
>  
> -	failure = 0;
> -	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> -					     type, &failure);
> -	errno = failure;
> -	return result;
> +	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> +					   type, failure_errno);
>  }

Ah, so here we drop the whole intermediate step of having this function
not take a failure_errno itself. I think this would be better squashed
into that earlier change.

>  /* This function needs to return a meaningful errno on failure */
> @@ -1726,9 +1724,10 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
>  
>  	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
>  		unsigned int read_flags = 0;
> +		int read_failure = 0;

Let's call it failure_errno consistently if we end up keeping it.

> -		if (refs_read_raw_ref(refs, refname,
> -				      oid, &sb_refname, &read_flags)) {
> +		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
> +				      &read_flags, &read_failure)) {
>  			*flags |= read_flags;
>  
>  			/* In reading mode, refs must eventually resolve */
> @@ -1740,9 +1739,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
>  			 * may show errors besides ENOENT if there are
>  			 * similarly-named refs.
>  			 */
> -			if (errno != ENOENT &&
> -			    errno != EISDIR &&
> -			    errno != ENOTDIR)
> +			if (read_failure != ENOENT && read_failure != EISDIR &&
> +			    read_failure != ENOTDIR)
>  				return NULL;

But ditto my previous comments, this seems like a whole dance to avoid
reading errno directly in cases where doing so is actually OK. I.e. the
"last_errno" pattern is for things that encounter an errno, do some
other stuff (such as a printf) where they might get /another/ errno (or
reset it), but in this case we can just document "these will set errno
on failure".

>  			oidclr(oid);
> @@ -2254,7 +2252,8 @@ int refs_verify_refname_available(struct ref_store *refs,
>  		if (skip && string_list_has_string(skip, dirname.buf))
>  			continue;
>  
> -		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
> +		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
> +				       &type, NULL)) {
>  			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
>  				    dirname.buf, refname);
>  			goto cleanup;


And if we do care about errno at all, why would we not add it with a
strerror() here instead of explicitly ignoring it?

> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 5a430aabf623..01c9bd0dbf04 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -383,8 +383,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  	if (lstat(path, &st) < 0) {
>  		if (errno != ENOENT)
>  			goto out;
> -		if (refs_read_raw_ref(refs->packed_ref_store, refname,
> -				      oid, referent, type)) {
> +		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
> +				      referent, type, NULL)) {
>  			errno = ENOENT;
>  			goto out;
>  		}

In this case that seems to make sense, since we substitute our own
errno. Maybe I haven't read this all carefully enough, but why are we
not caring about the errno refs_read_raw_ref() might return here? Is it
because we might take the refs_read_special_head() codepath?

In any case, I for one would appreciate a comment/commit message note
about why we ignore the errno these "I'll give you an errno on failure"
functions return in some cases, but not others.

I think it's probably best to split those into another commit, i.e. do
the "we just ferry errno around differently now (if we'll do that at
all)" as one step, and "here we might get errno, but we like our own
better" as another.

> @@ -423,8 +423,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  		 * ref is supposed to be, there could still be a
>  		 * packed ref:
>  		 */
> -		if (refs_read_raw_ref(refs->packed_ref_store, refname,
> -				      oid, referent, type)) {
> +		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
> +				      referent, type, NULL)) {
>  			errno = EISDIR;
>  			goto out;

Ditto.

> diff --git a/refs/packed-backend.c b/refs/packed-backend.c
> index a457f18e93c8..03353ce48869 100644
> --- a/refs/packed-backend.c
> +++ b/refs/packed-backend.c
> @@ -739,7 +739,8 @@ static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  
>  	if (!rec) {
>  		/* refname is not a packed reference. */
> -		*failure_errno = ENOENT;
> +		if (failure_errno)
> +			*failure_errno = ENOENT;
>  		return -1;
>  	}

FWIW I'd think it's better in terms of code readability to not do this,
and make callers who don't care about our errno explicitly provide a
throwaway variable to show that they're not caring, but I don't have the
full picture yet. I.e. make them do something like:

	int ret;
	int got_errno;
	ret = func(..., &got_errno);
	if (!ret) {
		if (!got_errno)
			BUG("error but no errno?");
		/* We don't care, use our own */
		got_errno = ENOSOMETHING
		...

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe()
  2021-06-10 12:57   ` [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 12:19     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 12:19 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> This is done in a separate commit, to pinpoint the precise cause should there be
> regressions in error reporting.
>
> This is implemented by renaming the existing logic to a static function
> refs_resolve_unsafe_implicit_errno(), minimizing the code diff.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs.c | 22 ++++++++++++++++------
>  1 file changed, 16 insertions(+), 6 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index ed2dde1c0c6d..191cbf5a330f 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1687,10 +1687,10 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  }
>  
>  /* This function needs to return a meaningful errno on failure */
> -const char *refs_resolve_ref_unsafe(struct ref_store *refs,
> -				    const char *refname,
> -				    int resolve_flags,
> -				    struct object_id *oid, int *flags)
> +static const char *

The "let's make this static" seems like an unrelated change we should
squash into the earlier "refs: use refs_resolve_ref_unsafe_with_errno()
where needed", we stopped using it in the files backend then. No?

> +refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
> +				       const char *refname, int resolve_flags,
> +				       struct object_id *oid, int *flags)
>  {
>  	static struct strbuf sb_refname = STRBUF_INIT;
>  	struct object_id unused_oid;
> @@ -1779,14 +1779,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
>  	return NULL;
>  }
>  
> +const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
> +				    int resolve_flags, struct object_id *oid,
> +				    int *flags)
> +{
> +	const char *result = refs_resolve_ref_unsafe_implicit_errno(
> +		refs, refname, resolve_flags, oid, flags);
> +	errno = 0;
> +	return result;
> +}
> +
>  const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
>  					       const char *refname,
>  					       int resolve_flags,
>  					       struct object_id *oid,
>  					       int *flags, int *failure_errno)
>  {
> -	const char *result = refs_resolve_ref_unsafe(refs, refname,
> -						     resolve_flags, oid, flags);
> +	const char *result = refs_resolve_ref_unsafe_implicit_errno(
> +		refs, refname, resolve_flags, oid, flags);
>  	*failure_errno = errno;
>  	return result;
>  }

Per my earlier comments this whole thing again seems a bit backwards. We
explicitly clear errno instead of telling the caller to care.

Has this whole thing perhaps had the unstated aim that you were trying
to distinguish the now-remaining refs_resolve_ref_unsafe() callers from
the "I care about errno" by explicitly clearing out errno for them, thus
ensuring that they need to use the wrapper function with "errno" in the
name to get the errno they'd otherwise get?

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-06-10 12:57   ` [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
@ 2021-07-01 12:26     ` Ævar Arnfjörð Bjarmason
  2021-07-05 16:09       ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-01 12:26 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Thu, Jun 10 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> The function refs_resolve_ref_unsafe_with_errno should produce an errno output.
> Rather than taking the value from the errno (which might contain garbage
> beforehand), explicitly propagate the failure_errno coming out of
> refs_read_raw_ref().
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  refs.c | 38 ++++++++++++++------------------------
>  1 file changed, 14 insertions(+), 24 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 191cbf5a330f..92c4796078bb 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1686,11 +1686,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  					   type, failure_errno);
>  }
>  
> -/* This function needs to return a meaningful errno on failure */
> -static const char *
> -refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
> -				       const char *refname, int resolve_flags,
> -				       struct object_id *oid, int *flags)
> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
> +					       const char *refname,
> +					       int resolve_flags,
> +					       struct object_id *oid,
> +					       int *flags, int *failure_errno)
>  {

At this point I'm lost. We just introduced this function two commits
ago, and now it's gone again. I guess the "we should update the comment"
from an earlier reply of mine is addressed in some way here...

>  	static struct strbuf sb_refname = STRBUF_INIT;
>  	struct object_id unused_oid;
> @@ -1703,11 +1703,12 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,

As an aside I really wish our hunk title detection would spot the rename
& use the new name...

>  		flags = &unused_flags;
>  
>  	*flags = 0;
> +	*failure_errno = 0;
>  
>  	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
>  		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
>  		    !refname_is_safe(refname)) {
> -			errno = EINVAL;
> +			*failure_errno = EINVAL;
>  			return NULL;
>  		}
>  
> @@ -1730,6 +1731,8 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
>  				      &read_flags, &read_failure)) {
>  			*flags |= read_flags;
>  
> +			*failure_errno = read_failure;
> +
>  			/* In reading mode, refs must eventually resolve */
>  			if (resolve_flags & RESOLVE_REF_READING)
>  				return NULL;
> @@ -1767,7 +1770,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
>  		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
>  			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
>  			    !refname_is_safe(refname)) {
> -				errno = EINVAL;
> +				*failure_errno = EINVAL;
>  				return NULL;
>  			}
>  
> @@ -1775,7 +1778,7 @@ refs_resolve_ref_unsafe_implicit_errno(struct ref_store *refs,
>  		}
>  	}
>  
> -	errno = ELOOP;
> +	*failure_errno = ELOOP;
>  	return NULL;
>  }
>  
> @@ -1783,22 +1786,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
>  				    int resolve_flags, struct object_id *oid,
>  				    int *flags)
>  {
> -	const char *result = refs_resolve_ref_unsafe_implicit_errno(
> -		refs, refname, resolve_flags, oid, flags);
> -	errno = 0;
> -	return result;
> -}
> -
> -const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
> -					       const char *refname,
> -					       int resolve_flags,
> -					       struct object_id *oid,
> -					       int *flags, int *failure_errno)
> -{
> -	const char *result = refs_resolve_ref_unsafe_implicit_errno(
> -		refs, refname, resolve_flags, oid, flags);
> -	*failure_errno = errno;
> -	return result;
> +	int ignore;
> +	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
> +						  oid, flags, &ignore);
>  }

Okey, so here we have something like the "caller should explicitly
ignore it" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no
longer set "errno" at all for the benefit of the callers of that
function, but instead explicitly throw it away, our
refs_resolve_ref_unsafe_with_errno() no longer sets errno.

But this end-state seems to have resulted in introducing new bugs. A
bunch of functions are thin wrappers for
refs_resolve_ref_unsafe(). Previously they could inspect errno on a -1
return, now they can't.

I didn't look at them all, but just the second one I looked at,
refs_read_ref_full() has a verify_lock() caller which calls it, and that
function then expects a meaningful errno that it in turn ferries up with
"can't verify ref". It in turn is called by lock_ref_oid_basic(), it
ferries up that strbuf with the error, which'll now have a meaningless
strerror(0) in it.

So it seems to me that the refactoring being done here is halfway done,
and results in a buggy end-state. The below diff is my attempt to fix
that up for just one caller, things thath call resolve_gitlink_ref()
(it's literally the third caller out of N, so I'm not picking and
choosing).

In that case I believe that we don't actually have a regression because
of your changes, but I think the changes below are the logical end-state
of what you're starting here. I.e. we have a low-level function that may
encounter an errno, and then callers removed from that via:

    index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()

Where the common case for index_path() is to report the error we got,
but it doesn't call die_errno() or error_errno() now, but really should,
per the below diff (which you can consider to have by SOB if you want to
pick it up).

That passes all tests, except one where we actually fail because of what
I'd argue is a better error message. i.e this in t3700-add.sh:
	
	+ diff -u expect actual
	--- expect      2021-07-01 13:01:23.267001881 +0000
	+++ actual      2021-07-01 13:01:23.267001881 +0000
	@@ -1,2 +1,2 @@
	-error: 'empty/' does not have a commit checked out
	+error: 'empty/' does not have a commit checked out: No such file or directory
	 fatal: adding files failed

In one sense I don't think you need to do this excercise of going all
the way to the top of callers like index_path() that don't get an errno
now, but in another sense I think you really do.

By explicitly clearing "errno" as your end-state does and declaring that
callers of refs_resolve_ref_unsafe() must be as happy to ignore "errno"
as your own callers are, you're potentially introducing subtle bugs
unless we go all the way to the top of the callstack and check if
anything above us cares about errno, which we'll need due to do due to
its global nature.

I think I'd really only be confident that we've got them all if your
chance was amended to something where we add such "int *maybe_errno"
parameters to all upstream callers that may relay the <0 return value we
got.

diff --git a/builtin/update-index.c b/builtin/update-index.c
index f1f16f2de52..54fcbc5be10 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -272,6 +272,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
 {
 	int option;
 	struct cache_entry *ce;
+	int ignore_errno;
 
 	/* Was the old index entry already up-to-date? */
 	if (old && !ce_stage(old) && !ce_match_stat(old, st, 0))
@@ -285,7 +286,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
 	ce->ce_mode = ce_mode_from_stat(old, st->st_mode);
 
 	if (index_path(&the_index, &ce->oid, path, st,
-		       info_only ? 0 : HASH_WRITE_OBJECT)) {
+		       info_only ? 0 : HASH_WRITE_OBJECT, &ignore_errno)) {
 		discard_cache_entry(ce);
 		return -1;
 	}
@@ -325,14 +326,15 @@ static int process_directory(const char *path, int len, struct stat *st)
 {
 	struct object_id oid;
 	int pos = cache_name_pos(path, len);
+	int their_errno = 0;
 
 	/* Exact match: file or existing gitlink */
 	if (pos >= 0) {
 		const struct cache_entry *ce = active_cache[pos];
 		if (S_ISGITLINK(ce->ce_mode)) {
-
 			/* Do nothing to the index if there is no HEAD! */
-			if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+			int ignore_errno;
+			if (resolve_gitlink_ref(path, "HEAD", &oid, &ignore_errno) < 0)
 				return 0;
 
 			return add_one_path(ce, path, len, st);
@@ -358,11 +360,16 @@ static int process_directory(const char *path, int len, struct stat *st)
 	}
 
 	/* No match - should we add it as a gitlink? */
-	if (!resolve_gitlink_ref(path, "HEAD", &oid))
+	if (resolve_gitlink_ref(path, "HEAD", &oid, &their_errno) >= 0)
 		return add_one_path(NULL, path, len, st);
 
 	/* Error out. */
-	return error("%s: is a directory - add files inside instead", path);
+	if (their_errno) {
+		errno = their_errno;
+		return error_errno("%s: is a directory - add files inside instead", path);
+	} else {
+		return error("%s: is a directory - add files inside instead", path);
+	}
 }
 
 static int process_path(const char *path, struct stat *st, int stat_errno)
diff --git a/cache.h b/cache.h
index ba04ff8bd36..62e5ac335b8 100644
--- a/cache.h
+++ b/cache.h
@@ -879,7 +879,9 @@ int ie_modified(struct index_state *, const struct cache_entry *, struct stat *,
 #define HASH_FORMAT_CHECK 2
 #define HASH_RENORMALIZE  4
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
-int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
+int index_path(struct index_state *istate, struct object_id *oid,
+	       const char *path, struct stat *st, unsigned flags,
+	       int *maybe_errno);
 
 /*
  * Record to sd the data from st that we use to check whether a file
diff --git a/combine-diff.c b/combine-diff.c
index 7d925ce9ce4..7ac9bc6e943 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1059,7 +1059,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			elem->mode = canon_mode(st.st_mode);
 		} else if (S_ISDIR(st.st_mode)) {
 			struct object_id oid;
-			if (resolve_gitlink_ref(elem->path, "HEAD", &oid) < 0)
+			int ignore_errno;
+			if (resolve_gitlink_ref(elem->path, "HEAD", &oid, &ignore_errno) < 0)
 				result = grab_blob(opt->repo, &elem->oid,
 						   elem->mode, &result_size,
 						   NULL, NULL);
diff --git a/diff-lib.c b/diff-lib.c
index c2ac9250fe9..4ae27851cb9 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -40,6 +40,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en
 		return 1;
 	if (S_ISDIR(st->st_mode)) {
 		struct object_id sub;
+		int ignore_errno;
 
 		/*
 		 * If ce is already a gitlink, we can have a plain
@@ -53,7 +54,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en
 		 * a directory --- the blob was removed!
 		 */
 		if (!S_ISGITLINK(ce->ce_mode) &&
-		    resolve_gitlink_ref(ce->name, "HEAD", &sub))
+		    resolve_gitlink_ref(ce->name, "HEAD", &sub, &ignore_errno))
 			return 1;
 	}
 	return 0;
diff --git a/diff.c b/diff.c
index 52c791574b7..972ad0e57de 100644
--- a/diff.c
+++ b/diff.c
@@ -4426,6 +4426,7 @@ static void diff_fill_oid_info(struct diff_filespec *one, struct index_state *is
 {
 	if (DIFF_FILE_VALID(one)) {
 		if (!one->oid_valid) {
+			int maybe_errno = 0;
 			struct stat st;
 			if (one->is_stdin) {
 				oidclr(&one->oid);
@@ -4433,8 +4434,14 @@ static void diff_fill_oid_info(struct diff_filespec *one, struct index_state *is
 			}
 			if (lstat(one->path, &st) < 0)
 				die_errno("stat '%s'", one->path);
-			if (index_path(istate, &one->oid, one->path, &st, 0))
-				die("cannot hash %s", one->path);
+			if (index_path(istate, &one->oid, one->path, &st, 0, &maybe_errno)) {
+				if (maybe_errno) {
+					errno = maybe_errno;
+ 					die_errno("cannot hash %s", one->path);
+				} else {
+ 					die("cannot hash %s", one->path);
+				}
+			}
 		}
 	}
 	else
diff --git a/dir.c b/dir.c
index ebe5ec046e0..95eb4ada718 100644
--- a/dir.c
+++ b/dir.c
@@ -2967,9 +2967,10 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	int only_empty = (flag & REMOVE_DIR_EMPTY_ONLY);
 	int keep_toplevel = (flag & REMOVE_DIR_KEEP_TOPLEVEL);
 	struct object_id submodule_head;
+	int ignore_errno;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
-	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head)) {
+	    resolve_gitlink_ref(path->buf, "HEAD", &submodule_head, &ignore_errno) < 0) {
 		/* Do not descend and nuke a nested git work tree. */
 		if (kept_up)
 			*kept_up = 1;
diff --git a/notes-merge.c b/notes-merge.c
index 46c1f7c7f11..f7e0091bd72 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -698,6 +698,7 @@ int notes_merge_commit(struct notes_merge_options *o,
 	while ((e = readdir_skip_dot_and_dotdot(dir)) != NULL) {
 		struct stat st;
 		struct object_id obj_oid, blob_oid;
+		int maybe_errno = 0;
 
 		if (get_oid_hex(e->d_name, &obj_oid)) {
 			if (o->verbosity >= 3)
@@ -710,8 +711,14 @@ int notes_merge_commit(struct notes_merge_options *o,
 		/* write file as blob, and add to partial_tree */
 		if (stat(path.buf, &st))
 			die_errno("Failed to stat '%s'", path.buf);
-		if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT))
-			die("Failed to write blob object from '%s'", path.buf);
+		if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT, &maybe_errno)) {
+			if (maybe_errno) {
+				errno = maybe_errno;
+				die_errno("Failed to write blob object from '%s'", path.buf);
+			} else {
+				die("Failed to write blob object from '%s'", path.buf);
+			}
+		}
 		if (add_note(partial_tree, &obj_oid, &blob_oid, NULL))
 			die("Failed to add resolved note '%s' to notes tree",
 			    path.buf);
diff --git a/object-file.c b/object-file.c
index f233b440b22..0030854dcdf 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2260,7 +2260,8 @@ int index_fd(struct index_state *istate, struct object_id *oid,
 }
 
 int index_path(struct index_state *istate, struct object_id *oid,
-	       const char *path, struct stat *st, unsigned flags)
+	       const char *path, struct stat *st, unsigned flags,
+	       int *maybe_errno)
 {
 	int fd;
 	struct strbuf sb = STRBUF_INIT;
@@ -2286,7 +2287,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 		strbuf_release(&sb);
 		break;
 	case S_IFDIR:
-		return resolve_gitlink_ref(path, "HEAD", oid);
+		return resolve_gitlink_ref(path, "HEAD", oid, maybe_errno);
 	default:
 		return error(_("%s: unsupported file type"), path);
 	}
diff --git a/read-cache.c b/read-cache.c
index 77961a38854..a40a326587f 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -256,6 +256,7 @@ static int ce_compare_link(const struct cache_entry *ce, size_t expected_size)
 static int ce_compare_gitlink(const struct cache_entry *ce)
 {
 	struct object_id oid;
+	int ignore_errno;
 
 	/*
 	 * We don't actually require that the .git directory
@@ -265,7 +266,7 @@ static int ce_compare_gitlink(const struct cache_entry *ce)
 	 *
 	 * If so, we consider it always to match.
 	 */
-	if (resolve_gitlink_ref(ce->name, "HEAD", &oid) < 0)
+	if (resolve_gitlink_ref(ce->name, "HEAD", &oid, &ignore_errno) < 0)
 		return 0;
 	return !oideq(&oid, &ce->oid);
 }
@@ -748,8 +749,13 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 
 	namelen = strlen(path);
 	if (S_ISDIR(st_mode)) {
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
-			return error(_("'%s' does not have a commit checked out"), path);
+		int their_errno = 0;
+		if (resolve_gitlink_ref(path, "HEAD", &oid, &their_errno) < 0) {
+			if (their_errno)
+				return error_errno(_("'%s' does not have a commit checked out"), path);
+			else
+				return error(_("'%s' does not have a commit checked out"), path);
+		}
 		while (namelen && path[namelen-1] == '/')
 			namelen--;
 	}
@@ -799,9 +805,16 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 		}
 	}
 	if (!intent_only) {
-		if (index_path(istate, &ce->oid, path, st, hash_flags)) {
+		int maybe_errno = 0;
+		if (index_path(istate, &ce->oid, path, st, hash_flags,
+			       &maybe_errno)) {
 			discard_cache_entry(ce);
-			return error(_("unable to index file '%s'"), path);
+			if (maybe_errno) {
+				errno = maybe_errno;
+				return error_errno(_("unable to index file '%s'"), path);
+			} else {
+				return error(_("unable to index file '%s'"), path);
+			}
 		}
 	} else
 		set_object_name_for_intent_to_add_entry(ce);
diff --git a/refs.c b/refs.c
index 92c4796078b..082b6b3d133 100644
--- a/refs.c
+++ b/refs.c
@@ -1807,7 +1807,7 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 }
 
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid)
+			struct object_id *oid, int *maybe_errno)
 {
 	struct ref_store *refs;
 	int flags;
@@ -1817,7 +1817,8 @@ int resolve_gitlink_ref(const char *submodule, const char *refname,
 	if (!refs)
 		return -1;
 
-	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
+	if (!refs_resolve_ref_unsafe_with_errno(refs, refname, 0, oid, &flags,
+		    maybe_errno) ||
 	    is_null_oid(oid))
 		return -1;
 	return 0;
diff --git a/refs.h b/refs.h
index 48970dfc7e0..b0505b35698 100644
--- a/refs.h
+++ b/refs.h
@@ -133,10 +133,16 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled);
  * Resolve refname in the nested "gitlink" repository in the specified
  * submodule (which must be non-NULL). If the resolution is
  * successful, return 0 and set oid to the name of the object;
- * otherwise, return a non-zero value.
+ * otherwise, return negative value.
+ *
+ * If we encounter an error we may have an errno to report from
+ * refs_resolve_ref_unsafe_with_errno(), or we might have hit another
+ * error. Set your `maybe_errno` to 0 beforehand, and check it after
+ * an error where we return a negative value to see if you should
+ * report the errno as well.
  */
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid);
+			struct object_id *oid, int *maybe_errno);
 
 /*
  * Return true iff abbrev_name is a possible abbreviation for
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 01c9bd0dbf0..9ed58c40de6 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -870,6 +870,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,
 			       &lock->old_oid, NULL)) {
 		if (old_oid) {
 			int save_errno = errno;
+			BUG("got here");
 			strbuf_addf(err, "can't verify ref '%s'", lock->ref_name);
 			errno = save_errno;
 			return -1;
diff --git a/unpack-trees.c b/unpack-trees.c
index f88a69f8e71..dd90787c56c 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1986,7 +1986,9 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 
 	if (S_ISGITLINK(ce->ce_mode)) {
 		struct object_id oid;
-		int sub_head = resolve_gitlink_ref(ce->name, "HEAD", &oid);
+		int ignore_errno;
+		int sub_head = resolve_gitlink_ref(ce->name, "HEAD", &oid,
+						   &ignore_errno);
 		/*
 		 * If we are not going to update the submodule, then
 		 * we don't care.

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-01 11:13     ` Ævar Arnfjörð Bjarmason
@ 2021-07-05 14:16       ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-05 14:16 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Thu, Jul 1, 2021 at 1:30 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >       }
> >       if (!resolved) {
> > -             last_errno = errno;
> > +             int last_errno = errno;
> >               if (last_errno != ENOTDIR ||
> >                   !refs_verify_refname_available(&refs->base, refname,
> >                                                  extras, skip, err))
>
> ...this particular change gives me some pause, because all the rest is
> about squirreling away our own errno for our own caller (which it turns
> out, we didn't need).
>
> But in this case we're only guarding against
> refs_verify_refname_available() possibly clobbering the errno we just
> got on !resolved in refs_verify_refname_available().

This comes from 5b2d8d6f2184381b76c13504a2f5ec8a62cd584e

   .. invoke verify_refname_available() to try
    to generate a more helpful error message.

    That function might not detect an error. For example, some
    non-reference file might be blocking the deletion of an
    otherwise-empty directory tree, or there might be a race with another
    process that just deleted the offending reference. In such cases,
    generate the strerror-based error message like before.

I think we want to keep this behavior, hence I think this should be kept as is.

I'll add a comment.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn
  2021-07-01 11:34     ` Ævar Arnfjörð Bjarmason
@ 2021-07-05 14:34       ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-05 14:34 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Thu, Jul 1, 2021 at 1:55 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > diff --git a/refs.c b/refs.c
> > index 8c9490235ea6..bebe3f584da7 100644
> > --- a/refs.c
> > +++ b/refs.c
> > @@ -1675,13 +1675,17 @@ int refs_read_raw_ref(struct ref_store *ref_store,
> >                     const char *refname, struct object_id *oid,
> >                     struct strbuf *referent, unsigned int *type)
> >  {
> > +     int result, failure;
>
> Style nit: Different variables should be on different lines, except
> where they're the same, so "int i, j", not "int ret, i". Perhaps "ret"
> and "errno" are similar enough, but I'd split them, just my 0.02.
>
> Also nit: s/failure/failure_errno/, just like the function signature.

done.

> >       if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
> >               return refs_read_special_head(ref_store, refname, oid, referent,
> >                                             type);
> >       }
> >
> > -     return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> > -                                        type);
> > +     failure = 0;
> > +     result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> > +                                          type, &failure);
> > +     errno = failure;
> > +     return result;
> >  }
>
> Is there some subtlety here where this isn't equivalent to the more
> simple/straightforward:
>
>         diff --git a/refs.c b/refs.c
>         index bebe3f584da..49ab7555de9 100644
>         --- a/refs.c
>         +++ b/refs.c
>         @@ -1675,17 +1675,14 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>                               const char *refname, struct object_id *oid,
>                               struct strbuf *referent, unsigned int *type)
>          {
>         -       int result, failure;
>                 if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>                         return refs_read_special_head(ref_store, refname, oid, referent,
>                                                       type);
>                 }
>
>         -       failure = 0;
>         -       result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
>         -                                            type, &failure);
>         -       errno = failure;
>         -       return result;
>         +       errno = 0;
>         +       return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
>         +                                            type, &errno);

There is a big difference: in this case, in read_raw_ref, the
failure_errno pointer is aliased to errno. Which means that things
like

   *failure_errno = 0
   some_syscall();

might mysteriously clobber *failure_errno again, which is the kind of
effect-at-a-distance we're trying to get rid of.

> I can only imagine a case where we think files_read_raw_ref() would
> encounter a new errno after it assigned to *failure_errno, which is just
> a couple of strbuf_release() calls.
>
> if that is a case we're worried about then like in my comment on 2/8
> shouldn't we be explicitly checking for such a lost/different errno?
> I.e. something like (should probably be less fatal than BUG(...):
>
>         diff --git a/refs.c b/refs.c
>         index bebe3f584da..9584ddae392 100644
>         --- a/refs.c
>         +++ b/refs.c
>         @@ -1675,16 +1675,22 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>                               const char *refname, struct object_id *oid,
>                               struct strbuf *referent, unsigned int *type)
>          {
>         -       int result, failure;
>         +       int result, failure_errno = 0;
>         +
>                 if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>                         return refs_read_special_head(ref_store, refname, oid, referent,
>                                                       type);
>                 }
>
>         -       failure = 0;
>         +       errno = 0;
>                 result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
>         -                                            type, &failure);
>         -       errno = failure;
>         +                                            type, &failure_errno);
>         +       if (errno)
>         +               BUG("Got another errno from read_raw_ref()?: %s, failure errno: %s",
>         +                   strerror(errno),
>         +                   strerror(failure_errno));

The whole idea is to make sure that important (ie. errno values that
influence downstream logic) are separate from unimportant (ie. used
for error messages). By adding a BUG() here, you're making a drastic
downstream logic change depending on errno,  so your example code is
going precisely in the opposite direction of where I want to go, and
forces implementers of read_raw_ref to walk on eggshells again wrt.
errno handling to avoid triggering error messages.

> > + * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
> > + * and return -1. If the ref exists but is neither a symbolic ref nor an object
> > + * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno
> > + * should not be ENOENT). The files backend may return EISDIR (if the ref name
> > + * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is
> > + * another error reading the ref, set failure_errno appropriately and return -1.
> >   *
>
> This documentation is a bit confusing in light of the above comments. I
> assume that "set failure_errno appropriately" here means that it will do
> it, but it's really worded like it's suggesting that you should do it,
> and does the "another error" suggest that a caller may need to deal with
> an arbitrary errno, not just the ENOENT|EISDIR|ENOTDIR?

Tried to clarify this.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-07-01 12:26     ` Ævar Arnfjörð Bjarmason
@ 2021-07-05 16:09       ` Han-Wen Nienhuys
  2021-07-05 19:08         ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-05 16:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Thu, Jul 1, 2021 at 3:06 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > -     const char *result = refs_resolve_ref_unsafe_implicit_errno(
> > -             refs, refname, resolve_flags, oid, flags);
> > -     *failure_errno = errno;
> > -     return result;
> > +     int ignore;
> > +     return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
> > +                                               oid, flags, &ignore);
> >  }
>
> Okey, so here we have something like the "caller should explicitly
> ignore it" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no
> longer set "errno" at all for the benefit of the callers of that
> function, but instead explicitly throw it away, our
> refs_resolve_ref_unsafe_with_errno() no longer sets errno.
>
> But this end-state seems to have resulted in introducing new bugs. A
> bunch of functions are thin wrappers for
> refs_resolve_ref_unsafe(). Previously they could inspect errno on a -1
> return, now they can't.
>
> I didn't look at them all, but just the second one I looked at,
> refs_read_ref_full() has a verify_lock() caller which calls it, and that

It looks like you were lucky. I looked at the output of

egrep -A7 -B7 --color -nH -e
'(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\('
*.[ch] */*.[ch]

and the case you pointed out is the only one which inspects errno
after calling resolve_ref_unsafe (transitively through the grepped
functions.)

> function then expects a meaningful errno that it in turn ferries up with
> "can't verify ref". It in turn is called by lock_ref_oid_basic(), it
> ferries up that strbuf with the error, which'll now have a meaningless
> strerror(0) in it.
>
> So it seems to me that the refactoring being done here is halfway done,
> [..]
> of what you're starting here. I.e. we have a low-level function that may
> encounter an errno, and then callers removed from that via:
>
>     index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()
>..
> Where the common case for index_path() is to report the error we got,
> but it doesn't call die_errno() or error_errno() now, but really should,

"but really should": Why?

Why should all functions that resolve a ref print out system level
errors? For example, that would entail large changes to sequencer.c
which does "error" == "does not exist".

Also, the errors you print out will be fundamentally limited. An error
in resolving HEAD could be caused by a problem in any of the following
files

   .git/HEAD
   .git/packed-refs
   .git/refs/
   .git/refs/heads/
   .git/refs/heads/main

Since we have no way of reporting the filename involved, what is a
user going to do with the error message ("file doesn't exist")? I
think refs_verify_refname_available() provides a more realistic
approach: create custom error messages for situations that a user is
more likely to encounter.

Also, for something like the reftable backend, there are error classes
that have no direct errno counterpart. For example, if reftable finds
corrupt zlib data, should it invent a bogus errno code so it can work
with the errno reporting system?

I'll take from this review that I should elide the part where errno is
cleared, and leave it to someone else to figure out a more holistic
strategy to error reporting.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-07-05 16:09       ` Han-Wen Nienhuys
@ 2021-07-05 19:08         ` Ævar Arnfjörð Bjarmason
  2021-07-05 19:39           ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-05 19:08 UTC (permalink / raw)
  To: Han-Wen Nienhuys
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys


On Mon, Jul 05 2021, Han-Wen Nienhuys wrote:

> On Thu, Jul 1, 2021 at 3:06 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> > -     const char *result = refs_resolve_ref_unsafe_implicit_errno(
>> > -             refs, refname, resolve_flags, oid, flags);
>> > -     *failure_errno = errno;
>> > -     return result;
>> > +     int ignore;
>> > +     return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
>> > +                                               oid, flags, &ignore);
>> >  }
>>
>> Okey, so here we have something like the "caller should explicitly
>> ignore it" I suggested in <878s2qdy8y.fsf@evledraar.gmail.com>. We no
>> longer set "errno" at all for the benefit of the callers of that
>> function, but instead explicitly throw it away, our
>> refs_resolve_ref_unsafe_with_errno() no longer sets errno.
>>
>> But this end-state seems to have resulted in introducing new bugs. A
>> bunch of functions are thin wrappers for
>> refs_resolve_ref_unsafe(). Previously they could inspect errno on a -1
>> return, now they can't.
>>
>> I didn't look at them all, but just the second one I looked at,
>> refs_read_ref_full() has a verify_lock() caller which calls it, and that
>
> It looks like you were lucky. I looked at the output of

Maybe so...

> egrep -A7 -B7 --color -nH -e
> '(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\('
> *.[ch] */*.[ch]

FWIW this gives better results:

    git grep -E -W '\b(refs_resolve_refdup|resolve_refdup|refs_read_ref_full|read_ref_full|read_ref|refs_ref_exists|ref_exists|resolve_gitlink_ref|index_path)\('

> and the case you pointed out is the only one which inspects errno
> after calling resolve_ref_unsafe (transitively through the grepped
> functions.)

Isn't the `errno == EISDIR` in files_copy_or_rename_ref() another one?
It's just after calling refs_read_ref_full().

Yes, but aside from that it does look like I got lucky. I assumed this
issue was more widespread. Phew!

>> function then expects a meaningful errno that it in turn ferries up with
>> "can't verify ref". It in turn is called by lock_ref_oid_basic(), it
>> ferries up that strbuf with the error, which'll now have a meaningless
>> strerror(0) in it.
>>
>> So it seems to me that the refactoring being done here is halfway done,
>> [..]
>> of what you're starting here. I.e. we have a low-level function that may
>> encounter an errno, and then callers removed from that via:
>>
>>     index_path() -> resolve_gitlink_ref() -> refs_resolve_ref_unsafe()
>>..
>> Where the common case for index_path() is to report the error we got,
>> but it doesn't call die_errno() or error_errno() now, but really should,
>
> "but really should": Why?
>
> Why should all functions that resolve a ref print out system level
> errors? For example, that would entail large changes to sequencer.c
> which does "error" == "does not exist".
>
> Also, the errors you print out will be fundamentally limited. An error
> in resolving HEAD could be caused by a problem in any of the following
> files
>
>    .git/HEAD
>    .git/packed-refs
>    .git/refs/
>    .git/refs/heads/
>    .git/refs/heads/main
>
> Since we have no way of reporting the filename involved, what is a
> user going to do with the error message ("file doesn't exist")? I
> think refs_verify_refname_available() provides a more realistic
> approach: create custom error messages for situations that a user is
> more likely to encounter.

Let me rephrase: "we really shouldn't", as in I agree that this API
sucks and that refs_verify_refname_available() is a much better
approach.

But as long as errno was the abstraction we were passing around it
seemed like e.g. callers like process_directory() should be getting the
errno and using the *_errno() error functions, but of course it would be
much better if they used a better error return flow.

But having looked at the grep above with fresh eyes I think it's fine to
leave it at the refs_resolve_ref_unsafe_with_errno() boundary. The error
messages are clear enough, and we're trying to phase out errno anyway.

I was more paranoid about things that would actually do equality checks
against the errno, as it appearsfiles_copy_or_rename_ref() is still
doing.

> Also, for something like the reftable backend, there are error classes
> that have no direct errno counterpart. For example, if reftable finds
> corrupt zlib data, should it invent a bogus errno code so it can work
> with the errno reporting system?

No, that would be crazy. We should be moving away from errno
entirely. My concern is only that we might be introducing subtle bugs
further up the callstack now that we clear "errno" ...

> I'll take from this review that I should elide the part where errno is
> cleared, and leave it to someone else to figure out a more holistic
> strategy to error reporting.

...or if we don't clear errno introducing those sorts of subtle bugs
when we use reftable instead of the files backend. I.e. no, we really
should be clearing errno, if not in this series then in some other
pre-reftable series.

To not do so would be kicking this particular can down the road, and
leaving those bugs for reftable users to find. Which given that I've
found a few cases with no test coverage...

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref
  2021-07-05 19:08         ` Ævar Arnfjörð Bjarmason
@ 2021-07-05 19:39           ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-05 19:39 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Mon, Jul 5, 2021 at 9:31 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > and the case you pointed out is the only one which inspects errno
> > after calling resolve_ref_unsafe (transitively through the grepped
> > functions.)
>
> Isn't the `errno == EISDIR` in files_copy_or_rename_ref() another one?
> It's just after calling refs_read_ref_full().

no. It's handling the EISDIR for refs_delete_ref.

> > I'll take from this review that I should elide the part where errno is
> > cleared, and leave it to someone else to figure out a more holistic
> > strategy to error reporting.
>
> ...or if we don't clear errno introducing those sorts of subtle bugs
> when we use reftable instead of the files backend. I.e. no, we really
> should be clearing errno, if not in this series then in some other
> pre-reftable series.
>
> To not do so would be kicking this particular can down the road, and
> leaving those bugs for reftable users to find. Which given that I've
> found a few cases with no test coverage...

too late :-)

I've already kicked the can down the road by not clearing errno in the
current version of the series.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v3 0/5] refs: cleanup errno sideband ref related functions
  2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
                     ` (8 preceding siblings ...)
  2021-06-14 10:10   ` [PATCH v2 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys
@ 2021-07-05 20:56   ` Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                       ` (6 more replies)
  9 siblings, 7 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

v5

 * address Ævar's comment; punt on clearing errno.

Han-Wen Nienhuys (5):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: make errno output explicit for refs_resolve_ref_unsafe

 refs.c                | 52 ++++++++++++++++++++++++++-----------
 refs/debug.c          |  4 +--
 refs/files-backend.c  | 60 ++++++++++++++++++++-----------------------
 refs/packed-backend.c | 15 ++++++-----
 refs/refs-internal.h  | 34 ++++++++++++++++--------
 5 files changed, 98 insertions(+), 67 deletions(-)


base-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v3
Pull-Request: https://github.com/git/git/pull/1012

Range-diff vs v2:

 1:  f9b92e62b59 = 1:  e2a0e5387ab refs: remove EINVAL errno output from specification of read_raw_ref_fn
 2:  cbe09a48036 ! 2:  c594c9c5c67 refs/files-backend: stop setting errno from lock_ref_oid_basic
     @@ Commit message
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
          Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      -		last_errno = errno;
      +		int last_errno = errno;
       		if (last_errno != ENOTDIR ||
     - 		    !refs_verify_refname_available(&refs->base, refname,
     - 						   extras, skip, err))
     +-		    !refs_verify_refname_available(&refs->base, refname,
     +-						   extras, skip, err))
     ++		    /* in case of D/F conflict, try to generate a better error
     ++		     * message. If that fails, fall back to strerror(ENOTDIR).
     ++		     */
     ++		    !refs_verify_refname_available(&refs->base, refname, extras,
     ++						   skip, err))
     + 			strbuf_addf(err, "unable to resolve reference '%s': %s",
     + 				    refname, strerror(last_errno));
     + 
      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
       	if (is_null_oid(&lock->old_oid) &&
       	    refs_verify_refname_available(refs->packed_ref_store, refname,
 3:  3e2831e59c8 ! 3:  b017caf54ba refs: make errno output explicit for read_raw_ref_fn
     @@ Metadata
       ## Commit message ##
          refs: make errno output explicit for read_raw_ref_fn
      
     +    This makes it explicit how alternative ref backends should report errors in
     +    read_raw_ref_fn.
     +
          read_raw_ref_fn needs to supply a credible errno for a number of cases. These
          are primarily:
      
     @@ Commit message
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
          Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
       		      const char *refname, struct object_id *oid,
       		      struct strbuf *referent, unsigned int *type)
       {
     -+	int result, failure;
     ++	int result;
     ++	int failure_errno;
       	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
       		return refs_read_special_head(ref_store, refname, oid, referent,
       					      type);
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
       
      -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
      -					   type);
     -+	failure = 0;
     ++	failure_errno = 0;
      +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
     -+					     type, &failure);
     -+	errno = failure;
     ++					     type, &failure_errno);
     ++	if (failure_errno)
     ++		errno = failure_errno;
      +	return result;
       }
       
     @@ refs/files-backend.c: stat_ref:
       
       out:
      -	save_errno = errno;
     -+	if (failure_errno)
     -+		*failure_errno = errno;
     ++	*failure_errno = errno;
       	strbuf_release(&sb_path);
       	strbuf_release(&sb_contents);
      -	errno = save_errno;
     @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,
      - * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
      - * (errno should not be ENOENT) If there is another error reading the
      - * ref, set errno appropriately and return -1.
     -+ * Return 0 on success. If the ref doesn't exist, set failure_errno to ENOENT
     -+ * and return -1. If the ref exists but is neither a symbolic ref nor an object
     -+ * ID, it is broken; set REF_ISBROKEN in type, and return -1 (failure_errno
     -+ * should not be ENOENT). The files backend may return EISDIR (if the ref name
     -+ * is a directory) and ENOTDIR (if a ref prefix is not a directory). If there is
     -+ * another error reading the ref, set failure_errno appropriately and return -1.
     ++ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
     ++ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
     ++ * type, and return -1 (failure_errno should not be ENOENT)
     ++ *
     ++ * failure_errno provides errno codes that are interpreted beyond error
     ++ * reporting. The following error codes have special meaning:
     ++ *    * ENOENT: the ref doesn't exist
     ++ *    * EISDIR: ref name is a directory
     ++ *    * ENOTDIR: ref prefix is not a directory
        *
        * Backend-specific flags might be set in type as well, regardless of
        * outcome.
 4:  11b2184044d < -:  ----------- refs: make errno output explicit for refs_resolve_ref_unsafe
 6:  2b346caf1ae ! 4:  4aaa9d3bd6f refs: add failure_errno to refs_read_raw_ref() signature
     @@ Metadata
       ## Commit message ##
          refs: add failure_errno to refs_read_raw_ref() signature
      
     -    This makes the errno output of refs_read_raw_ref explicit.
     +    This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
          Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: done:
     @@ refs.c: done:
      +		      struct object_id *oid, struct strbuf *referent,
      +		      unsigned int *type, int *failure_errno)
       {
     --	int result, failure;
     +-	int result;
     +-	int failure_errno;
     ++	int ignore;
      +	if (failure_errno)
      +		*failure_errno = 0;
     ++	else
     ++		failure_errno = &ignore;
     ++
       	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
       		return refs_read_special_head(ref_store, refname, oid, referent,
       					      type);
       	}
       
     --	failure = 0;
     +-	failure_errno = 0;
      -	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
     --					     type, &failure);
     --	errno = failure;
     +-					     type, &failure_errno);
     +-	if (failure_errno)
     +-		errno = failure_errno;
      -	return result;
      +	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
      +					   type, failure_errno);
       }
       
     - /* This function needs to return a meaningful errno on failure */
     +-/* This function needs to return a meaningful errno on failure */
     +-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     +-				    const char *refname,
     +-				    int resolve_flags,
     +-				    struct object_id *oid, int *flags)
     ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
     ++					       const char *refname,
     ++					       int resolve_flags,
     ++					       struct object_id *oid,
     ++					       int *flags, int *failure_errno)
     + {
     + 	static struct strbuf sb_refname = STRBUF_INIT;
     + 	struct object_id unused_oid;
     + 	int unused_flags;
     ++	int unused_errno;
     + 	int symref_count;
     + 
     + 	if (!oid)
     +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     + 		flags = &unused_flags;
     + 
     + 	*flags = 0;
     ++	if (!failure_errno)
     ++		failure_errno = &unused_errno;
     ++	*failure_errno = 0;
     + 
     + 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
     + 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
      @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
       
       	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
      +				      &read_flags, &read_failure)) {
       			*flags |= read_flags;
       
     ++			*failure_errno = read_failure;
     ++
       			/* In reading mode, refs must eventually resolve */
     + 			if (resolve_flags & RESOLVE_REF_READING)
     + 				return NULL;
      @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
       			 * may show errors besides ENOENT if there are
       			 * similarly-named refs.
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
       				return NULL;
       
       			oidclr(oid);
     +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     + 	return NULL;
     + }
     + 
     ++const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
     ++				    int resolve_flags, struct object_id *oid,
     ++				    int *flags)
     ++{
     ++	int ignore = 0;
     ++	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
     ++						  oid, flags, &ignore);
     ++}
     ++
     + /* backend functions */
     + int refs_init_db(struct strbuf *err)
     + {
      @@ refs.c: int refs_verify_refname_available(struct ref_store *refs,
       		if (skip && string_list_has_string(skip, dirname.buf))
       			continue;
     @@ refs/files-backend.c: stat_ref:
       		}
      
       ## refs/packed-backend.c ##
     -@@ refs/packed-backend.c: static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
     - 
     - 	if (!rec) {
     - 		/* refname is not a packed reference. */
     --		*failure_errno = ENOENT;
     -+		if (failure_errno)
     -+			*failure_errno = ENOENT;
     - 		return -1;
     - 	}
     - 
      @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_store,
       	ret = 0;
       	for (i = 0; i < transaction->nr; i++) {
       		struct ref_update *update = transaction->updates[i];
     -+		int failure;
     ++		int failure_errno = 0;
       		unsigned int type;
       		struct object_id oid;
       
     @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_st
      -				       &oid, &referent, &type) ||
      -		    errno != ENOENT) {
      +		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
     -+				       &referent, &type, &failure) ||
     -+		    failure != ENOENT) {
     ++				       &referent, &type, &failure_errno) ||
     ++		    failure_errno != ENOENT) {
       			/*
       			 * We have to actually delete that reference
       			 * -> this transaction is needed.
     @@ refs/refs-internal.h: struct ref_update {
      +		      struct object_id *oid, struct strbuf *referent,
      +		      unsigned int *type, int *failure_errno);
       
     - /* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
     -  * failure. */
     + /*
     +  * Write an error to `err` and return a nonzero value iff the same
 5:  005ee8e6fb2 ! 5:  ed5347d7bb6 refs: use refs_resolve_ref_unsafe_with_errno() where needed
     @@ Metadata
      Author: Han-Wen Nienhuys <hanwen@google.com>
      
       ## Commit message ##
     -    refs: use refs_resolve_ref_unsafe_with_errno() where needed
     +    refs: make errno output explicit for refs_resolve_ref_unsafe
     +
     +    This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
     +    contract for the errno output explicit. The implementation still relies on
     +    the global errno variable to ensure no side effects of this refactoring.
      
          lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
     -    that needs error information to make logic decisions.
     +    that needs error information to make logic decisions, so update that caller
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
          Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
       	if (!resolved) {
      -		int last_errno = errno;
      -		if (last_errno != ENOTDIR ||
     --		    !refs_verify_refname_available(&refs->base, refname,
     --						   extras, skip, err))
      +		if (resolve_errno != ENOTDIR ||
     -+		    !refs_verify_refname_available(&refs->base, refname, extras,
     -+						   skip, err))
     + 		    /* in case of D/F conflict, try to generate a better error
     + 		     * message. If that fails, fall back to strerror(ENOTDIR).
     + 		     */
     + 		    !refs_verify_refname_available(&refs->base, refname, extras,
     + 						   skip, err))
       			strbuf_addf(err, "unable to resolve reference '%s': %s",
      -				    refname, strerror(last_errno));
      +				    refname, strerror(resolve_errno));
       
       		goto error_return;
       	}
     +
     + ## refs/refs-internal.h ##
     +@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     + 		      struct object_id *oid, struct strbuf *referent,
     + 		      unsigned int *type, int *failure_errno);
     + 
     ++/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
     ++ * failure. */
     ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
     ++					       const char *refname,
     ++					       int resolve_flags,
     ++					       struct object_id *oid,
     ++					       int *flags, int *failure_errno);
     ++
     + /*
     +  * Write an error to `err` and return a nonzero value iff the same
     +  * refname appears multiple times in `refnames`. `refnames` must be
 7:  d8651621968 < -:  ----------- refs: clear errno return in refs_resolve_ref_unsafe()
 8:  2a9ebe43dea < -:  ----------- refs: explicitly propagate errno from refs_read_raw_ref

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
@ 2021-07-05 20:56     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 2/5] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
                       ` (5 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936..f4445e32904 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v3 2/5] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-05 20:56     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 3/5] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                       ` (4 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 677b7e4cdd2..83ddfb3b627 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		last_errno = errno;
+		int last_errno = errno;
 		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
+		    /* in case of D/F conflict, try to generate a better error
+		     * message. If that fails, fall back to strerror(ENOTDIR).
+		     */
+		    !refs_verify_refname_available(&refs->base, refname, extras,
+						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
 				    refname, strerror(last_errno));
 
@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v3 3/5] refs: make errno output explicit for read_raw_ref_fn
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 2/5] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-07-05 20:56     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-05 20:56     ` [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
                       ` (3 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 10 ++++++++--
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 23 +++++++++++------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 37 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235ea..5e5e3af8da0 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
+	int result;
+	int failure_errno;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+	failure_errno = 0;
+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					     type, &failure_errno);
+	if (failure_errno)
+		errno = failure_errno;
+	return result;
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc34..f12413a9bc0 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 83ddfb3b627..f98b1be9386 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ stat_ref:
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno = 0;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -629,9 +628,9 @@ retry:
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +654,7 @@ retry:
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +692,13 @@ retry:
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db6..a457f18e93c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e32904..79dfb3af484 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
                       ` (2 preceding siblings ...)
  2021-07-05 20:56     ` [PATCH v3 3/5] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-05 20:56     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 19:28       ` Junio C Hamano
  2021-07-05 20:56     ` [PATCH v3 5/5] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
                       ` (2 subsequent siblings)
  6 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 60 +++++++++++++++++++++++++++----------------
 refs/files-backend.c  |  8 +++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 49 insertions(+), 32 deletions(-)

diff --git a/refs.c b/refs.c
index 5e5e3af8da0..25d80e544d0 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,34 +1671,35 @@ done:
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
-	int result;
-	int failure_errno;
+	int ignore;
+	if (failure_errno)
+		*failure_errno = 0;
+	else
+		failure_errno = &ignore;
+
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	failure_errno = 0;
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure_errno);
-	if (failure_errno)
-		errno = failure_errno;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
+	int unused_errno;
 	int symref_count;
 
 	if (!oid)
@@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		flags = &unused_flags;
 
 	*flags = 0;
+	if (!failure_errno)
+		failure_errno = &unused_errno;
+	*failure_errno = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
@@ -1728,11 +1732,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
+		int read_failure = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
+			*failure_errno = read_failure;
+
 			/* In reading mode, refs must eventually resolve */
 			if (resolve_flags & RESOLVE_REF_READING)
 				return NULL;
@@ -1742,9 +1749,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (read_failure != ENOENT && read_failure != EISDIR &&
+			    read_failure != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1783,6 +1789,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int ignore = 0;
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore);
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2244,7 +2259,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, NULL)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f98b1be9386..f38c9703504 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -383,8 +383,8 @@ stat_ref:
 	if (lstat(path, &st) < 0) {
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -423,8 +423,8 @@ stat_ref:
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c..530dba29788 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno = 0;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af484..54f57c6a2df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v3 5/5] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
                       ` (3 preceding siblings ...)
  2021-07-05 20:56     ` [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-05 20:56     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06  0:38     ` [PATCH v3 0/5] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-05 20:56 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 15 ++++++++-------
 refs/refs-internal.h |  8 ++++++++
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index f38c9703504..8506c8b3bde 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -923,6 +923,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -935,10 +936,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -958,15 +960,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
+		if (resolve_errno != ENOTDIR ||
 		    /* in case of D/F conflict, try to generate a better error
 		     * message. If that fails, fall back to strerror(ENOTDIR).
 		     */
 		    !refs_verify_refname_available(&refs->base, refname, extras,
 						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2df..c52a64b081b 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		      struct object_id *oid, struct strbuf *referent,
 		      unsigned int *type, int *failure_errno);
 
+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
+ * failure. */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
+
 /*
  * Write an error to `err` and return a nonzero value iff the same
  * refname appears multiple times in `refnames`. `refnames` must be
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
                       ` (4 preceding siblings ...)
  2021-07-05 20:56     ` [PATCH v3 5/5] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06  0:38     ` Ævar Arnfjörð Bjarmason
  2021-07-06  9:53       ` Han-Wen Nienhuys
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
  6 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-06  0:38 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Han-Wen Nienhuys, Jonathan Tan, Han-Wen Nienhuys


On Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> v5

v3?

>  * address Ævar's comment; punt on clearing errno.
> [...]
>       
>           Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
>           Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
>      +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

FWIW per Documentation/SubmittingPatches:
    
    . `Reviewed-by:`, unlike the other tags, can only be offered by the
      reviewer and means that she is completely satisfied that the patch
      is ready for application.  It is usually offered only after a
      detailed review.

It's not that I'm hard to please, but I can honestly say that I don't
quite understand some parts of what you're gonig for, so that trailer is
probably premature :)

>        ## refs/files-backend.c ##
>       @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
>      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
>       -		last_errno = errno;
>       +		int last_errno = errno;
>        		if (last_errno != ENOTDIR ||
>      - 		    !refs_verify_refname_available(&refs->base, refname,
>      - 						   extras, skip, err))
>      +-		    !refs_verify_refname_available(&refs->base, refname,
>      +-						   extras, skip, err))
>      ++		    /* in case of D/F conflict, try to generate a better error
>      ++		     * message. If that fails, fall back to strerror(ENOTDIR).
>      ++		     */
>      ++		    !refs_verify_refname_available(&refs->base, refname, extras,
>      ++						   skip, err))
>      + 			strbuf_addf(err, "unable to resolve reference '%s': %s",
>      + 				    refname, strerror(last_errno));
>      + 
>       @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>        	if (is_null_oid(&lock->old_oid) &&
>        	    refs_verify_refname_available(refs->packed_ref_store, refname,
>  3:  3e2831e59c8 ! 3:  b017caf54ba refs: make errno output explicit for read_raw_ref_fn
>      @@ Metadata
>        ## Commit message ##
>           refs: make errno output explicit for read_raw_ref_fn
>       
>      +    This makes it explicit how alternative ref backends should report errors in
>      +    read_raw_ref_fn.
>      +
>           read_raw_ref_fn needs to supply a credible errno for a number of cases. These
>           are primarily:
>       
>      @@ Commit message
>       
>           Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
>           Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
>      +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>       
>        ## refs.c ##
>       @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
>        		      const char *refname, struct object_id *oid,
>        		      struct strbuf *referent, unsigned int *type)
>        {
>      -+	int result, failure;
>      ++	int result;
>      ++	int failure_errno;
>        	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>        		return refs_read_special_head(ref_store, refname, oid, referent,
>        					      type);
>      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
>        
>       -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
>       -					   type);
>      -+	failure = 0;
>      ++	failure_errno = 0;
>       +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
>      -+					     type, &failure);
>      -+	errno = failure;
>      ++					     type, &failure_errno);
>      ++	if (failure_errno)
>      ++		errno = failure_errno;
>       +	return result;
>        }

To rephrase my comment on the v2 to hopefully better get at the
point/question I had.

It wasn't that I don't get why you wouldn't save/restore errno in
general.

It's that the pattern of doing so seems backwards to me. I.e. surely the
goal here should be to one function at a time, and from the bottom-up,
figure out where we rely on "errno" and convert that to a
"failure_errno".

Instead not even files_read_raw_ref() resets "errno = 0" at the end, so
the errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.

I'm all for clearing it as mentioned in another mail, surely that should
be the point of this whole thing, i.e. to refactor this part of the API
so that we're assured that nothing upstream of us relies on errno & prep
things for reftable (which won't set it at all, except to fake it).

Having dug a bit further, it seems what you're doing, whether it's
intentional or not, is relying on the parse_loose_ref_contents() setting
EINVAL, but you clobber your *failure_errno with it whether it returned
-1 or not.

Seemingly to make that logic work in files_read_raw_ref() you, after
getting an errno, assigned it "errno", instead of to a
saved_errno". Thus requiring an inverse of the pattern where you need a
"saved_errno" dance to save away an errno you explicitly don't care
about (because you didn't save the one you wanted earlier).

I think something like the following diff on top, this whole dance was
only needed because you didn't pass the failure_errno down to
refs_read_special_head() and parse_loose_ref_contents().

diff --git a/refs.c b/refs.c
index 25d80e544d0..eca7310e7a4 100644
--- a/refs.c
+++ b/refs.c
@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 8506c8b3bde..823325cc97f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,11 +381,12 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
-		if (errno != ENOENT)
+		*failure_errno = errno;
+		if (*failure_errno != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, NULL)) {
-			errno = ENOENT;
+			*failure_errno = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -396,7 +397,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			*failure_errno = errno;
+			if (*failure_errno == ENOENT || *failure_errno == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -425,7 +427,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, NULL)) {
-			errno = EISDIR;
+			*failure_errno = errno = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -438,7 +440,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		if (*failure_errno == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -446,26 +448,25 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
 
 out:
-	*failure_errno = errno;
+	errno = 0; /* saved in *failure_errno */
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -484,7 +485,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index c52a64b081b..530999f50df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -700,7 +700,8 @@ struct ref_store {
  * Parse contents of a loose ref file.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions
  2021-07-06  0:38     ` [PATCH v3 0/5] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-06  9:53       ` Han-Wen Nienhuys
  2021-07-06 14:27         ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-06  9:53 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Tue, Jul 6, 2021 at 2:48 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
>
> On Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:
>
> > v5
>
> v3?

I erroneously closed out the github PR I used for the first 2 versions
of this change series.  (https://github.com/git/git/pull/1011)

> >  * address Ævar's comment; punt on clearing errno.
> > [...]
> >
> >           Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> >           Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
> >      +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> FWIW per Documentation/SubmittingPatches:
>
>     . `Reviewed-by:`, unlike the other tags, can only be offered by the
>       reviewer and means that she is completely satisfied that the patch
>       is ready for application.  It is usually offered only after a
>       detailed review.
>
> It's not that I'm hard to please, but I can honestly say that I don't
> quite understand some parts of what you're gonig for, so that trailer is
> probably premature :)

oh, ok. Removed them.

Does that mean you'd have to repost my patchseries just to add the
reviewed-by header?

> To rephrase my comment on the v2 to hopefully better get at the
> point/question I had.
>
> It wasn't that I don't get why you wouldn't save/restore errno in
> general.
>
> It's that the pattern of doing so seems backwards to me. I.e. surely the
> goal here should be to one function at a time, and from the bottom-up,
> figure out where we rely on "errno" and convert that to a
> "failure_errno".
>
> Instead not even files_read_raw_ref() resets "errno = 0" at the end, so
> the errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.

I would really want errno to be an explicit output for more functions
in refs.h, but I don't have the time to see that through, and it's a
distraction from the reftable work, so I have changed the objective of
the patch series.

I'm limiting myself to explicitly propagating errno values that are
used beyond error reporting. The EINVAL value for
parse_loose_ref_contents is a borderline case: it has an extra "if ()"
body in lock_raw_ref, but it's for an error message.

This means that we should avoid clearing errno, because we'd otherwise
have to track down all the places where it's being put into
strerror(). The one case you found earlier is still there, and by not
clearing errno, I can keep this series smaller by not also having to
rework verify_lock/read_refs_full.

In general, it's frustrating to observe that the files backend is a
complex beast that nobody understands, but in order to replace it with
something more principled, I have to spend lots of time cleaning it
up.

> Having dug a bit further, it seems what you're doing, whether it's
> intentional or not, is relying on the parse_loose_ref_contents() setting
> EINVAL, but you clobber your *failure_errno with it whether it returned
> -1 or not.

I've added a failure_errno argument parse_loose_ref_contents now.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions
  2021-07-06  9:53       ` Han-Wen Nienhuys
@ 2021-07-06 14:27         ` Ævar Arnfjörð Bjarmason
  2021-07-06 18:36           ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-06 14:27 UTC (permalink / raw)
  To: Han-Wen Nienhuys
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys


On Tue, Jul 06 2021, Han-Wen Nienhuys wrote:

> On Tue, Jul 6, 2021 at 2:48 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>>
>> On Mon, Jul 05 2021, Han-Wen Nienhuys via GitGitGadget wrote:
>>
>> > v5
>>
>> v3?
>
> I erroneously closed out the github PR I used for the first 2 versions
> of this change series.  (https://github.com/git/git/pull/1011)
>
>> >  * address Ævar's comment; punt on clearing errno.
>> > [...]
>> >
>> >           Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
>> >           Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
>> >      +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>
>> FWIW per Documentation/SubmittingPatches:
>>
>>     . `Reviewed-by:`, unlike the other tags, can only be offered by the
>>       reviewer and means that she is completely satisfied that the patch
>>       is ready for application.  It is usually offered only after a
>>       detailed review.
>>
>> It's not that I'm hard to please, but I can honestly say that I don't
>> quite understand some parts of what you're gonig for, so that trailer is
>> probably premature :)
>
> oh, ok. Removed them.
>
> Does that mean you'd have to repost my patchseries just to add the
> reviewed-by header?

It's fine to keep it. I just noted this (with a ":)") since I
Reviewed-By is understood by Junio (I believe) to mean something closer
to "this person has fully understood this code & endorses it". So far I
was only confident to say I was on right-hand-side of that "&",
give-or-take suggested changes :)

With the below suggestions for a v4 you can keep my Reviewed-By if you'd
like.

>> To rephrase my comment on the v2 to hopefully better get at the
>> point/question I had.
>>
>> It wasn't that I don't get why you wouldn't save/restore errno in
>> general.
>>
>> It's that the pattern of doing so seems backwards to me. I.e. surely the
>> goal here should be to one function at a time, and from the bottom-up,
>> figure out where we rely on "errno" and convert that to a
>> "failure_errno".
>>
>> Instead not even files_read_raw_ref() resets "errno = 0" at the end, so
>> the errno /there/ can propagate upwards, and in this v3 we're not clearing it at all.
>
> I would really want errno to be an explicit output for more functions
> in refs.h, but I don't have the time to see that through, and it's a
> distraction from the reftable work, so I have changed the objective of
> the patch series.
>
> I'm limiting myself to explicitly propagating errno values that are
> used beyond error reporting. The EINVAL value for
> parse_loose_ref_contents is a borderline case: it has an extra "if ()"
> body in lock_raw_ref, but it's for an error message.
>
> This means that we should avoid clearing errno, because we'd otherwise
> have to track down all the places where it's being put into
> strerror(). The one case you found earlier is still there, and by not
> clearing errno, I can keep this series smaller by not also having to
> rework verify_lock/read_refs_full.

I really think your v2 (or v4?, whatever) that set "errno = 0" plus
eyeballing that grep command as we've done is justification enough to
keep the "errno = 0".

I just had the comment to the effect of "uh, oh, maybe we've missed more
of these" on v2, but I think since then with your grep we can be fairly
sure we haven't.

I.e. the codepaths involved return non-zero anyway, so any breakage
would be limited to ignoring that and directly checking errno, or using
errno to format error messages.

If there is something like that still it'll be fairly far up the stack,
I think it's unplausible that there's anything left.

We can do it in a follow-up series if you'd like, or not do it before
reftable. I just think it's cleaner to be assured that we're past this
particular hump, and are guaranteed not to gain future callers that rely
on the errno in some subtle way (without the explicit pass-in variable,
that is).

> In general, it's frustrating to observe that the files backend is a
> complex beast that nobody understands, but in order to replace it with
> something more principled, I have to spend lots of time cleaning it
> up.

Yeah, it sucks. Or more specifically, that our "abstract" API isn't very
"abstract", but basically a 1=1 mapping to the files backend.

>> Having dug a bit further, it seems what you're doing, whether it's
>> intentional or not, is relying on the parse_loose_ref_contents() setting
>> EINVAL, but you clobber your *failure_errno with it whether it returned
>> -1 or not.
>
> I've added a failure_errno argument parse_loose_ref_contents now.

Great, it's not clear if you picked up the full diff-on-top but I think
consistently using *failure_errno in files_read_raw_ref() instead of
"errno" and the "*failure_errno = errno" at the end also makes the code
much more readable (and allows for the removal of the "saved_errno" we
don't want).

By using "errno" itself for the body of the function it needs to be
really carefully read to assure oneself that one of the functions it
calls doesn't make a syscall, and even if we're assured of that now if
one of them has a new syscall added in the future it might be clobbered
at a distance.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v3 0/5] refs: cleanup errno sideband ref related functions
  2021-07-06 14:27         ` Ævar Arnfjörð Bjarmason
@ 2021-07-06 18:36           ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-06 18:36 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Tue, Jul 6, 2021 at 4:37 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

> > I've added a failure_errno argument parse_loose_ref_contents now.
>
> Great, it's not clear if you picked up the full diff-on-top but I think
> consistently using *failure_errno in files_read_raw_ref() instead of
> "errno" and the "*failure_errno = errno" at the end also makes the code
> much more readable (and allows for the removal of the "saved_errno" we
> don't want).

It's hard to get right: there are many system calls, and in most cases
EISDIR/ENOTDIR have to be propagated to the caller. I tried doing
this, and promptly got a failing test related to dir/file conflicts.
By doing

  *failure_errno = errno

near the exit of files_read_raw_ref(), we can be sure there are no
changes in behavior.  If anyone wants to understand the files backend
in more depth, they're welcome to disentangle this in a follow-up
patch.

> By using "errno" itself for the body of the function it needs to be
> really carefully read to assure oneself that one of the functions it
> calls doesn't make a syscall, and even if we're assured of that now if
> one of them has a new syscall added in the future it might be clobbered
> at a distance.

Yes, anyone adding a syscall to files_read_raw_ref has to be very careful.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v4 0/6] refs: cleanup errno sideband ref related functions
  2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
                       ` (5 preceding siblings ...)
  2021-07-06  0:38     ` [PATCH v3 0/5] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-06 18:55     ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                         ` (6 more replies)
  6 siblings, 7 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

v5

 * address Ævar's comment; punt on clearing errno.

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

 refs.c                | 60 ++++++++++++++++++++++++-----------
 refs/debug.c          |  4 +--
 refs/files-backend.c  | 73 ++++++++++++++++++++++---------------------
 refs/packed-backend.c | 15 ++++-----
 refs/refs-internal.h  | 40 ++++++++++++++++--------
 5 files changed, 116 insertions(+), 76 deletions(-)


base-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v4
Pull-Request: https://github.com/git/git/pull/1012

Range-diff vs v3:

 1:  e2a0e5387ab ! 1:  d6a41c3c0cb refs: remove EINVAL errno output from specification of read_raw_ref_fn
     @@ Commit message
          Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
      
       ## refs/refs-internal.h ##
      @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,
 2:  c594c9c5c67 ! 2:  ff7ea6efcba refs/files-backend: stop setting errno from lock_ref_oid_basic
     @@ Commit message
          documented to use and/or preserve errno)
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
 3:  b017caf54ba ! 3:  ff5696b0875 refs: make errno output explicit for read_raw_ref_fn
     @@ Commit message
          relevant.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
 4:  4aaa9d3bd6f ! 4:  9c933706cb0 refs: add failure_errno to refs_read_raw_ref() signature
     @@ Commit message
          This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: done:
 -:  ----------- > 5:  ab147afb38d refs: explicitly return failure_errno from parse_loose_ref_contents
 5:  ed5347d7bb6 ! 6:  0526a41b4b3 refs: make errno output explicit for refs_resolve_ref_unsafe
     @@ Commit message
          that needs error information to make logic decisions, so update that caller
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
     -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
                         ` (5 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936..f4445e32904 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v4 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                         ` (4 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 677b7e4cdd2..83ddfb3b627 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		last_errno = errno;
+		int last_errno = errno;
 		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
+		    /* in case of D/F conflict, try to generate a better error
+		     * message. If that fails, fall back to strerror(ENOTDIR).
+		     */
+		    !refs_verify_refname_available(&refs->base, refname, extras,
+						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
 				    refname, strerror(last_errno));
 
@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v4 3/6] refs: make errno output explicit for read_raw_ref_fn
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 18:55       ` [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
                         ` (3 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 10 ++++++++--
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 23 +++++++++++------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 37 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235ea..5e5e3af8da0 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
+	int result;
+	int failure_errno;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+	failure_errno = 0;
+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					     type, &failure_errno);
+	if (failure_errno)
+		errno = failure_errno;
+	return result;
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc34..f12413a9bc0 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 83ddfb3b627..f98b1be9386 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ stat_ref:
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno = 0;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -629,9 +628,9 @@ retry:
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +654,7 @@ retry:
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +692,13 @@ retry:
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db6..a457f18e93c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e32904..79dfb3af484 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
                         ` (2 preceding siblings ...)
  2021-07-06 18:55       ` [PATCH v4 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 19:39         ` Junio C Hamano
  2021-07-06 18:55       ` [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
                         ` (2 subsequent siblings)
  6 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 60 +++++++++++++++++++++++++++----------------
 refs/files-backend.c  |  8 +++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 49 insertions(+), 32 deletions(-)

diff --git a/refs.c b/refs.c
index 5e5e3af8da0..25d80e544d0 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,34 +1671,35 @@ done:
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
-	int result;
-	int failure_errno;
+	int ignore;
+	if (failure_errno)
+		*failure_errno = 0;
+	else
+		failure_errno = &ignore;
+
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	failure_errno = 0;
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure_errno);
-	if (failure_errno)
-		errno = failure_errno;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
+	int unused_errno;
 	int symref_count;
 
 	if (!oid)
@@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		flags = &unused_flags;
 
 	*flags = 0;
+	if (!failure_errno)
+		failure_errno = &unused_errno;
+	*failure_errno = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
@@ -1728,11 +1732,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
+		int read_failure = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
+			*failure_errno = read_failure;
+
 			/* In reading mode, refs must eventually resolve */
 			if (resolve_flags & RESOLVE_REF_READING)
 				return NULL;
@@ -1742,9 +1749,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (read_failure != ENOENT && read_failure != EISDIR &&
+			    read_failure != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1783,6 +1789,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int ignore = 0;
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore);
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2244,7 +2259,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, NULL)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f98b1be9386..f38c9703504 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -383,8 +383,8 @@ stat_ref:
 	if (lstat(path, &st) < 0) {
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -423,8 +423,8 @@ stat_ref:
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c..530dba29788 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno = 0;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af484..54f57c6a2df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
                         ` (3 preceding siblings ...)
  2021-07-06 18:55       ` [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-06 19:37         ` Junio C Hamano
  2021-07-06 18:55       ` [PATCH v4 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  6 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 13 +++++++++----
 refs/refs-internal.h |  6 ++++--
 3 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index 25d80e544d0..eca7310e7a4 100644
--- a/refs.c
+++ b/refs.c
@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f38c9703504..b43ec4c66cb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -455,9 +455,13 @@ stat_ref:
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
-
+	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
+	errno = *failure_errno;
 out:
+	/* Collect all types of failures from errno. Many system calls in this
+	 * function can fail with ENOTDIR/EISDIR, and we want to collect all of
+	 * them.
+	 */
 	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
@@ -465,7 +469,8 @@ out:
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -484,7 +489,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2df..bf581e70cf6 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -689,10 +689,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v4 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
                         ` (4 preceding siblings ...)
  2021-07-06 18:55       ` [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 18:55       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  6 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-06 18:55 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 15 ++++++++-------
 refs/refs-internal.h |  8 ++++++++
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index b43ec4c66cb..d86893c6e0a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -928,6 +928,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -940,10 +941,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -963,15 +965,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
+		if (resolve_errno != ENOTDIR ||
 		    /* in case of D/F conflict, try to generate a better error
 		     * message. If that fails, fall back to strerror(ENOTDIR).
 		     */
 		    !refs_verify_refname_available(&refs->base, refname, extras,
 						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index bf581e70cf6..df01d5dc8df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		      struct object_id *oid, struct strbuf *referent,
 		      unsigned int *type, int *failure_errno);
 
+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
+ * failure. */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
+
 /*
  * Write an error to `err` and return a nonzero value iff the same
  * refname appears multiple times in `refnames`. `refnames` must be
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-05 20:56     ` [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 19:28       ` Junio C Hamano
  0 siblings, 0 replies; 148+ messages in thread
From: Junio C Hamano @ 2021-07-06 19:28 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> -	int result;
> -	int failure_errno;
> +	int ignore;
> +	if (failure_errno)
> +		*failure_errno = 0;
> +	else
> +		failure_errno = &ignore;

Hmph, I would have expected that a piece of code that gives fallback
location for ignored return parameter to initialize the location the
same way, i.e.

	int ignore

	if (!failure_errno)
		failure_errno = &ignore
	*failure_errno = 0;

That way, the code that follows that takes failure_errno does not
have to care if that return parameter location was supplied by the
caller or given by the fallback logic.

> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
> +					       const char *refname,
> +					       int resolve_flags,
> +					       struct object_id *oid,
> +					       int *flags, int *failure_errno)
>  {
>  	static struct strbuf sb_refname = STRBUF_INIT;
>  	struct object_id unused_oid;
>  	int unused_flags;
> +	int unused_errno;
>  	int symref_count;
>  
>  	if (!oid)
> @@ -1707,6 +1708,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
>  		flags = &unused_flags;
>  
>  	*flags = 0;
> +	if (!failure_errno)
> +		failure_errno = &unused_errno;
> +	*failure_errno = 0;

And you do use that pattern correctly here.  We probably would want
to be consistent.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-06 18:55       ` [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 19:37         ` Junio C Hamano
  2021-07-07  8:20           ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Junio C Hamano @ 2021-07-06 19:37 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> The EINVAL error from parse_loose_ref_contents is used in files-backend
> to create a custom error message.
>
> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

I think a -By in the trailer is spelled with lowercase, i.e. "Reviewed-by".

> ---
>  refs.c               |  8 +++++---
>  refs/files-backend.c | 13 +++++++++----
>  refs/refs-internal.h |  6 ++++--
>  3 files changed, 18 insertions(+), 9 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 25d80e544d0..eca7310e7a4 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
>  
>  static int refs_read_special_head(struct ref_store *ref_store,
>  				  const char *refname, struct object_id *oid,
> -				  struct strbuf *referent, unsigned int *type)
> +				  struct strbuf *referent, unsigned int *type,
> +				  int *failure_errno)
>  {
>  	struct strbuf full_path = STRBUF_INIT;
>  	struct strbuf content = STRBUF_INIT;
> @@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
>  	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
>  		goto done;
>  
> -	result = parse_loose_ref_contents(content.buf, oid, referent, type);
> +	result = parse_loose_ref_contents(content.buf, oid, referent, type,
> +					  failure_errno);
>  
>  done:
>  	strbuf_release(&full_path);
> @@ -1683,7 +1685,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  
>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
> -					      type);
> +					      type, failure_errno);
>  	}
>  
>  	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index f38c9703504..b43ec4c66cb 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -455,9 +455,13 @@ stat_ref:
>  	strbuf_rtrim(&sb_contents);
>  	buf = sb_contents.buf;
>  
> -	ret = parse_loose_ref_contents(buf, oid, referent, type);
> -
> +	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
> +	errno = *failure_errno;

All of the above makes sense.

>  out:
> +	/* Collect all types of failures from errno. Many system calls in this
> +	 * function can fail with ENOTDIR/EISDIR, and we want to collect all of
> +	 * them.
> +	 */

Style.

>  	*failure_errno = errno;
>  	strbuf_release(&sb_path);
>  	strbuf_release(&sb_contents);
> @@ -465,7 +469,8 @@ out:
>  }
>  
>  int parse_loose_ref_contents(const char *buf, struct object_id *oid,
> -			     struct strbuf *referent, unsigned int *type)
> +			     struct strbuf *referent, unsigned int *type,
> +			     int *failure_errno)
>  {
>  	const char *p;
>  	if (skip_prefix(buf, "ref:", &buf)) {
> @@ -484,7 +489,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
>  	if (parse_oid_hex(buf, oid, &p) ||
>  	    (*p != '\0' && !isspace(*p))) {
>  		*type |= REF_ISBROKEN;
> -		errno = EINVAL;
> +		*failure_errno = EINVAL;
>  		return -1;
>  	}

OK.

>  	return 0;
> diff --git a/refs/refs-internal.h b/refs/refs-internal.h
> index 54f57c6a2df..bf581e70cf6 100644
> --- a/refs/refs-internal.h
> +++ b/refs/refs-internal.h
> @@ -689,10 +689,12 @@ struct ref_store {
>  };
>  
>  /*
> - * Parse contents of a loose ref file.
> + * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
> + * invalid contents.
>   */

OK.

>  int parse_loose_ref_contents(const char *buf, struct object_id *oid,
> -			     struct strbuf *referent, unsigned int *type);
> +			     struct strbuf *referent, unsigned int *type,
> +			     int *failure_errno);
>  
>  /*
>   * Fill in the generic part of refs and add it to our collection of

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-06 18:55       ` [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-06 19:39         ` Junio C Hamano
  0 siblings, 0 replies; 148+ messages in thread
From: Junio C Hamano @ 2021-07-06 19:39 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> -int refs_read_raw_ref(struct ref_store *ref_store,
> -		      const char *refname, struct object_id *oid,
> -		      struct strbuf *referent, unsigned int *type)
> +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
> +		      struct object_id *oid, struct strbuf *referent,
> +		      unsigned int *type, int *failure_errno)
>  {
> -	int result;
> -	int failure_errno;
> +	int ignore;
> +	if (failure_errno)
> +		*failure_errno = 0;
> +	else
> +		failure_errno = &ignore;
> +

Same comment as v3(v5).  Squashable fix-up follows.

diff --git a/refs.c b/refs.c
index eca7310e7a..aaea9b34c6 100644
--- a/refs.c
+++ b/refs.c
@@ -1677,11 +1677,11 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		      struct object_id *oid, struct strbuf *referent,
 		      unsigned int *type, int *failure_errno)
 {
-	int ignore;
-	if (failure_errno)
-		*failure_errno = 0;
-	else
-		failure_errno = &ignore;
+	int unused_errno;
+
+	if (!failure_errno)
+		failure_errno = &unused_errno;
+	*failure_errno = 0;
 
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-06 19:37         ` Junio C Hamano
@ 2021-07-07  8:20           ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-07  8:20 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

On Tue, Jul 6, 2021 at 9:37 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Han-Wen Nienhuys <hanwen@google.com>
> >
> > The EINVAL error from parse_loose_ref_contents is used in files-backend
> > to create a custom error message.
> >
> > Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
> > Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> I think a -By in the trailer is spelled with lowercase, i.e. "Reviewed-by".

.. All done.

I also addressed your other fixup.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v5 0/6] refs: cleanup errno sideband ref related functions
  2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
                         ` (5 preceding siblings ...)
  2021-07-06 18:55       ` [PATCH v4 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07       ` Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07         ` [PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                           ` (7 more replies)
  6 siblings, 8 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

v5

 * address Ævar's comment; punt on clearing errno.

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

 refs.c                | 58 ++++++++++++++++++++++-----------
 refs/debug.c          |  4 +--
 refs/files-backend.c  | 74 ++++++++++++++++++++++---------------------
 refs/packed-backend.c | 15 +++++----
 refs/refs-internal.h  | 40 +++++++++++++++--------
 5 files changed, 115 insertions(+), 76 deletions(-)


base-commit: 670b81a890388c60b7032a4f5b879f2ece8c4558
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1012%2Fhanwen%2Feinval-sideband-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1012/hanwen/einval-sideband-v5
Pull-Request: https://github.com/git/git/pull/1012

Range-diff vs v4:

 1:  d6a41c3c0cb = 1:  d6a41c3c0cb refs: remove EINVAL errno output from specification of read_raw_ref_fn
 2:  ff7ea6efcba ! 2:  95025080c16 refs/files-backend: stop setting errno from lock_ref_oid_basic
     @@ Commit message
          documented to use and/or preserve errno)
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
 3:  ff5696b0875 ! 3:  7feedb97201 refs: make errno output explicit for read_raw_ref_fn
     @@ Commit message
          relevant.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
 4:  9c933706cb0 ! 4:  ef91f5cee13 refs: add failure_errno to refs_read_raw_ref() signature
     @@ Commit message
          This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: done:
     @@ refs.c: done:
      -int refs_read_raw_ref(struct ref_store *ref_store,
      -		      const char *refname, struct object_id *oid,
      -		      struct strbuf *referent, unsigned int *type)
     +-{
     +-	int result;
     +-	int failure_errno;
      +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
      +		      struct object_id *oid, struct strbuf *referent,
      +		      unsigned int *type, int *failure_errno)
     - {
     --	int result;
     --	int failure_errno;
     -+	int ignore;
     -+	if (failure_errno)
     -+		*failure_errno = 0;
     -+	else
     -+		failure_errno = &ignore;
     -+
     ++{
     ++	int unused_errno;
     ++	if (!failure_errno)
     ++		failure_errno = &unused_errno;
     ++	*failure_errno = 0;
       	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
       		return refs_read_special_head(ref_store, refname, oid, referent,
       					      type);
 5:  ab147afb38d ! 5:  6918c214d1b refs: explicitly return failure_errno from parse_loose_ref_contents
     @@ Commit message
          to create a custom error message.
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs.c ##
      @@ refs.c: int for_each_fullref_in_prefixes(const char *namespace,
     @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
       done:
       	strbuf_release(&full_path);
      @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     - 
     + 	*failure_errno = 0;
       	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
       		return refs_read_special_head(ref_store, refname, oid, referent,
      -					      type);
     @@ refs/files-backend.c: stat_ref:
      +	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
      +	errno = *failure_errno;
       out:
     -+	/* Collect all types of failures from errno. Many system calls in this
     -+	 * function can fail with ENOTDIR/EISDIR, and we want to collect all of
     -+	 * them.
     ++	/*
     ++	 * Many system calls in this function can fail with ENOTDIR/EISDIR, and
     ++	 * we want to collect all of them, so simply copy the error out from
     ++	 * errno.
      +	 */
       	*failure_errno = errno;
       	strbuf_release(&sb_path);
 6:  0526a41b4b3 ! 6:  85a14bde904 refs: make errno output explicit for refs_resolve_ref_unsafe
     @@ Commit message
          that needs error information to make logic decisions, so update that caller
      
          Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
     -    Reviewed-By: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     +    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
      
       ## refs/files-backend.c ##
      @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07         ` [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
                           ` (6 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936..f4445e32904 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07         ` [PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-11 11:38           ` Ævar Arnfjörð Bjarmason
  2021-07-07 19:07         ` [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
                           ` (5 subsequent siblings)
  7 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 677b7e4cdd2..83ddfb3b627 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		last_errno = errno;
+		int last_errno = errno;
 		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
+		    /* in case of D/F conflict, try to generate a better error
+		     * message. If that fails, fall back to strerror(ENOTDIR).
+		     */
+		    !refs_verify_refname_available(&refs->base, refname, extras,
+						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
 				    refname, strerror(last_errno));
 
@@ -981,20 +981,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +1002,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07         ` [PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
  2021-07-07 19:07         ` [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-11 11:52           ` Ævar Arnfjörð Bjarmason
  2021-07-07 19:07         ` [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
                           ` (4 subsequent siblings)
  7 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 10 ++++++++--
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 23 +++++++++++------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 37 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235ea..5e5e3af8da0 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
+	int result;
+	int failure_errno;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+	failure_errno = 0;
+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					     type, &failure_errno);
+	if (failure_errno)
+		errno = failure_errno;
+	return result;
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc34..f12413a9bc0 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 83ddfb3b627..f98b1be9386 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ stat_ref:
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno = 0;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -629,9 +628,9 @@ retry:
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +654,7 @@ retry:
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +692,13 @@ retry:
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db6..a457f18e93c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e32904..79dfb3af484 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                           ` (2 preceding siblings ...)
  2021-07-07 19:07         ` [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-11 11:59           ` Ævar Arnfjörð Bjarmason
  2021-07-07 19:07         ` [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
                           ` (3 subsequent siblings)
  7 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 60 ++++++++++++++++++++++++++-----------------
 refs/files-backend.c  |  8 +++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 48 insertions(+), 33 deletions(-)

diff --git a/refs.c b/refs.c
index 5e5e3af8da0..de3826f60c9 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,34 +1671,33 @@ done:
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
-{
-	int result;
-	int failure_errno;
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
+{
+	int unused_errno;
+	if (!failure_errno)
+		failure_errno = &unused_errno;
+	*failure_errno = 0;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	failure_errno = 0;
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure_errno);
-	if (failure_errno)
-		errno = failure_errno;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
+	int unused_errno;
 	int symref_count;
 
 	if (!oid)
@@ -1707,6 +1706,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		flags = &unused_flags;
 
 	*flags = 0;
+	if (!failure_errno)
+		failure_errno = &unused_errno;
+	*failure_errno = 0;
 
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
@@ -1728,11 +1730,14 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
+		int read_failure = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, &read_failure)) {
 			*flags |= read_flags;
 
+			*failure_errno = read_failure;
+
 			/* In reading mode, refs must eventually resolve */
 			if (resolve_flags & RESOLVE_REF_READING)
 				return NULL;
@@ -1742,9 +1747,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (read_failure != ENOENT && read_failure != EISDIR &&
+			    read_failure != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1783,6 +1787,15 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int ignore = 0;
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore);
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2244,7 +2257,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, NULL)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f98b1be9386..f38c9703504 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -383,8 +383,8 @@ stat_ref:
 	if (lstat(path, &st) < 0) {
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -423,8 +423,8 @@ stat_ref:
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, NULL)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c..530dba29788 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno = 0;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af484..54f57c6a2df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                           ` (3 preceding siblings ...)
  2021-07-07 19:07         ` [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-11 12:41           ` Ævar Arnfjörð Bjarmason
  2021-07-07 19:07         ` [PATCH v5 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
                           ` (2 subsequent siblings)
  7 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 14 ++++++++++----
 refs/refs-internal.h |  6 ++++--
 3 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index de3826f60c9..96df78a9509 100644
--- a/refs.c
+++ b/refs.c
@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1681,7 +1683,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	*failure_errno = 0;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f38c9703504..7aafdf2ce3d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -455,9 +455,14 @@ stat_ref:
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
-
+	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
+	errno = *failure_errno;
 out:
+	/*
+	 * Many system calls in this function can fail with ENOTDIR/EISDIR, and
+	 * we want to collect all of them, so simply copy the error out from
+	 * errno.
+	 */
 	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
@@ -465,7 +470,8 @@ out:
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -484,7 +490,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2df..bf581e70cf6 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -689,10 +689,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v5 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                           ` (4 preceding siblings ...)
  2021-07-07 19:07         ` [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 19:07         ` Han-Wen Nienhuys via GitGitGadget
  2021-07-07 20:44         ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Junio C Hamano
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys via GitGitGadget @ 2021-07-07 19:07 UTC (permalink / raw)
  To: git
  Cc: Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys,
	Han-Wen Nienhuys

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 15 ++++++++-------
 refs/refs-internal.h |  8 ++++++++
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7aafdf2ce3d..d6a7a0ee919 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -929,6 +929,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -941,10 +942,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -964,15 +966,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
+		if (resolve_errno != ENOTDIR ||
 		    /* in case of D/F conflict, try to generate a better error
 		     * message. If that fails, fall back to strerror(ENOTDIR).
 		     */
 		    !refs_verify_refname_available(&refs->base, refname, extras,
 						   skip, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index bf581e70cf6..df01d5dc8df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -153,6 +153,14 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		      struct object_id *oid, struct strbuf *referent,
 		      unsigned int *type, int *failure_errno);
 
+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
+ * failure. */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
+
 /*
  * Write an error to `err` and return a nonzero value iff the same
  * refname appears multiple times in `refnames`. `refnames` must be
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 0/6] refs: cleanup errno sideband ref related functions
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                           ` (5 preceding siblings ...)
  2021-07-07 19:07         ` [PATCH v5 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
@ 2021-07-07 20:44         ` Junio C Hamano
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Junio C Hamano @ 2021-07-07 20:44 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys

"Han-Wen Nienhuys via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Han-Wen Nienhuys (6):
>   refs: remove EINVAL errno output from specification of read_raw_ref_fn
>   refs/files-backend: stop setting errno from lock_ref_oid_basic
>   refs: make errno output explicit for read_raw_ref_fn
>   refs: add failure_errno to refs_read_raw_ref() signature
>   refs: explicitly return failure_errno from parse_loose_ref_contents
>   refs: make errno output explicit for refs_resolve_ref_unsafe

Thanks, incremental changes look sensible.
Will requeue.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-07 19:07         ` [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
@ 2021-07-11 11:38           ` Ævar Arnfjörð Bjarmason
  2021-07-13  8:00             ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 11:38 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:

>  /*
>   * Locks a ref returning the lock on success and NULL on failure.
> - * On failure errno is set to something meaningful.
>   */
>  static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  					   const char *refname,
> @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  {
>  	struct strbuf ref_file = STRBUF_INIT;
>  	struct ref_lock *lock;
> -	int last_errno = 0;
>  	int mustexist = (old_oid && !is_null_oid(old_oid));
>  	int resolve_flags = RESOLVE_REF_NO_RECURSE;
>  	int resolved;
> @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  		 * to remain.
>  		 */
>  		if (remove_empty_directories(&ref_file)) {
> -			last_errno = errno;
>  			if (!refs_verify_refname_available(
>  					    &refs->base,
>  					    refname, extras, skip, err))
> @@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
>  						     &lock->old_oid, type);
>  	}
>  	if (!resolved) {
> -		last_errno = errno;
> +		int last_errno = errno;
>  		if (last_errno != ENOTDIR ||
> -		    !refs_verify_refname_available(&refs->base, refname,
> -						   extras, skip, err))
> +		    /* in case of D/F conflict, try to generate a better error
> +		     * message. If that fails, fall back to strerror(ENOTDIR).
> +		     */
> +		    !refs_verify_refname_available(&refs->base, refname, extras,
> +						   skip, err))
>  			strbuf_addf(err, "unable to resolve reference '%s': %s",
>  				    refname, strerror(last_errno));

I don't think it's some dealbreaker and we can move on, but just FWIW I
think what I mentioned ending in your
https://lore.kernel.org/git/CAFQ2z_NpyJQLuM70MhJ8K1h2v3QXFuAZRjN=SvSsjnukNRJ8pw@mail.gmail.com/
is still outstanding.

I.e. you added the comment, which is just says what the error emitting
looks like, that's all well & good.

But what I was pointing out that it didn't make sense to do any
"last_errno" here at all anymore. You pointed to 5b2d8d6f218
(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
2015-05-11), we started setting "last_errno" there, but that was *not*
to avoid clobbering between the !resolved and the
strbuf_add(strerror(last_errno)) here, but rather to carry the
"last_errno" forward to the end of this lock_ref_oid_basic(), because
other things (after this hunk) might reset/clear errno.

Anyway, as noted there it doesn't actually matter, just reviewing &
looking if there's any loose ends, and for future source spelunking for
anyone reading this thread.

I.e. something like what I mentioned in
https://lore.kernel.org/git/87k0mae0ga.fsf@evledraar.gmail.com/ could be
squashed in, or better yet (probably) this:

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 83ddfb3b627..f0ce0aac857 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -958,17 +958,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     refname, resolve_flags,
 						     &lock->old_oid, type);
 	}
-	if (!resolved) {
-		int last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    /* in case of D/F conflict, try to generate a better error
-		     * message. If that fails, fall back to strerror(ENOTDIR).
-		     */
-		    !refs_verify_refname_available(&refs->base, refname, extras,
-						   skip, err))
-			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
-
+	if (!resolved &&
+	    (errno != ENOTDIR ||
+	     /* in case of D/F conflict, try to generate a better error
+	      * message. If that fails, fall back to strerror(ENOTDIR).
+	      */
+	     !refs_verify_refname_available(&refs->base, refname, extras,
+					    skip, err))) {
+		strbuf_addf(err, "unable to resolve reference '%s': %s",
+			    refname, strerror(errno));
 		goto error_return;
 	}
 

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn
  2021-07-07 19:07         ` [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
@ 2021-07-11 11:52           ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 11:52 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> diff --git a/refs.c b/refs.c
> index 8c9490235ea..5e5e3af8da0 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1675,13 +1675,19 @@ int refs_read_raw_ref(struct ref_store *ref_store,
>  		      const char *refname, struct object_id *oid,
>  		      struct strbuf *referent, unsigned int *type)
>  {
> +	int result;
> +	int failure_errno;
>  	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
>  		return refs_read_special_head(ref_store, refname, oid, referent,
>  					      type);
>  	}
>  
> -	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> -					   type);
> +	failure_errno = 0;
> +	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
> +					     type, &failure_errno);
> +	if (failure_errno)
> +		errno = failure_errno;
> +	return result;
>  }

Just reading along again in sequence, and having forgotten where (if
anywhere) this was headed I still find this verbose pattern for /looking
like/ we'll be doing something that's not setting errno, but actually
just setting errno. I.e. the logic is the same as the below squash, but
let's read on...

diff --git a/refs.c b/refs.c
index 5e5e3af8da0..7b3f05a66ff 100644
--- a/refs.c
+++ b/refs.c
@@ -1675,19 +1675,13 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 		      const char *refname, struct object_id *oid,
 		      struct strbuf *referent, unsigned int *type)
 {
-	int result;
-	int failure_errno;
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
-	failure_errno = 0;
-	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					     type, &failure_errno);
-	if (failure_errno)
-		errno = failure_errno;
-	return result;
+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */

^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-07 19:07         ` [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
@ 2021-07-11 11:59           ` Ævar Arnfjörð Bjarmason
  2021-07-13  8:02             ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 11:59 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> From: Han-Wen Nienhuys <hanwen@google.com>
>
> This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
> [...]
> +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,

This and the subsequent commit don't compile for me, because this lacks
a prototype that you finally add in 6/6.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-07 19:07         ` [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
@ 2021-07-11 12:41           ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 12:41 UTC (permalink / raw)
  To: Han-Wen Nienhuys via GitGitGadget
  Cc: git, Jonathan Tan, Han-Wen Nienhuys, Han-Wen Nienhuys


On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:

> Subject: refs: explicitly return failure_errno from parse_loose_ref_contents
> [...]
> The EINVAL error from parse_loose_ref_contents is used in files-backend
> to create a custom error message.

Except as my https://lore.kernel.org/git/87v95o5ku8.fsf@evledraar.gmail.com/ shows, and if you remove this:

> +	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
> +	errno = *failure_errno;

We aren't explicitly returning things yet, instead we set it for some,
but others still fail if we don't support that one caller relying on
this "errno" and not "failure_errno" through the backdoor, why not
simply convert that remaining caller?

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v6? 00/17] refs API: get rid of errno setting entirely
  2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
                           ` (6 preceding siblings ...)
  2021-07-07 20:44         ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Junio C Hamano
@ 2021-07-11 16:30         ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
                             ` (18 more replies)
  7 siblings, 19 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

After starting a review of v5 of Han-Wen's where I was coming up with
squashes-on-top I continued with it and saw if I could get rid of
errno setting entirely in refs.

The end-state of that is that we do that:
	
	$ git -P grep -w errno refs.c
	refs.c:                 if (errno != EPIPE) {
	refs.c:                         /* Don't leak errno outside this API */
	refs.c:                         errno = 0;

The rest has various bugfixes etc. to Han-Wen's version.

I'm not sure about:

    refs: make errno ignoring explicit in lock_ref_oid_basic()

But it's an edge case that's present in Han-Wen's, but just
ignored/skipped past. Now it's a BUG(), I haven't been able to hit it
in testing, but maybe we do hit it somehow.

My amendmend of:

    refs: explicitly return failure_errno from parse_loose_ref_contents

Has it calling BUG() to assert that whenever we return non-zero we set
errno as expected, there were a few cases where functions omitted one
errno case, but set another.

Arguably everything as of 08/17 should be its follow-up series. I've
included it because I think looking at those patches assures the
reader that we didn't miss any subtle errno edge cases.

(I removed my Reviewed-by trailer because it seemed redundant to SOB,
and if I have SOB already on something saying I reviewed it seemed
odd...)

1. https://lore.kernel.org/git/pull.1012.v5.git.git.1625684869.gitgitgadget@gmail.com/

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

Ævar Arnfjörð Bjarmason (11):
  refs: make errno ignoring explicit in lock_ref_oid_basic()
  refs file-backend.c: stop setting "EBUSY" in verify_lock()
  refs file-backend.c: deal with errno directly in verify_lock()
  refs API: remove refs_read_ref_full() wrapper
  refs API: make resolve_gitlink_ref() not set errno
  refs API: make refs_resolve_ref_unsafe() static
  refs API: make refs_resolve_refdup() not set errno
  refs API: make refs_ref_exists() not set errno
  refs API: make resolve_ref_unsafe() not set errno
  refs API: make expand_ref() and repo_dwim_log() not set errno
  refs API: don't leak "errno" in run_transaction_hook()

 refs.c                    | 122 ++++++++++++++++----------
 refs.h                    |  17 ++--
 refs/debug.c              |   4 +-
 refs/files-backend.c      | 180 +++++++++++++++++++++-----------------
 refs/packed-backend.c     |  15 ++--
 refs/refs-internal.h      |  32 ++++---
 sequencer.c               |  10 ++-
 t/helper/test-ref-store.c |   5 +-
 worktree.c                |  27 +++---
 9 files changed, 244 insertions(+), 168 deletions(-)

Range-diff:
 1:  d6a41c3c0cb !  1:  57517368c34 refs: remove EINVAL errno output from specification of read_raw_ref_fn
    @@ Commit message
         Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.
     
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs/refs-internal.h ##
     @@ refs/refs-internal.h: typedef int reflog_expire_fn(struct ref_store *ref_store,
 2:  95025080c16 !  2:  61cf7611473 refs/files-backend: stop setting errno from lock_ref_oid_basic
    @@ Commit message
         refs_reflog_exists() (which calls a function in a vtable that is not
         documented to use and/or preserve errno)
     
    +    In the case of the "errno != ENOTDIR" case that originates in 5b2d8d6f218
    +    (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
    +    2015-05-11), there the "last_errno" was saved away to return it from
    +    lock_ref_oid_basic(), now that we're no longer doing that we can skip
    +    that entirely and use "errno" directly. A follow-up change will
    +    extract the specific errno we want earlier in this function.
    +
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      					    &refs->base,
      					    refname, extras, skip, err))
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    + 						     refname, resolve_flags,
      						     &lock->old_oid, type);
      	}
    - 	if (!resolved) {
    +-	if (!resolved) {
     -		last_errno = errno;
    -+		int last_errno = errno;
    - 		if (last_errno != ENOTDIR ||
    +-		if (last_errno != ENOTDIR ||
     -		    !refs_verify_refname_available(&refs->base, refname,
     -						   extras, skip, err))
    -+		    /* in case of D/F conflict, try to generate a better error
    -+		     * message. If that fails, fall back to strerror(ENOTDIR).
    -+		     */
    -+		    !refs_verify_refname_available(&refs->base, refname, extras,
    -+						   skip, err))
    - 			strbuf_addf(err, "unable to resolve reference '%s': %s",
    - 				    refname, strerror(last_errno));
    +-			strbuf_addf(err, "unable to resolve reference '%s': %s",
    +-				    refname, strerror(last_errno));
    +-
    ++	if (!resolved &&
    ++	    (errno != ENOTDIR ||
    ++	     /* in case of D/F conflict, try to generate a better error
    ++	      * message. If that fails, fall back to strerror(ENOTDIR).
    ++	      */
    ++	     !refs_verify_refname_available(&refs->base, refname, extras,
    ++					    skip, err))) {
    ++		strbuf_addf(err, "unable to resolve reference '%s': %s",
    ++			    refname, strerror(errno));
    + 		goto error_return;
    + 	}
      
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      	if (is_null_oid(&lock->old_oid) &&
 3:  7feedb97201 !  3:  a4e5f2d02fb refs: make errno output explicit for read_raw_ref_fn
    @@ Commit message
         relevant.
     
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs.c ##
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store,
    - 		      const char *refname, struct object_id *oid,
    - 		      struct strbuf *referent, unsigned int *type)
    - {
    -+	int result;
    -+	int failure_errno;
    - 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
    - 		return refs_read_special_head(ref_store, refname, oid, referent,
    - 					      type);
      	}
      
    --	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
    + 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
     -					   type);
    -+	failure_errno = 0;
    -+	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
    -+					     type, &failure_errno);
    -+	if (failure_errno)
    -+		errno = failure_errno;
    -+	return result;
    ++					   type, &errno);
      }
      
      /* This function needs to return a meaningful errno on failure */
    @@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
      	struct strbuf ref_file = STRBUF_INIT;
      	int attempts_remaining = 3;
      	int ret = TRANSACTION_GENERIC_ERROR;
    -+	int failure_errno = 0;
    ++	int failure_errno;
      
      	assert(err);
      	files_assert_main_repository(refs, "lock_raw_ref");
    +@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
    + 	if (hold_lock_file_for_update_timeout(
    + 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
    + 			    get_files_ref_lock_timeout_ms()) < 0) {
    +-		if (errno == ENOENT && --attempts_remaining > 0) {
    ++		int myerr = errno;
    ++		errno = 0;
    ++		if (myerr == ENOENT && --attempts_remaining > 0) {
    + 			/*
    + 			 * Maybe somebody just deleted one of the
    + 			 * directories leading to ref_file.  Try
    +@@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
    + 			 */
    + 			goto retry;
    + 		} else {
    +-			unable_to_lock_message(ref_file.buf, errno, err);
    ++			unable_to_lock_message(ref_file.buf, myerr, err);
    + 			goto error_return;
    + 		}
    + 	}
     @@ refs/files-backend.c: static int lock_raw_ref(struct files_ref_store *refs,
      	 * fear that its value will change.
      	 */
 4:  ef91f5cee13 !  4:  270cda29c3a refs: add failure_errno to refs_read_raw_ref() signature
    @@ Commit message
     
         This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
     
    +    Some of our callers explicitly do not care about the errno, rather
    +    than understanding NULL let's have them declare that they don't care
    +    by passing in an "ignore_errno". There's only three of them, and using
    +    that pattern will make it more obvious that they want to throw away
    +    data, let's also add a comment to one of the callers about why we'd
    +    like to ignore the errno.
    +
    +    Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
    +    has a large set of legacy callers, so we're faking up the old "errno"
    +    behavior for it. We can convert those callers to
    +    refs_resolve_ref_unsafe_with_errno() later.
    +
    +    We are leaving out out the refs_read_special_head() in
    +    refs_read_raw_ref() for now, as noted in the next commit moving it to
    +    "failure_errno" will require some special consideration.
    +
    +    We're intentionally mis-indenting the argument list of the new
    +    refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
    +    subsequent commit, doing it this way makes that diff smaller.
    +
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs.c ##
     @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
    @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
     -int refs_read_raw_ref(struct ref_store *ref_store,
     -		      const char *refname, struct object_id *oid,
     -		      struct strbuf *referent, unsigned int *type)
    --{
    --	int result;
    --	int failure_errno;
     +int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     +		      struct object_id *oid, struct strbuf *referent,
     +		      unsigned int *type, int *failure_errno)
    -+{
    -+	int unused_errno;
    -+	if (!failure_errno)
    -+		failure_errno = &unused_errno;
    -+	*failure_errno = 0;
    + {
    ++	assert(failure_errno);
      	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
      		return refs_read_special_head(ref_store, refname, oid, referent,
      					      type);
      	}
      
    --	failure_errno = 0;
    --	result = ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
    --					     type, &failure_errno);
    --	if (failure_errno)
    --		errno = failure_errno;
    --	return result;
    -+	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
    + 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
    +-					   type, &errno);
     +					   type, failure_errno);
      }
      
    @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
     -				    const char *refname,
     -				    int resolve_flags,
     -				    struct object_id *oid, int *flags)
    -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
    ++static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
     +					       const char *refname,
     +					       int resolve_flags,
     +					       struct object_id *oid,
    @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
      	static struct strbuf sb_refname = STRBUF_INIT;
      	struct object_id unused_oid;
      	int unused_flags;
    -+	int unused_errno;
      	int symref_count;
      
    ++	assert(failure_errno);
    ++
      	if (!oid)
    + 		oid = &unused_oid;
    + 	if (!flags)
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
    - 		flags = &unused_flags;
    - 
    - 	*flags = 0;
    -+	if (!failure_errno)
    -+		failure_errno = &unused_errno;
    -+	*failure_errno = 0;
    - 
      	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
      		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
    -@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
    + 		    !refname_is_safe(refname)) {
    +-			errno = EINVAL;
    ++			*failure_errno = EINVAL;
    + 			return NULL;
    + 		}
      
    +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
      	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
      		unsigned int read_flags = 0;
    -+		int read_failure = 0;
      
     -		if (refs_read_raw_ref(refs, refname,
     -				      oid, &sb_refname, &read_flags)) {
     +		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
    -+				      &read_flags, &read_failure)) {
    ++				      &read_flags, failure_errno)) {
      			*flags |= read_flags;
      
    -+			*failure_errno = read_failure;
    -+
      			/* In reading mode, refs must eventually resolve */
    - 			if (resolve_flags & RESOLVE_REF_READING)
    - 				return NULL;
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
      			 * may show errors besides ENOENT if there are
      			 * similarly-named refs.
    @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     -			if (errno != ENOENT &&
     -			    errno != EISDIR &&
     -			    errno != ENOTDIR)
    -+			if (read_failure != ENOENT && read_failure != EISDIR &&
    -+			    read_failure != ENOTDIR)
    ++			if (*failure_errno != ENOENT &&
    ++			    *failure_errno != EISDIR &&
    ++			    *failure_errno != ENOTDIR)
      				return NULL;
      
      			oidclr(oid);
     @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
    + 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
    + 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
    + 			    !refname_is_safe(refname)) {
    +-				errno = EINVAL;
    ++				*failure_errno = EINVAL;
    + 				return NULL;
    + 			}
    + 
    +@@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
    + 		}
    + 	}
    + 
    +-	errno = ELOOP;
    ++	*failure_errno = ELOOP;
      	return NULL;
      }
      
    @@ refs.c: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     +				    int resolve_flags, struct object_id *oid,
     +				    int *flags)
     +{
    -+	int ignore = 0;
    -+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
    -+						  oid, flags, &ignore);
    ++	int failure_errno = 0;
    ++	const char *refn;
    ++	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
    ++						  oid, flags, &failure_errno);
    ++	if (!refn)
    ++		/* For unmigrated legacy callers */
    ++		errno = failure_errno;
    ++	return refn;
     +}
     +
      /* backend functions */
      int refs_init_db(struct strbuf *err)
      {
    +@@ refs.c: int refs_verify_refname_available(struct ref_store *refs,
    + 
    + 	strbuf_grow(&dirname, strlen(refname) + 1);
    + 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
    ++		/*
    ++		 * Just saying "Is a directory" when we e.g. can't
    ++		 * lock some multi-level ref isn't very informative,
    ++		 * the user won't be told *what* is a directory, so
    ++		 * let's not use strerror() below.
    ++		 */
    ++		int ignore_errno;
    + 		/* Expand dirname to the new prefix, not including the trailing slash: */
    + 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
    + 
     @@ refs.c: int refs_verify_refname_available(struct ref_store *refs,
      		if (skip && string_list_has_string(skip, dirname.buf))
      			continue;
      
     -		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
     +		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
    -+				       &type, NULL)) {
    ++				       &type, &ignore_errno)) {
      			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
      				    dirname.buf, refname);
      			goto cleanup;
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 		goto out;
    + 
      	if (lstat(path, &st) < 0) {
    ++		int ignore_errno;
      		if (errno != ENOENT)
      			goto out;
     -		if (refs_read_raw_ref(refs->packed_ref_store, refname,
     -				      oid, referent, type)) {
     +		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
    -+				      referent, type, NULL)) {
    ++				      referent, type, &ignore_errno)) {
      			errno = ENOENT;
      			goto out;
      		}
     @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 
    + 	/* Is it a directory? */
    + 	if (S_ISDIR(st.st_mode)) {
    ++		int ignore_errno;
    + 		/*
    + 		 * Even though there is a directory where the loose
      		 * ref is supposed to be, there could still be a
      		 * packed ref:
      		 */
     -		if (refs_read_raw_ref(refs->packed_ref_store, refname,
     -				      oid, referent, type)) {
     +		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
    -+				      referent, type, NULL)) {
    ++				      referent, type, &ignore_errno)) {
      			errno = EISDIR;
      			goto out;
      		}
    @@ refs/packed-backend.c: int is_packed_transaction_needed(struct ref_store *ref_st
      	ret = 0;
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		int failure_errno = 0;
    ++		int failure_errno;
      		unsigned int type;
      		struct object_id oid;
      
 5:  6918c214d1b !  5:  f5197cdc0b8 refs: explicitly return failure_errno from parse_loose_ref_contents
    @@ Commit message
         The EINVAL error from parse_loose_ref_contents is used in files-backend
         to create a custom error message.
     
    +    In untangling this we discovered a tricky edge case. The
    +    refs_read_special_head() function was relying on
    +    parse_loose_ref_contents() setting EINVAL.
    +
    +    By converting it to use "saved_errno" we can migrate away from "errno"
    +    in this part of the code entirely, and do away with an existing
    +    "save_errno" pattern, its only purpose was to not clobber the "errno"
    +    we previously needed at the end of files_read_raw_ref().
    +
    +    Let's assert that we can do that by not having files_read_raw_ref()
    +    itself operate on *failure_errno in addition to passing it on. Instead
    +    we'll assert that if we return non-zero we actually do set errno, thus
    +    assuring ourselves and callers that they can trust the resulting
    +    "failure_errno".
    +
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs.c ##
     @@ refs.c: int for_each_fullref_in_prefixes(const char *namespace,
    @@ refs.c: static int refs_read_special_head(struct ref_store *ref_store,
      done:
      	strbuf_release(&full_path);
     @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
    - 	*failure_errno = 0;
    + 	assert(failure_errno);
      	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
      		return refs_read_special_head(ref_store, refname, oid, referent,
     -					      type);
    @@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	int fd;
    + 	int ret = -1;
    + 	int remaining_retries = 3;
    ++	int myerr = 0;
    + 
    + 	*type = 0;
    + 	strbuf_reset(&sb_path);
    +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 
    + 	if (lstat(path, &st) < 0) {
    + 		int ignore_errno;
    +-		if (errno != ENOENT)
    ++		myerr = errno;
    ++		errno = 0;
    ++		if (myerr != ENOENT)
    + 			goto out;
    + 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
    + 				      referent, type, &ignore_errno)) {
    +-			errno = ENOENT;
    ++			myerr = ENOENT;
    + 			goto out;
    + 		}
    + 		ret = 0;
    +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	if (S_ISLNK(st.st_mode)) {
    + 		strbuf_reset(&sb_contents);
    + 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
    +-			if (errno == ENOENT || errno == EINVAL)
    ++			myerr = errno;
    ++			errno = 0;
    ++			if (myerr == ENOENT || myerr == EINVAL)
    + 				/* inconsistent with lstat; retry */
    + 				goto stat_ref;
    + 			else
    +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 		 */
    + 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
    + 				      referent, type, &ignore_errno)) {
    +-			errno = EISDIR;
    ++			myerr = EISDIR;
    + 			goto out;
    + 		}
    + 		ret = 0;
    +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	 */
    + 	fd = open(path, O_RDONLY);
    + 	if (fd < 0) {
    +-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
    ++		myerr = errno;
    ++		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
    + 			/* inconsistent with lstat; retry */
    + 			goto stat_ref;
    + 		else
    +@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	}
    + 	strbuf_reset(&sb_contents);
    + 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
    +-		int save_errno = errno;
    + 		close(fd);
    +-		errno = save_errno;
    + 		goto out;
    + 	}
    + 	close(fd);
      	strbuf_rtrim(&sb_contents);
      	buf = sb_contents.buf;
      
     -	ret = parse_loose_ref_contents(buf, oid, referent, type);
    --
    -+	ret = parse_loose_ref_contents(buf, oid, referent, type, failure_errno);
    -+	errno = *failure_errno;
    ++	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
    + 
      out:
    -+	/*
    -+	 * Many system calls in this function can fail with ENOTDIR/EISDIR, and
    -+	 * we want to collect all of them, so simply copy the error out from
    -+	 * errno.
    -+	 */
    - 	*failure_errno = errno;
    +-	*failure_errno = errno;
    ++	if (ret && !myerr)
    ++		BUG("returning non-zero %d, should have set myerr!", ret);
    ++	*failure_errno = myerr;
    ++
      	strbuf_release(&sb_path);
      	strbuf_release(&sb_contents);
    -@@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	return ret;
      }
      
      int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 6:  85a14bde904 !  6:  96689e523f1 refs: make errno output explicit for refs_resolve_ref_unsafe
    @@ Commit message
         that needs error information to make logic decisions, so update that caller
     
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
    -    Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +
    + ## refs.c ##
    +@@ refs.c: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 					   type, failure_errno);
    + }
    + 
    +-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
    ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
    + 					       const char *refname,
    + 					       int resolve_flags,
    + 					       struct object_id *oid,
    +
    + ## refs.h ##
    +@@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
    + 				    int resolve_flags,
    + 				    struct object_id *oid,
    + 				    int *flags);
    ++/**
    ++ * refs_resolve_ref_unsafe_with_errno() is like
    ++ * refs_resolve_ref_unsafe(), but provide access to errno code that
    ++ * lead to a failure. We guarantee that errno is set to a meaningful
    ++ * value on non-zero return.
    ++ */
    ++const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
    ++					       const char *refname,
    ++					       int resolve_flags,
    ++					       struct object_id *oid,
    ++					       int *flags, int *failure_errno);
    + const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
    + 			       struct object_id *oid, int *flags);
    + 
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      						     &lock->old_oid, type);
      	}
    - 	if (!resolved) {
    --		int last_errno = errno;
    --		if (last_errno != ENOTDIR ||
    -+		if (resolve_errno != ENOTDIR ||
    - 		    /* in case of D/F conflict, try to generate a better error
    - 		     * message. If that fails, fall back to strerror(ENOTDIR).
    - 		     */
    - 		    !refs_verify_refname_available(&refs->base, refname, extras,
    - 						   skip, err))
    - 			strbuf_addf(err, "unable to resolve reference '%s': %s",
    --				    refname, strerror(last_errno));
    -+				    refname, strerror(resolve_errno));
    - 
    + 	if (!resolved &&
    +-	    (errno != ENOTDIR ||
    ++	    (resolve_errno != ENOTDIR ||
    + 	     /* in case of D/F conflict, try to generate a better error
    + 	      * message. If that fails, fall back to strerror(ENOTDIR).
    + 	      */
    + 	     !refs_verify_refname_available(&refs->base, refname, extras,
    + 					    skip, err))) {
    + 		strbuf_addf(err, "unable to resolve reference '%s': %s",
    +-			    refname, strerror(errno));
    ++			    refname, strerror(resolve_errno));
      		goto error_return;
      	}
    -
    - ## refs/refs-internal.h ##
    -@@ refs/refs-internal.h: int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
    - 		      struct object_id *oid, struct strbuf *referent,
    - 		      unsigned int *type, int *failure_errno);
      
    -+/* Like refs_resolve_ref_unsafe, but provide access to errno code that lead to a
    -+ * failure. */
    -+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
    -+					       const char *refname,
    -+					       int resolve_flags,
    -+					       struct object_id *oid,
    -+					       int *flags, int *failure_errno);
    -+
    - /*
    -  * Write an error to `err` and return a nonzero value iff the same
    -  * refname appears multiple times in `refnames`. `refnames` must be
 -:  ----------- >  7:  10a40c9244e refs: make errno ignoring explicit in lock_ref_oid_basic()
 -:  ----------- >  8:  ff38a3f1936 refs file-backend.c: stop setting "EBUSY" in verify_lock()
 -:  ----------- >  9:  cdec272f06f refs file-backend.c: deal with errno directly in verify_lock()
 -:  ----------- > 10:  bd0639945a4 refs API: remove refs_read_ref_full() wrapper
 -:  ----------- > 11:  a359d1533bd refs API: make resolve_gitlink_ref() not set errno
 -:  ----------- > 12:  07d550015ac refs API: make refs_resolve_ref_unsafe() static
 -:  ----------- > 13:  73b70491515 refs API: make refs_resolve_refdup() not set errno
 -:  ----------- > 14:  1e9de48d850 refs API: make refs_ref_exists() not set errno
 -:  ----------- > 15:  446b50280be refs API: make resolve_ref_unsafe() not set errno
 -:  ----------- > 16:  37c5b88d7d8 refs API: make expand_ref() and repo_dwim_log() not set errno
 -:  ----------- > 17:  f914df0bb25 refs API: don't leak "errno" in run_transaction_hook()
-- 
2.32.0-dev


^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 02/17] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
                             ` (17 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936..f4445e32904 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 02/17] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 03/17] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
                             ` (16 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

In the case of the "errno != ENOTDIR" case that originates in 5b2d8d6f218
(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
2015-05-11), there the "last_errno" was saved away to return it from
lock_ref_oid_basic(), now that we're no longer doing that we can skip
that entirely and use "errno" directly. A follow-up change will
extract the specific errno we want earlier in this function.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 24 +++++++++---------------
 1 file changed, 9 insertions(+), 15 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 677b7e4cdd2..f0ce0aac857 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		 * to remain.
 		 */
 		if (remove_empty_directories(&ref_file)) {
-			last_errno = errno;
 			if (!refs_verify_refname_available(
 					    &refs->base,
 					    refname, extras, skip, err))
@@ -961,14 +958,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     refname, resolve_flags,
 						     &lock->old_oid, type);
 	}
-	if (!resolved) {
-		last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
-			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
-
+	if (!resolved &&
+	    (errno != ENOTDIR ||
+	     /* in case of D/F conflict, try to generate a better error
+	      * message. If that fails, fall back to strerror(ENOTDIR).
+	      */
+	     !refs_verify_refname_available(&refs->base, refname, extras,
+					    skip, err))) {
+		strbuf_addf(err, "unable to resolve reference '%s': %s",
+			    refname, strerror(errno));
 		goto error_return;
 	}
 
@@ -981,20 +979,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -1005,7 +1000,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 03/17] refs: make errno output explicit for read_raw_ref_fn
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 02/17] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 04/17] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
                             ` (15 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                |  2 +-
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 29 +++++++++++++++--------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235ea..7b3f05a66ff 100644
--- a/refs.c
+++ b/refs.c
@@ -1681,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc34..f12413a9bc0 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f0ce0aac857..b9df7862e57 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -611,7 +610,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	if (hold_lock_file_for_update_timeout(
 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
 			    get_files_ref_lock_timeout_ms()) < 0) {
-		if (errno == ENOENT && --attempts_remaining > 0) {
+		int myerr = errno;
+		errno = 0;
+		if (myerr == ENOENT && --attempts_remaining > 0) {
 			/*
 			 * Maybe somebody just deleted one of the
 			 * directories leading to ref_file.  Try
@@ -619,7 +620,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 			 */
 			goto retry;
 		} else {
-			unable_to_lock_message(ref_file.buf, errno, err);
+			unable_to_lock_message(ref_file.buf, myerr, err);
 			goto error_return;
 		}
 	}
@@ -629,9 +630,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +656,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +694,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db6..a457f18e93c 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e32904..79dfb3af484 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 04/17] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (2 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 03/17] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 05/17] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
                             ` (14 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Some of our callers explicitly do not care about the errno, rather
than understanding NULL let's have them declare that they don't care
by passing in an "ignore_errno". There's only three of them, and using
that pattern will make it more obvious that they want to throw away
data, let's also add a comment to one of the callers about why we'd
like to ignore the errno.

Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
has a large set of legacy callers, so we're faking up the old "errno"
behavior for it. We can convert those callers to
refs_resolve_ref_unsafe_with_errno() later.

We are leaving out out the refs_read_special_head() in
refs_read_raw_ref() for now, as noted in the next commit moving it to
"failure_errno" will require some special consideration.

We're intentionally mis-indenting the argument list of the new
refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
subsequent commit, doing it this way makes that diff smaller.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 61 ++++++++++++++++++++++++++++++-------------
 refs/files-backend.c  | 10 ++++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 7b3f05a66ff..406e27213b8 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,30 +1671,33 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
+	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type, &errno);
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
 	int symref_count;
 
+	assert(failure_errno);
+
 	if (!oid)
 		oid = &unused_oid;
 	if (!flags)
@@ -1705,7 +1708,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1723,8 +1726,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, failure_errno)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1736,9 +1739,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (*failure_errno != ENOENT &&
+			    *failure_errno != EISDIR &&
+			    *failure_errno != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1765,7 +1768,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1773,10 +1776,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2227,6 +2244,13 @@ int refs_verify_refname_available(struct ref_store *refs,
 
 	strbuf_grow(&dirname, strlen(refname) + 1);
 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
+		/*
+		 * Just saying "Is a directory" when we e.g. can't
+		 * lock some multi-level ref isn't very informative,
+		 * the user won't be told *what* is a directory, so
+		 * let's not use strerror() below.
+		 */
+		int ignore_errno;
 		/* Expand dirname to the new prefix, not including the trailing slash: */
 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
 
@@ -2238,7 +2262,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, &ignore_errno)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index b9df7862e57..79f96ba1058 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
+		int ignore_errno;
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	/* Is it a directory? */
 	if (S_ISDIR(st.st_mode)) {
+		int ignore_errno;
 		/*
 		 * Even though there is a directory where the loose
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93c..928e5d2a994 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af484..54f57c6a2df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 05/17] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (3 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 04/17] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 06/17] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
                             ` (13 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

In untangling this we discovered a tricky edge case. The
refs_read_special_head() function was relying on
parse_loose_ref_contents() setting EINVAL.

By converting it to use "saved_errno" we can migrate away from "errno"
in this part of the code entirely, and do away with an existing
"save_errno" pattern, its only purpose was to not clobber the "errno"
we previously needed at the end of files_read_raw_ref().

Let's assert that we can do that by not having files_read_raw_ref()
itself operate on *failure_errno in addition to passing it on. Instead
we'll assert that if we return non-zero we actually do set errno, thus
assuring ourselves and callers that they can trust the resulting
"failure_errno".

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 30 +++++++++++++++++++-----------
 refs/refs-internal.h |  6 ++++--
 3 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index 406e27213b8..af01a692cb9 100644
--- a/refs.c
+++ b/refs.c
@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1678,7 +1680,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 79f96ba1058..702e38a2b06 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	int fd;
 	int ret = -1;
 	int remaining_retries = 3;
+	int myerr = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (lstat(path, &st) < 0) {
 		int ignore_errno;
-		if (errno != ENOENT)
+		myerr = errno;
+		errno = 0;
+		if (myerr != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = ENOENT;
+			myerr = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			myerr = errno;
+			errno = 0;
+			if (myerr == ENOENT || myerr == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = EISDIR;
+			myerr = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		myerr = errno;
+		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
 
 out:
-	*failure_errno = errno;
+	if (ret && !myerr)
+		BUG("returning non-zero %d, should have set myerr!", ret);
+	*failure_errno = myerr;
+
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2df..bf581e70cf6 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -689,10 +689,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 06/17] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (4 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 05/17] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 07/17] refs: make errno ignoring explicit in lock_ref_oid_basic() Ævar Arnfjörð Bjarmason
                             ` (12 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  2 +-
 refs.h               | 11 +++++++++++
 refs/files-backend.c | 14 ++++++++------
 3 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index af01a692cb9..1dacb5fe27b 100644
--- a/refs.c
+++ b/refs.c
@@ -1687,7 +1687,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
diff --git a/refs.h b/refs.h
index 48970dfc7e0..344a3c51a8c 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+/**
+ * refs_resolve_ref_unsafe_with_errno() is like
+ * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * lead to a failure. We guarantee that errno is set to a meaningful
+ * value on non-zero return.
+ */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 702e38a2b06..d6ee7c1172f 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -935,6 +935,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -947,10 +948,11 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
-	if (!resolved && errno == EISDIR) {
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
+	if (!resolved && resolve_errno == EISDIR) {
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -970,14 +972,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 						     &lock->old_oid, type);
 	}
 	if (!resolved &&
-	    (errno != ENOTDIR ||
+	    (resolve_errno != ENOTDIR ||
 	     /* in case of D/F conflict, try to generate a better error
 	      * message. If that fails, fall back to strerror(ENOTDIR).
 	      */
 	     !refs_verify_refname_available(&refs->base, refname, extras,
 					    skip, err))) {
 		strbuf_addf(err, "unable to resolve reference '%s': %s",
-			    refname, strerror(errno));
+			    refname, strerror(resolve_errno));
 		goto error_return;
 	}
 
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 07/17] refs: make errno ignoring explicit in lock_ref_oid_basic()
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (5 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 06/17] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock() Ævar Arnfjörð Bjarmason
                             ` (11 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

In the preceding commit we moved away from refs_resolve_ref_unsafe()
in this function and started using a "resolve_errno" variable, which
means that if this refs_resolve_ref_unsafe() invocation sets "errno"
we'd end up ignoring it for our earlier "resolve_errno" (if any).

I'm not sure if this is what we should be doing, but let's add a BUG()
here for now to indicate that we hit a case we'd previously "handle",
but which we now ignore.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index d6ee7c1172f..a4e9344ac8b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -953,6 +953,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 							&lock->old_oid, type,
 							&resolve_errno);
 	if (!resolved && resolve_errno == EISDIR) {
+		int ignore_errno = 0;
 		/*
 		 * we are trying to lock foo but we used to
 		 * have foo/bar which now does not exist;
@@ -967,9 +968,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					    refname);
 			goto error_return;
 		}
-		resolved = !!refs_resolve_ref_unsafe(&refs->base,
-						     refname, resolve_flags,
-						     &lock->old_oid, type);
+		resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base,
+								refname, resolve_flags,
+								&lock->old_oid, type,
+								&ignore_errno);
+		if (ignore_errno)
+			BUG("hit errno %d (%s) that will be ignored, "
+			    "resolve errno is %d (%s)",
+			    ignore_errno, strerror(ignore_errno),
+			    resolve_errno, strerror(resolve_errno));
 	}
 	if (!resolved &&
 	    (resolve_errno != ENOTDIR ||
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock()
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (6 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 07/17] refs: make errno ignoring explicit in lock_ref_oid_basic() Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-13  8:08             ` Han-Wen Nienhuys
  2021-07-11 16:30           ` [PATCH v6? 09/17] refs file-backend.c: deal with errno directly " Ævar Arnfjörð Bjarmason
                             ` (10 subsequent siblings)
  18 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

This EBUSY was set in 835e3c992fd (refs.c: verify_lock should set
errno to something meaningful, 2014-06-20) to fix a bug in code that
was later refactored in cd94f765720 (fetch.c: change s_update_ref to
use a ref transaction, 2014-04-28) to not use errno at all, the two
were a part of the same series.

So this was only ever needed for an intra-series bugfix, and we kept
it around for no reason. Removing it makes subsequent commits where we
refactor code surrounding verify_lock() (which may use errno) easier
to reason about.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index a4e9344ac8b..abed0572fea 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -880,9 +880,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,
 			       mustexist ? RESOLVE_REF_READING : 0,
 			       &lock->old_oid, NULL)) {
 		if (old_oid) {
-			int save_errno = errno;
 			strbuf_addf(err, "can't verify ref '%s'", lock->ref_name);
-			errno = save_errno;
 			return -1;
 		} else {
 			oidclr(&lock->old_oid);
@@ -894,7 +892,6 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,
 			    lock->ref_name,
 			    oid_to_hex(&lock->old_oid),
 			    oid_to_hex(old_oid));
-		errno = EBUSY;
 		return -1;
 	}
 	return 0;
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 09/17] refs file-backend.c: deal with errno directly in verify_lock()
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (7 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock() Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 10/17] refs API: remove refs_read_ref_full() wrapper Ævar Arnfjörð Bjarmason
                             ` (9 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Instead of using the refs_read_ref_full() wrapper function let's use
the refs_resolve_ref_unsafe_with_errno() function it resolves to (note
the inverted return values).

Per a preceding change to remove an "errno" assignment from
verify_lock() we can explicitly ignore errno here, we use the strbuf
"err" API instead.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index abed0572fea..e3d27001b86 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -874,11 +874,12 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,
 		       const struct object_id *old_oid, int mustexist,
 		       struct strbuf *err)
 {
+	int ignore_errno;
 	assert(err);
 
-	if (refs_read_ref_full(ref_store, lock->ref_name,
-			       mustexist ? RESOLVE_REF_READING : 0,
-			       &lock->old_oid, NULL)) {
+	if (!refs_resolve_ref_unsafe_with_errno(ref_store, lock->ref_name,
+						mustexist ? RESOLVE_REF_READING : 0,
+						&lock->old_oid, NULL, &ignore_errno)) {
 		if (old_oid) {
 			strbuf_addf(err, "can't verify ref '%s'", lock->ref_name);
 			return -1;
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 10/17] refs API: remove refs_read_ref_full() wrapper
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (8 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 09/17] refs file-backend.c: deal with errno directly " Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 11/17] refs API: make resolve_gitlink_ref() not set errno Ævar Arnfjörð Bjarmason
                             ` (8 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Remove the refs_read_ref_full() wrapper in favor of migrating various
refs.c API users to the underlying
refs_resolve_ref_unsafe_with_errno() function. A careful reading of
these callers shows that the callers of this function did not care
about "errno", by moving away from the refs_resolve_ref_unsafe()
wrapper we can be sure that nothing relies on it anymore.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               | 22 ++++++++++------------
 refs.h               |  2 --
 refs/files-backend.c | 30 +++++++++++++++++++-----------
 worktree.c           |  9 +++++----
 4 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/refs.c b/refs.c
index 1dacb5fe27b..91320f092fb 100644
--- a/refs.c
+++ b/refs.c
@@ -294,20 +294,17 @@ struct ref_filter {
 	void *cb_data;
 };
 
-int refs_read_ref_full(struct ref_store *refs, const char *refname,
-		       int resolve_flags, struct object_id *oid, int *flags)
+int read_ref_full(const char *refname, int resolve_flags, struct object_id *oid, int *flags)
 {
-	if (refs_resolve_ref_unsafe(refs, refname, resolve_flags, oid, flags))
+	int ignore_errno;
+	struct ref_store *refs = get_main_ref_store(the_repository);
+
+	if (refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+					       oid, flags, &ignore_errno))
 		return 0;
 	return -1;
 }
 
-int read_ref_full(const char *refname, int resolve_flags, struct object_id *oid, int *flags)
-{
-	return refs_read_ref_full(get_main_ref_store(the_repository), refname,
-				  resolve_flags, oid, flags);
-}
-
 int read_ref(const char *refname, struct object_id *oid)
 {
 	return read_ref_full(refname, RESOLVE_REF_READING, oid, NULL);
@@ -1398,9 +1395,10 @@ int refs_head_ref(struct ref_store *refs, each_ref_fn fn, void *cb_data)
 {
 	struct object_id oid;
 	int flag;
-
-	if (!refs_read_ref_full(refs, "HEAD", RESOLVE_REF_READING,
-				&oid, &flag))
+	int ignore_errno;
+	
+	if (refs_resolve_ref_unsafe_with_errno(refs, "HEAD", RESOLVE_REF_READING,
+					       &oid, &flag, &ignore_errno))
 		return fn("HEAD", &oid, flag, cb_data);
 
 	return 0;
diff --git a/refs.h b/refs.h
index 344a3c51a8c..d80184dd8e0 100644
--- a/refs.h
+++ b/refs.h
@@ -88,8 +88,6 @@ char *refs_resolve_refdup(struct ref_store *refs,
 char *resolve_refdup(const char *refname, int resolve_flags,
 		     struct object_id *oid, int *flags);
 
-int refs_read_ref_full(struct ref_store *refs, const char *refname,
-		       int resolve_flags, struct object_id *oid, int *flags);
 int read_ref_full(const char *refname, int resolve_flags,
 		  struct object_id *oid, int *flags);
 int read_ref(const char *refname, struct object_id *oid);
diff --git a/refs/files-backend.c b/refs/files-backend.c
index e3d27001b86..b3bc2f57387 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1342,6 +1342,7 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,
 	struct strbuf tmp_renamed_log = STRBUF_INIT;
 	int log, ret;
 	struct strbuf err = STRBUF_INIT;
+	int ignore_errno;
 
 	files_reflog_path(refs, &sb_oldref, oldrefname);
 	files_reflog_path(refs, &sb_newref, newrefname);
@@ -1399,9 +1400,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,
 	 * the safety anyway; we want to delete the reference whatever
 	 * its current value.
 	 */
-	if (!copy && !refs_read_ref_full(&refs->base, newrefname,
-				RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
-				NULL, NULL) &&
+	if (!copy && refs_resolve_ref_unsafe_with_errno(&refs->base, newrefname,
+							RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
+							NULL, NULL, &ignore_errno) &&
 	    refs_delete_ref(&refs->base, NULL, newrefname,
 			    NULL, REF_NO_DEREF)) {
 		if (errno == EISDIR) {
@@ -1815,9 +1816,12 @@ static void update_symref_reflog(struct files_ref_store *refs,
 {
 	struct strbuf err = STRBUF_INIT;
 	struct object_id new_oid;
+	int ignore_errno;
+
 	if (logmsg &&
-	    !refs_read_ref_full(&refs->base, target,
-				RESOLVE_REF_READING, &new_oid, NULL) &&
+	    refs_resolve_ref_unsafe_with_errno(&refs->base, target,
+					       RESOLVE_REF_READING, &new_oid, NULL,
+					       &ignore_errno) &&
 	    files_log_ref_write(refs, refname, &lock->old_oid,
 				&new_oid, logmsg, 0, &err)) {
 		error("%s", err.buf);
@@ -2093,6 +2097,7 @@ static int files_reflog_iterator_advance(struct ref_iterator *ref_iterator)
 		(struct files_reflog_iterator *)ref_iterator;
 	struct dir_iterator *diter = iter->dir_iterator;
 	int ok;
+	int ignore_errno;
 
 	while ((ok = dir_iterator_advance(diter)) == ITER_OK) {
 		int flags;
@@ -2104,9 +2109,10 @@ static int files_reflog_iterator_advance(struct ref_iterator *ref_iterator)
 		if (ends_with(diter->basename, ".lock"))
 			continue;
 
-		if (refs_read_ref_full(iter->ref_store,
-				       diter->relative_path, 0,
-				       &iter->oid, &flags)) {
+		if (!refs_resolve_ref_unsafe_with_errno(iter->ref_store,
+							diter->relative_path, 0,
+							&iter->oid, &flags,
+							&ignore_errno)) {
 			error("bad ref for %s", diter->path.buf);
 			continue;
 		}
@@ -2450,9 +2456,11 @@ static int lock_ref_for_update(struct files_ref_store *refs,
 			 * the transaction, so we have to read it here
 			 * to record and possibly check old_oid:
 			 */
-			if (refs_read_ref_full(&refs->base,
-					       referent.buf, 0,
-					       &lock->old_oid, NULL)) {
+			int ignore_errno;
+			if (!refs_resolve_ref_unsafe_with_errno(&refs->base,
+								referent.buf, 0,
+								&lock->old_oid, NULL,
+								&ignore_errno)) {
 				if (update->flags & REF_HAVE_OLD) {
 					strbuf_addf(err, "cannot lock ref '%s': "
 						    "error reading reference",
diff --git a/worktree.c b/worktree.c
index 237517baee6..1dcdb481d61 100644
--- a/worktree.c
+++ b/worktree.c
@@ -562,16 +562,17 @@ int other_head_refs(each_ref_fn fn, void *cb_data)
 		struct worktree *wt = *p;
 		struct object_id oid;
 		int flag;
+		int ignore_errno;
 
 		if (wt->is_current)
 			continue;
 
 		strbuf_reset(&refname);
 		strbuf_worktree_ref(wt, &refname, "HEAD");
-		if (!refs_read_ref_full(get_main_ref_store(the_repository),
-					refname.buf,
-					RESOLVE_REF_READING,
-					&oid, &flag))
+		if (refs_resolve_ref_unsafe_with_errno(get_main_ref_store(the_repository),
+						       refname.buf,
+						       RESOLVE_REF_READING,
+						       &oid, &flag, &ignore_errno))
 			ret = fn(refname.buf, &oid, flag, cb_data);
 		if (ret)
 			break;
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 11/17] refs API: make resolve_gitlink_ref() not set errno
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (9 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 10/17] refs API: remove refs_read_ref_full() wrapper Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 12/17] refs API: make refs_resolve_ref_unsafe() static Ævar Arnfjörð Bjarmason
                             ` (7 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

I have carefully read the upstream callers of resolve_gitlink_ref()
and determined that they don't care about errno. So let's move away
from the errno-setting refs_resolve_ref_unsafe() wrapper to
refs_resolve_ref_unsafe_with_errno(), and explicitly ignore the errno
it sets for us.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 91320f092fb..7a9cdb1d720 100644
--- a/refs.c
+++ b/refs.c
@@ -1814,14 +1814,15 @@ int resolve_gitlink_ref(const char *submodule, const char *refname,
 {
 	struct ref_store *refs;
 	int flags;
+	int ignore_errno;
 
 	refs = get_submodule_ref_store(submodule);
 
 	if (!refs)
 		return -1;
 
-	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
-	    is_null_oid(oid))
+	if (!refs_resolve_ref_unsafe_with_errno(refs, refname, 0, oid, &flags,
+		    &ignore_errno) || is_null_oid(oid))
 		return -1;
 	return 0;
 }
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 12/17] refs API: make refs_resolve_ref_unsafe() static
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (10 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 11/17] refs API: make resolve_gitlink_ref() not set errno Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 13/17] refs API: make refs_resolve_refdup() not set errno Ævar Arnfjörð Bjarmason
                             ` (6 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Remove public users of refs_resolve_ref_unsafe() over to
refs_resolve_ref_unsafe_with_errno(), and explicitly ignore the
provided errno in those callers. At that point the only remaining
users are the legacy functions in refs.c itself, so we can make the
wrapper static.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                    | 29 +++++++++++++++--------------
 refs.h                    |  8 +-------
 refs/files-backend.c      | 24 ++++++++++++++----------
 t/helper/test-ref-store.c |  5 +++--
 worktree.c                | 18 ++++++++++++------
 5 files changed, 45 insertions(+), 39 deletions(-)

diff --git a/refs.c b/refs.c
index 7a9cdb1d720..214f3f4f79f 100644
--- a/refs.c
+++ b/refs.c
@@ -267,6 +267,21 @@ int ref_resolves_to_object(const char *refname,
 	return 1;
 }
 
+static const char *refs_resolve_ref_unsafe(struct ref_store *refs,
+					   const char *refname,
+					   int resolve_flags,
+					   struct object_id *oid, int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 char *refs_resolve_refdup(struct ref_store *refs,
 			  const char *refname, int resolve_flags,
 			  struct object_id *oid, int *flags)
@@ -1780,20 +1795,6 @@ const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 	return NULL;
 }
 
-const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
-				    int resolve_flags, struct object_id *oid,
-				    int *flags)
-{
-	int failure_errno = 0;
-	const char *refn;
-	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
-						  oid, flags, &failure_errno);
-	if (!refn)
-		/* For unmigrated legacy callers */
-		errno = failure_errno;
-	return refn;
-}
-
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
diff --git a/refs.h b/refs.h
index d80184dd8e0..a3284adb8f8 100644
--- a/refs.h
+++ b/refs.h
@@ -63,14 +63,8 @@ struct worktree;
 #define RESOLVE_REF_NO_RECURSE 0x02
 #define RESOLVE_REF_ALLOW_BAD_NAME 0x04
 
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid,
-				    int *flags);
 /**
- * refs_resolve_ref_unsafe_with_errno() is like
- * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * refs_resolve_ref_unsafe_with_errno() provides access to errno code that
  * lead to a failure. We guarantee that errno is set to a meaningful
  * value on non-zero return.
  */
diff --git a/refs/files-backend.c b/refs/files-backend.c
index b3bc2f57387..158c0146484 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -280,10 +280,11 @@ static void loose_fill_ref_dir(struct ref_store *ref_store,
 					 create_dir_entry(dir->cache, refname.buf,
 							  refname.len, 1));
 		} else {
-			if (!refs_resolve_ref_unsafe(&refs->base,
-						     refname.buf,
-						     RESOLVE_REF_READING,
-						     &oid, &flag)) {
+			int ignore_errno;
+			if (!refs_resolve_ref_unsafe_with_errno(&refs->base,
+								refname.buf,
+								RESOLVE_REF_READING,
+								&oid, &flag, &ignore_errno)) {
 				oidclr(&oid);
 				flag |= REF_ISBROKEN;
 			} else if (is_null_oid(&oid)) {
@@ -1354,9 +1355,9 @@ static int files_copy_or_rename_ref(struct ref_store *ref_store,
 		goto out;
 	}
 
-	if (!refs_resolve_ref_unsafe(&refs->base, oldrefname,
-				     RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
-				&orig_oid, &flag)) {
+	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, oldrefname,
+						RESOLVE_REF_READING | RESOLVE_REF_NO_RECURSE,
+						&orig_oid, &flag, &ignore_errno)) {
 		ret = error("refname %s not found", oldrefname);
 		goto out;
 	}
@@ -1769,10 +1770,13 @@ static int commit_ref_update(struct files_ref_store *refs,
 		 */
 		int head_flag;
 		const char *head_ref;
+		int ignore_errno;
 
-		head_ref = refs_resolve_ref_unsafe(&refs->base, "HEAD",
-						   RESOLVE_REF_READING,
-						   NULL, &head_flag);
+		head_ref = refs_resolve_ref_unsafe_with_errno(&refs->base,
+							      "HEAD",
+							      RESOLVE_REF_READING,
+							      NULL, &head_flag,
+							      &ignore_errno);
 		if (head_ref && (head_flag & REF_ISSYMREF) &&
 		    !strcmp(head_ref, lock->ref_name)) {
 			struct strbuf log_err = STRBUF_INIT;
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index bba5f841c6a..ca69a793171 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -123,9 +123,10 @@ static int cmd_resolve_ref(struct ref_store *refs, const char **argv)
 	int resolve_flags = arg_flags(*argv++, "resolve-flags");
 	int flags;
 	const char *ref;
+	int ignore_errno;
 
-	ref = refs_resolve_ref_unsafe(refs, refname, resolve_flags,
-				      &oid, &flags);
+	ref = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						 &oid, &flags, &ignore_errno);
 	printf("%s %s 0x%x\n", oid_to_hex(&oid), ref ? ref : "(null)", flags);
 	return ref ? 0 : 1;
 }
diff --git a/worktree.c b/worktree.c
index 1dcdb481d61..1dbdec82461 100644
--- a/worktree.c
+++ b/worktree.c
@@ -28,11 +28,13 @@ static void add_head_info(struct worktree *wt)
 {
 	int flags;
 	const char *target;
+	int ignore_errno;
 
-	target = refs_resolve_ref_unsafe(get_worktree_ref_store(wt),
-					 "HEAD",
-					 0,
-					 &wt->head_oid, &flags);
+	target = refs_resolve_ref_unsafe_with_errno(get_worktree_ref_store(wt),
+						    "HEAD",
+						    0,
+						    &wt->head_oid, &flags,
+						    &ignore_errno);
 	if (!target)
 		return;
 
@@ -417,6 +419,7 @@ const struct worktree *find_shared_symref(const char *symref,
 		const char *symref_target;
 		struct ref_store *refs;
 		int flags;
+		int ignore_errno;
 
 		if (wt->is_bare)
 			continue;
@@ -433,8 +436,11 @@ const struct worktree *find_shared_symref(const char *symref,
 		}
 
 		refs = get_worktree_ref_store(wt);
-		symref_target = refs_resolve_ref_unsafe(refs, symref, 0,
-							NULL, &flags);
+		symref_target = refs_resolve_ref_unsafe_with_errno(refs,
+								   symref,
+								   0, NULL,
+								   &flags,
+								   &ignore_errno);
 		if ((flags & REF_ISSYMREF) &&
 		    symref_target && !strcmp(symref_target, target)) {
 			existing = wt;
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 13/17] refs API: make refs_resolve_refdup() not set errno
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (11 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 12/17] refs API: make refs_resolve_ref_unsafe() static Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 14/17] refs API: make refs_ref_exists() " Ævar Arnfjörð Bjarmason
                             ` (5 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Move refs_resolve_refdup() from the legacy refs_resolve_ref_unsafe()
to the new refs_resolve_ref_unsafe_with_errno(). I have read its
callers and determined that they don't care about errno being set.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 214f3f4f79f..45b51c0a532 100644
--- a/refs.c
+++ b/refs.c
@@ -287,9 +287,11 @@ char *refs_resolve_refdup(struct ref_store *refs,
 			  struct object_id *oid, int *flags)
 {
 	const char *result;
+	int ignore_errno;
 
-	result = refs_resolve_ref_unsafe(refs, refname, resolve_flags,
-					 oid, flags);
+	result = refs_resolve_ref_unsafe_with_errno(refs, refname,
+						    resolve_flags, oid, flags,
+						    &ignore_errno);
 	return xstrdup_or_null(result);
 }
 
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 14/17] refs API: make refs_ref_exists() not set errno
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (12 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 13/17] refs API: make refs_resolve_refdup() not set errno Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() " Ævar Arnfjörð Bjarmason
                             ` (4 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Move refs_ref_exists from the legacy refs_resolve_ref_unsafe() to the
new refs_resolve_ref_unsafe_with_errno(). I have read its callers and
determined that they don't care about errno being set, in particular:

    git grep -W -w -e refs_ref_exists -e ref_exists

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 45b51c0a532..0364b68f8fa 100644
--- a/refs.c
+++ b/refs.c
@@ -329,7 +329,10 @@ int read_ref(const char *refname, struct object_id *oid)
 
 int refs_ref_exists(struct ref_store *refs, const char *refname)
 {
-	return !!refs_resolve_ref_unsafe(refs, refname, RESOLVE_REF_READING, NULL, NULL);
+	int ignore_errno;
+	return !!refs_resolve_ref_unsafe_with_errno(refs, refname,
+						    RESOLVE_REF_READING, NULL,
+						    NULL, &ignore_errno);
 }
 
 int ref_exists(const char *refname)
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (13 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 14/17] refs API: make refs_ref_exists() " Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-13  8:13             ` Han-Wen Nienhuys
  2021-07-11 16:30           ` [PATCH v6? 16/17] refs API: make expand_ref() and repo_dwim_log() " Ævar Arnfjörð Bjarmason
                             ` (3 subsequent siblings)
  18 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Change the resolve_ref_unsafe() wrapper function to use
refs_resolve_ref_unsafe_with_errno() directly.

From a reading of the callers I determined that the only one who cared
about errno was a sequencer.c caller added in e47c6cafcb5 (commit:
move print_commit_summary() to libgit, 2017-11-24), I'm migrating it
to using refs_resolve_ref_unsafe_with_errno() directly.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c      |  7 +++++--
 sequencer.c | 10 ++++++++--
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index 0364b68f8fa..1d44027497e 100644
--- a/refs.c
+++ b/refs.c
@@ -1811,8 +1811,11 @@ int refs_init_db(struct strbuf *err)
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags)
 {
-	return refs_resolve_ref_unsafe(get_main_ref_store(the_repository), refname,
-				       resolve_flags, oid, flags);
+	struct ref_store *refs = get_main_ref_store(the_repository);
+	int ignore_errno;
+
+	return refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &ignore_errno);
 }
 
 int resolve_gitlink_ref(const char *submodule, const char *refname,
diff --git a/sequencer.c b/sequencer.c
index 0bec01cf38e..c4772413b86 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1267,6 +1267,8 @@ void print_commit_summary(struct repository *r,
 	struct pretty_print_context pctx = {0};
 	struct strbuf author_ident = STRBUF_INIT;
 	struct strbuf committer_ident = STRBUF_INIT;
+	struct ref_store *refs;
+	int resolve_errno;
 
 	commit = lookup_commit(r, oid);
 	if (!commit)
@@ -1316,9 +1318,13 @@ void print_commit_summary(struct repository *r,
 	rev.diffopt.break_opt = 0;
 	diff_setup_done(&rev.diffopt);
 
-	head = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
-	if (!head)
+	refs = get_main_ref_store(the_repository);
+	head = refs_resolve_ref_unsafe_with_errno(refs, "HEAD", 0, NULL, NULL,
+						  &resolve_errno);
+	if (!head) {
+		errno = resolve_errno;
 		die_errno(_("unable to resolve HEAD after creating commit"));
+	}
 	if (!strcmp(head, "HEAD"))
 		head = _("detached HEAD");
 	else
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 16/17] refs API: make expand_ref() and repo_dwim_log() not set errno
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (14 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() " Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-11 16:30           ` [PATCH v6? 17/17] refs API: don't leak "errno" in run_transaction_hook() Ævar Arnfjörð Bjarmason
                             ` (2 subsequent siblings)
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

The use of these two is rather trivial, and it's easy to see none of
their callers care about errno. So let's move them from
refs_resolve_ref_unsafe() to refs_resolve_ref_unsafe_with_errno(),
these were the last two callers, so we can get rid of that wrapper
function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 32 +++++++++++---------------------
 1 file changed, 11 insertions(+), 21 deletions(-)

diff --git a/refs.c b/refs.c
index 1d44027497e..3357ba14e1d 100644
--- a/refs.c
+++ b/refs.c
@@ -267,21 +267,6 @@ int ref_resolves_to_object(const char *refname,
 	return 1;
 }
 
-static const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-					   const char *refname,
-					   int resolve_flags,
-					   struct object_id *oid, int *flags)
-{
-	int failure_errno = 0;
-	const char *refn;
-	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
-						  oid, flags, &failure_errno);
-	if (!refn)
-		/* For unmigrated legacy callers */
-		errno = failure_errno;
-	return refn;
-}
-
 char *refs_resolve_refdup(struct ref_store *refs,
 			  const char *refname, int resolve_flags,
 			  struct object_id *oid, int *flags)
@@ -675,13 +660,16 @@ int expand_ref(struct repository *repo, const char *str, int len,
 		struct object_id oid_from_ref;
 		struct object_id *this_result;
 		int flag;
+		struct ref_store *refs = get_main_ref_store(repo);
+		int ignore_errno;
 
 		this_result = refs_found ? &oid_from_ref : oid;
 		strbuf_reset(&fullref);
 		strbuf_addf(&fullref, *p, len, str);
-		r = refs_resolve_ref_unsafe(get_main_ref_store(repo),
-					    fullref.buf, RESOLVE_REF_READING,
-					    this_result, &flag);
+		r = refs_resolve_ref_unsafe_with_errno(refs, fullref.buf,
+						       RESOLVE_REF_READING,
+						       this_result, &flag,
+						       &ignore_errno);
 		if (r) {
 			if (!refs_found++)
 				*ref = xstrdup(r);
@@ -710,12 +698,14 @@ int repo_dwim_log(struct repository *r, const char *str, int len,
 	for (p = ref_rev_parse_rules; *p; p++) {
 		struct object_id hash;
 		const char *ref, *it;
+		int ignore_errno;
 
 		strbuf_reset(&path);
 		strbuf_addf(&path, *p, len, str);
-		ref = refs_resolve_ref_unsafe(refs, path.buf,
-					      RESOLVE_REF_READING,
-					      &hash, NULL);
+		ref = refs_resolve_ref_unsafe_with_errno(refs, path.buf,
+							 RESOLVE_REF_READING,
+							 &hash, NULL,
+							 &ignore_errno);
 		if (!ref)
 			continue;
 		if (refs_reflog_exists(refs, path.buf))
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v6? 17/17] refs API: don't leak "errno" in run_transaction_hook()
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (15 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 16/17] refs API: make expand_ref() and repo_dwim_log() " Ævar Arnfjörð Bjarmason
@ 2021-07-11 16:30           ` Ævar Arnfjörð Bjarmason
  2021-07-13  8:28           ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Han-Wen Nienhuys
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  18 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-11 16:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

In run_transaction_hook() we've checked errno since 67541597670 (refs:
implement reference transaction hook, 2020-06-19), let's reset errno
afterwards to make sure nobody using refs.c directly or indirectly
relies on it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 3357ba14e1d..f1477eed27a 100644
--- a/refs.c
+++ b/refs.c
@@ -2108,8 +2108,11 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 			    update->refname);
 
 		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
+			if (errno != EPIPE) {
+				/* Don't leak errno outside this API */
+				errno = 0;
 				ret = -1;
+			}
 			break;
 		}
 	}
-- 
2.32.0-dev


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-11 11:38           ` Ævar Arnfjörð Bjarmason
@ 2021-07-13  8:00             ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-13  8:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Sun, Jul 11, 2021 at 1:48 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:
>
> >  /*
> >   * Locks a ref returning the lock on success and NULL on failure.
> > - * On failure errno is set to something meaningful.
> >   */
> >  static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
> >                                          const char *refname,
> > @@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
> >  {
> >       struct strbuf ref_file = STRBUF_INIT;
> >       struct ref_lock *lock;
> > -     int last_errno = 0;
> >       int mustexist = (old_oid && !is_null_oid(old_oid));
> >       int resolve_flags = RESOLVE_REF_NO_RECURSE;
> >       int resolved;
> > @@ -949,7 +947,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
> >                * to remain.
> >                */
> >               if (remove_empty_directories(&ref_file)) {
> > -                     last_errno = errno;
> >                       if (!refs_verify_refname_available(
> >                                           &refs->base,
> >                                           refname, extras, skip, err))
> > @@ -962,10 +959,13 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
> >                                                    &lock->old_oid, type);
> >       }
> >       if (!resolved) {
> > -             last_errno = errno;
> > +             int last_errno = errno;
> >               if (last_errno != ENOTDIR ||
> > -                 !refs_verify_refname_available(&refs->base, refname,
> > -                                                extras, skip, err))
> > +                 /* in case of D/F conflict, try to generate a better error
> > +                  * message. If that fails, fall back to strerror(ENOTDIR).
> > +                  */
> > +                 !refs_verify_refname_available(&refs->base, refname, extras,
> > +                                                skip, err))
> >                       strbuf_addf(err, "unable to resolve reference '%s': %s",
> >                                   refname, strerror(last_errno));
>
> I don't think it's some dealbreaker and we can move on, but just FWIW I
> think what I mentioned ending in your
> https://lore.kernel.org/git/CAFQ2z_NpyJQLuM70MhJ8K1h2v3QXFuAZRjN=SvSsjnukNRJ8pw@mail.gmail.com/
> is still outstanding.
>
> I.e. you added the comment, which is just says what the error emitting
> looks like, that's all well & good.
>
> But what I was pointing out that it didn't make sense to do any
> "last_errno" here at all anymore. You pointed to 5b2d8d6f218
> (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
> 2015-05-11), we started setting "last_errno" there, but that was *not*
> to avoid clobbering between the !resolved and the
> strbuf_add(strerror(last_errno)) here, but rather to carry the
> "last_errno" forward to the end of this lock_ref_oid_basic(), because
> other things (after this hunk) might reset/clear errno.

I disagree. In your suggested change

> +       if (!resolved &&
> +           (errno != ENOTDIR ||
> +            /* in case of D/F conflict, try to generate a better error
> +             * message. If that fails, fall back to strerror(ENOTDIR).
> +             */
> +            !refs_verify_refname_available(&refs->base, refname, extras,
> +                                           skip, err))) {
> +               strbuf_addf(err, "unable to resolve reference '%s': %s",
> +                           refname, strerror(errno));
>                 goto error_return;

the refs_verify_refname_available() call happens only if
errno==ENOTDIR. The call might clobber the ENOTDIR errno and then
fail. Then we'd be printing the last errno that
refs_verify_refname_available() saw, which may be different from
ENOTDIR.

Other than that, for clarity's sake, it's always better to avoid the
use of global errno if we can.

--
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-11 11:59           ` Ævar Arnfjörð Bjarmason
@ 2021-07-13  8:02             ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-13  8:02 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Han-Wen Nienhuys via GitGitGadget, git, Jonathan Tan, Han-Wen Nienhuys

On Sun, Jul 11, 2021 at 2:00 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Wed, Jul 07 2021, Han-Wen Nienhuys via GitGitGadget wrote:
>
> > From: Han-Wen Nienhuys <hanwen@google.com>
> >
> > This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.
> > [...]
> > +const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
>
> This and the subsequent commit don't compile for me, because this lacks
> a prototype that you finally add in 6/6.

Whoops! thanks for correcting.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock()
  2021-07-11 16:30           ` [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock() Ævar Arnfjörð Bjarmason
@ 2021-07-13  8:08             ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-13  8:08 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Jonathan Tan

On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> This EBUSY was set in 835e3c992fd (refs.c: verify_lock should set
> errno to something meaningful, 2014-06-20) to fix a bug in code that
> was later refactored in cd94f765720 (fetch.c: change s_update_ref to
> use a ref transaction, 2014-04-28) to not use errno at all, the two
> were a part of the same series.
>
> So this was only ever needed for an intra-series bugfix, and we kept
> it around for no reason. Removing it makes subsequent commits where we
> refactor code surrounding verify_lock() (which may use errno) easier
> to reason about.
> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index a4e9344ac8b..abed0572fea 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -880,9 +880,7 @@ static int verify_lock(struct ref_store *ref_store, struct ref_lock *lock,
>                                mustexist ? RESOLVE_REF_READING : 0,
>                                &lock->old_oid, NULL)) {
>                 if (old_oid) {
> -                       int save_errno = errno;
>                         strbuf_addf(err, "can't verify ref '%s'", lock->ref_name);
> -                       errno = save_errno;

your message talks about EBUSY, and that change looks good, but how
does it relate to this change?


-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno
  2021-07-11 16:30           ` [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() " Ævar Arnfjörð Bjarmason
@ 2021-07-13  8:13             ` Han-Wen Nienhuys
  2021-07-14  8:32               ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-13  8:13 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Jonathan Tan

On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> +       refs = get_main_ref_store(the_repository);
> +       head = refs_resolve_ref_unsafe_with_errno(refs, "HEAD", 0, NULL, NULL,
> +                                                 &resolve_errno);
> +       if (!head) {
> +               errno = resolve_errno;

This adds another place where we'd use global errno to communicate.
Isn't there a die() function that takes an errno value explicitly?

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (16 preceding siblings ...)
  2021-07-11 16:30           ` [PATCH v6? 17/17] refs API: don't leak "errno" in run_transaction_hook() Ævar Arnfjörð Bjarmason
@ 2021-07-13  8:28           ` Han-Wen Nienhuys
  2021-07-13 18:26             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  18 siblings, 1 reply; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-13  8:28 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Jonathan Tan

On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> After starting a review of v5 of Han-Wen's where I was coming up with
> squashes-on-top I continued with it and saw if I could get rid of
> errno setting entirely in refs.

thanks for taking this further! Is there a place where I can pull this
code (applying patches is bothersome).

It adds resolve_refs_unsafe_with_errno() to the  public API in refs.h,
and I think it has an ugly signature, but I suppose it's better than
having the output be implicit through the global errno variable.

I read over your patches, and they seem OK to me, modulo small
comments I posted.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely
  2021-07-13  8:28           ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Han-Wen Nienhuys
@ 2021-07-13 18:26             ` Ævar Arnfjörð Bjarmason
  2021-07-14  8:38               ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-13 18:26 UTC (permalink / raw)
  To: Han-Wen Nienhuys; +Cc: git, Junio C Hamano, Jeff King, Jonathan Tan


On Tue, Jul 13 2021, Han-Wen Nienhuys wrote:

> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> After starting a review of v5 of Han-Wen's where I was coming up with
>> squashes-on-top I continued with it and saw if I could get rid of
>> errno setting entirely in refs.
>
> thanks for taking this further! Is there a place where I can pull this
> code (applying patches is bothersome).

I've got it pushed to avar-review/pr-git-1012/hanwen/einval-sideband-v5
at https://github.com/avar/git.git :
https://github.com/avar/git/tree/avar-review/pr-git-1012%2Fhanwen%2Feinval-sideband-v5

> It adds resolve_refs_unsafe_with_errno() to the  public API in refs.h,
> and I think it has an ugly signature, but I suppose it's better than
> having the output be implicit through the global errno variable.

Yeah it is a bit ugly, perhaps we should just end up using the old name
once there's no callers of the old one?

> I read over your patches, and they seem OK to me, modulo small
> comments I posted.

Thanks, don't have time to look it over in detail now, will reply later.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() not set errno
  2021-07-13  8:13             ` Han-Wen Nienhuys
@ 2021-07-14  8:32               ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14  8:32 UTC (permalink / raw)
  To: Han-Wen Nienhuys; +Cc: git, Junio C Hamano, Jeff King, Jonathan Tan


On Tue, Jul 13 2021, Han-Wen Nienhuys wrote:

> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> +       refs = get_main_ref_store(the_repository);
>> +       head = refs_resolve_ref_unsafe_with_errno(refs, "HEAD", 0, NULL, NULL,
>> +                                                 &resolve_errno);
>> +       if (!head) {
>> +               errno = resolve_errno;
>
> This adds another place where we'd use global errno to communicate.
> Isn't there a die() function that takes an errno value explicitly?

No, not that I know of. I suppose we could refactor fmt_with_err() in
usage.c to somehow be exposed and add e.g. a:

    die_saved_errno(&saved_errno, "msg %s", ....);

But I really don't see the point.

The trouble with using errno is generally introducing undesired action
at a distance, here we're setting it immediately before calling
die_errno(), so there's not much action or distance we should/could be
worried about.

E.g. xmkstemp() in wrapper.c uses this pattern, i.e.:

    errno = saved_errno;
    die_errno(...)

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely
  2021-07-13 18:26             ` Ævar Arnfjörð Bjarmason
@ 2021-07-14  8:38               ` Ævar Arnfjörð Bjarmason
  2021-07-14  8:43                 ` Han-Wen Nienhuys
  0 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14  8:38 UTC (permalink / raw)
  To: Han-Wen Nienhuys; +Cc: git, Junio C Hamano, Jeff King, Jonathan Tan


On Tue, Jul 13 2021, Ævar Arnfjörð Bjarmason wrote:

> On Tue, Jul 13 2021, Han-Wen Nienhuys wrote:
>
>> On Sun, Jul 11, 2021 at 6:30 PM Ævar Arnfjörð Bjarmason
>> <avarab@gmail.com> wrote:
>>>
>>> After starting a review of v5 of Han-Wen's where I was coming up with
>>> squashes-on-top I continued with it and saw if I could get rid of
>>> errno setting entirely in refs.
>>
>> thanks for taking this further! Is there a place where I can pull this
>> code (applying patches is bothersome).
>
> I've got it pushed to avar-review/pr-git-1012/hanwen/einval-sideband-v5
> at https://github.com/avar/git.git :
> https://github.com/avar/git/tree/avar-review/pr-git-1012%2Fhanwen%2Feinval-sideband-v5
>
>> It adds resolve_refs_unsafe_with_errno() to the  public API in refs.h,
>> and I think it has an ugly signature, but I suppose it's better than
>> having the output be implicit through the global errno variable.
>
> Yeah it is a bit ugly, perhaps we should just end up using the old name
> once there's no callers of the old one?

I looked at doing this, I don't think it's worth it.

For my additional patches on top we need to somehow migrate all callers
incrementally, so we could
s/refs_resolve_ref_unsafe_with_errno/refs_resolve_ref_unsafe/g at the
end, but it just seems like a lot more churn, and for the patches to be
self-contained I'd need to re-indent all the argument lists again.

It's a relatively obscure & scary API. I think it's OK that it's got a
bit of a longer/more verbose name as a side-effect of the migration.

And in any case with reftable I suspect that we'll end up refactoring
further down the line sooner than later, i.e. to pass a "struct strbuf
*err" or something, so getting it 100% right here seems premature.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v6? 00/17] refs API: get rid of errno setting entirely
  2021-07-14  8:38               ` Ævar Arnfjörð Bjarmason
@ 2021-07-14  8:43                 ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-07-14  8:43 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Jonathan Tan

On Wed, Jul 14, 2021 at 10:40 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> >> It adds resolve_refs_unsafe_with_errno() to the  public API in refs.h,
> >> and I think it has an ugly signature, but I suppose it's better than
> >> having the output be implicit through the global errno variable.
> >
> > Yeah it is a bit ugly, perhaps we should just end up using the old name
> > once there's no callers of the old one?
>
> I looked at doing this, I don't think it's worth it.
..
> And in any case with reftable I suspect that we'll end up refactoring
> further down the line sooner than later, i.e. to pass a "struct strbuf
> *err" or something, so getting it 100% right here seems premature.

SGTM.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v7 0/6] refs: cleanup errno sideband ref related functions
  2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
                             ` (17 preceding siblings ...)
  2021-07-13  8:28           ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Han-Wen Nienhuys
@ 2021-07-14 11:43           ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
                               ` (6 more replies)
  18 siblings, 7 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

Per the discussion & feedback on the "v6?" at [1] this is a version of
Han-Wen's patches which:

 A. Goes on top of my just-submitted one-patch
    https://lore.kernel.org/git/patch-1.1-de0838fe99-20210714T111351Z-avarab@gmail.com
    being able to remove that code in lock_ref_oid_basic() makes
    things simpler, the BUG() I added to that branch in my "v6?" is
    gone.

    That change simplifies this series quite a bit, more than one
    patch in this one dealt with that now-removed codepath.

 B. Rebases this on top of that

 C. I ejected my long series of RFC-esque patches at the end.

    Per [2] it looks like Han-Wen is happy to have some version of my
    re-roll picked up, but I hardly think it's fair that I inflate
    other people's serieses with a bunch of fixes on top, I cause
    myself enough trouble with that as it is :)

    The value of those patches of mine on top is to assert that the
    changes in this series are correct.

    But I think to just get this in it's enough that that work exists
    and that we did the full errno conversion and could demonstrate
    that there weren't bugs as a result of it.

    For now in 04/06 here we leave the "For unmigrated legacy callers"
    assignment of "errno" in refs_resolve_ref_unsafe().

    I'll just submit that cleanup series on top of this once this
    lands.

1. http://lore.kernel.org/git/cover-00.17-00000000000-20210711T162803Z-avarab@gmail.com
2. http://lore.kernel.org/git/CAFQ2z_P=9Suh0kO6E44hUOyNFOAKcmcUg_x6AQFP9jhHrBA6RQ@mail.gmail.com

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

 refs.c                | 69 ++++++++++++++++++++----------
 refs.h                | 11 +++++
 refs/debug.c          |  4 +-
 refs/files-backend.c  | 98 +++++++++++++++++++++++--------------------
 refs/packed-backend.c | 15 +++----
 refs/refs-internal.h  | 32 ++++++++------
 6 files changed, 141 insertions(+), 88 deletions(-)

Range-diff against v6:
 1:  57517368c3 =  1:  4beba4443a refs: remove EINVAL errno output from specification of read_raw_ref_fn
 2:  61cf761147 !  2:  fd8e356185 refs/files-backend: stop setting errno from lock_ref_oid_basic
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      	int resolve_flags = RESOLVE_REF_NO_RECURSE;
      	int resolved;
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 		 * to remain.
    - 		 */
    - 		if (remove_empty_directories(&ref_file)) {
    --			last_errno = errno;
    - 			if (!refs_verify_refname_available(
    - 					    &refs->base,
    - 					    refname, extras, skip, err))
    -@@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 						     refname, resolve_flags,
    - 						     &lock->old_oid, type);
    - 	}
    + 	resolved = !!refs_resolve_ref_unsafe(&refs->base,
    + 					     refname, resolve_flags,
    + 					     &lock->old_oid, type);
     -	if (!resolved) {
     -		last_errno = errno;
     -		if (last_errno != ENOTDIR ||
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
     -						   extras, skip, err))
     -			strbuf_addf(err, "unable to resolve reference '%s': %s",
     -				    refname, strerror(last_errno));
    --
     +	if (!resolved &&
     +	    (errno != ENOTDIR ||
     +	     /* in case of D/F conflict, try to generate a better error
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
     +					    skip, err))) {
     +		strbuf_addf(err, "unable to resolve reference '%s': %s",
     +			    refname, strerror(errno));
    + 
      		goto error_return;
      	}
    - 
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      	if (is_null_oid(&lock->old_oid) &&
      	    refs_verify_refname_available(refs->packed_ref_store, refname,
 3:  a4e5f2d02f =  3:  913e72e9ff refs: make errno output explicit for read_raw_ref_fn
 4:  270cda29c3 =  4:  dd191768f6 refs: add failure_errno to refs_read_raw_ref() signature
 5:  f5197cdc0b =  5:  77c53dc0d9 refs: explicitly return failure_errno from parse_loose_ref_contents
 6:  96689e523f !  6:  5d4b1b4ddf refs: make errno output explicit for refs_resolve_ref_unsafe
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
     -	resolved = !!refs_resolve_ref_unsafe(&refs->base,
     -					     refname, resolve_flags,
     -					     &lock->old_oid, type);
    --	if (!resolved && errno == EISDIR) {
     +	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
     +							resolve_flags,
     +							&lock->old_oid, type,
     +							&resolve_errno);
    -+	if (!resolved && resolve_errno == EISDIR) {
    - 		/*
    - 		 * we are trying to lock foo but we used to
    - 		 * have foo/bar which now does not exist;
    -@@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 						     &lock->old_oid, type);
    - 	}
      	if (!resolved &&
     -	    (errno != ENOTDIR ||
     +	    (resolve_errno != ENOTDIR ||
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      					    skip, err))) {
      		strbuf_addf(err, "unable to resolve reference '%s': %s",
     -			    refname, strerror(errno));
    +-
     +			    refname, strerror(resolve_errno));
      		goto error_return;
      	}
 7:  10a40c9244 <  -:  ---------- refs: make errno ignoring explicit in lock_ref_oid_basic()
 8:  ff38a3f193 <  -:  ---------- refs file-backend.c: stop setting "EBUSY" in verify_lock()
 9:  cdec272f06 <  -:  ---------- refs file-backend.c: deal with errno directly in verify_lock()
10:  bd0639945a <  -:  ---------- refs API: remove refs_read_ref_full() wrapper
11:  a359d1533b <  -:  ---------- refs API: make resolve_gitlink_ref() not set errno
12:  07d550015a <  -:  ---------- refs API: make refs_resolve_ref_unsafe() static
13:  73b7049151 <  -:  ---------- refs API: make refs_resolve_refdup() not set errno
14:  1e9de48d85 <  -:  ---------- refs API: make refs_ref_exists() not set errno
15:  446b50280b <  -:  ---------- refs API: make resolve_ref_unsafe() not set errno
16:  37c5b88d7d <  -:  ---------- refs API: make expand_ref() and repo_dwim_log() not set errno
17:  f914df0bb2 <  -:  ---------- refs API: don't leak "errno" in run_transaction_hook()
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
                               ` (5 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c93..f4445e3290 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v7 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 3/6] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
                               ` (4 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

In the case of the "errno != ENOTDIR" case that originates in 5b2d8d6f218
(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
2015-05-11), there the "last_errno" was saved away to return it from
lock_ref_oid_basic(), now that we're no longer doing that we can skip
that entirely and use "errno" directly. A follow-up change will
extract the specific errno we want earlier in this function.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7e4963fd07..45f198c3fc 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -910,7 +910,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -922,7 +921,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
@@ -941,13 +939,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	resolved = !!refs_resolve_ref_unsafe(&refs->base,
 					     refname, resolve_flags,
 					     &lock->old_oid, type);
-	if (!resolved) {
-		last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   extras, skip, err))
-			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+	if (!resolved &&
+	    (errno != ENOTDIR ||
+	     /* in case of D/F conflict, try to generate a better error
+	      * message. If that fails, fall back to strerror(ENOTDIR).
+	      */
+	     !refs_verify_refname_available(&refs->base, refname, extras,
+					    skip, err))) {
+		strbuf_addf(err, "unable to resolve reference '%s': %s",
+			    refname, strerror(errno));
 
 		goto error_return;
 	}
@@ -961,20 +961,17 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
 					  extras, skip, err)) {
-		last_errno = ENOTDIR;
 		goto error_return;
 	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
 
 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
-		last_errno = errno;
 		goto error_return;
 	}
 	goto out;
@@ -985,7 +982,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v7 3/6] refs: make errno output explicit for read_raw_ref_fn
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 4/6] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
                               ` (3 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                |  2 +-
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 29 +++++++++++++++--------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/refs.c b/refs.c
index 8c9490235e..7b3f05a66f 100644
--- a/refs.c
+++ b/refs.c
@@ -1681,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 7db4abccc3..f12413a9bc 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 45f198c3fc..c4b5b70c53 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -541,6 +539,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -611,7 +610,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	if (hold_lock_file_for_update_timeout(
 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
 			    get_files_ref_lock_timeout_ms()) < 0) {
-		if (errno == ENOENT && --attempts_remaining > 0) {
+		int myerr = errno;
+		errno = 0;
+		if (myerr == ENOENT && --attempts_remaining > 0) {
 			/*
 			 * Maybe somebody just deleted one of the
 			 * directories leading to ref_file.  Try
@@ -619,7 +620,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 			 */
 			goto retry;
 		} else {
-			unable_to_lock_message(ref_file.buf, errno, err);
+			unable_to_lock_message(ref_file.buf, myerr, err);
 			goto error_return;
 		}
 	}
@@ -629,9 +630,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -655,7 +656,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -693,13 +694,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index dfecdbc1db..a457f18e93 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e3290..79dfb3af48 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v7 4/6] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                               ` (2 preceding siblings ...)
  2021-07-14 11:43             ` [PATCH v7 3/6] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
                               ` (2 subsequent siblings)
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Some of our callers explicitly do not care about the errno, rather
than understanding NULL let's have them declare that they don't care
by passing in an "ignore_errno". There's only three of them, and using
that pattern will make it more obvious that they want to throw away
data, let's also add a comment to one of the callers about why we'd
like to ignore the errno.

Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
has a large set of legacy callers, so we're faking up the old "errno"
behavior for it. We can convert those callers to
refs_resolve_ref_unsafe_with_errno() later.

We are leaving out out the refs_read_special_head() in
refs_read_raw_ref() for now, as noted in the next commit moving it to
"failure_errno" will require some special consideration.

We're intentionally mis-indenting the argument list of the new
refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
subsequent commit, doing it this way makes that diff smaller.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 61 ++++++++++++++++++++++++++++++-------------
 refs/files-backend.c  | 10 ++++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 7b3f05a66f..406e27213b 100644
--- a/refs.c
+++ b/refs.c
@@ -1671,30 +1671,33 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
+	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type, &errno);
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
 	int symref_count;
 
+	assert(failure_errno);
+
 	if (!oid)
 		oid = &unused_oid;
 	if (!flags)
@@ -1705,7 +1708,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1723,8 +1726,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, failure_errno)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1736,9 +1739,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (*failure_errno != ENOENT &&
+			    *failure_errno != EISDIR &&
+			    *failure_errno != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1765,7 +1768,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1773,10 +1776,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2227,6 +2244,13 @@ int refs_verify_refname_available(struct ref_store *refs,
 
 	strbuf_grow(&dirname, strlen(refname) + 1);
 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
+		/*
+		 * Just saying "Is a directory" when we e.g. can't
+		 * lock some multi-level ref isn't very informative,
+		 * the user won't be told *what* is a directory, so
+		 * let's not use strerror() below.
+		 */
+		int ignore_errno;
 		/* Expand dirname to the new prefix, not including the trailing slash: */
 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
 
@@ -2238,7 +2262,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, &ignore_errno)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index c4b5b70c53..3eef442c24 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
+		int ignore_errno;
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	/* Is it a directory? */
 	if (S_ISDIR(st.st_mode)) {
+		int ignore_errno;
 		/*
 		 * Even though there is a directory where the loose
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index a457f18e93..928e5d2a99 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af48..54f57c6a2d 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v7 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                               ` (3 preceding siblings ...)
  2021-07-14 11:43             ` [PATCH v7 4/6] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-14 11:43             ` [PATCH v7 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

In untangling this we discovered a tricky edge case. The
refs_read_special_head() function was relying on
parse_loose_ref_contents() setting EINVAL.

By converting it to use "saved_errno" we can migrate away from "errno"
in this part of the code entirely, and do away with an existing
"save_errno" pattern, its only purpose was to not clobber the "errno"
we previously needed at the end of files_read_raw_ref().

Let's assert that we can do that by not having files_read_raw_ref()
itself operate on *failure_errno in addition to passing it on. Instead
we'll assert that if we return non-zero we actually do set errno, thus
assuring ourselves and callers that they can trust the resulting
"failure_errno".

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 30 +++++++++++++++++++-----------
 refs/refs-internal.h |  6 ++++--
 3 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index 406e27213b..af01a692cb 100644
--- a/refs.c
+++ b/refs.c
@@ -1653,7 +1653,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1663,7 +1664,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1678,7 +1680,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 3eef442c24..e6b1cd9ff7 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	int fd;
 	int ret = -1;
 	int remaining_retries = 3;
+	int myerr = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (lstat(path, &st) < 0) {
 		int ignore_errno;
-		if (errno != ENOENT)
+		myerr = errno;
+		errno = 0;
+		if (myerr != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = ENOENT;
+			myerr = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			myerr = errno;
+			errno = 0;
+			if (myerr == ENOENT || myerr == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = EISDIR;
+			myerr = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		myerr = errno;
+		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
 
 out:
-	*failure_errno = errno;
+	if (ret && !myerr)
+		BUG("returning non-zero %d, should have set myerr!", ret);
+	*failure_errno = myerr;
+
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2d..bf581e70cf 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -689,10 +689,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v7 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                               ` (4 preceding siblings ...)
  2021-07-14 11:43             ` [PATCH v7 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-07-14 11:43             ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-14 11:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  2 +-
 refs.h               | 11 +++++++++++
 refs/files-backend.c | 13 +++++++------
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index af01a692cb..1dacb5fe27 100644
--- a/refs.c
+++ b/refs.c
@@ -1687,7 +1687,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
diff --git a/refs.h b/refs.h
index 48970dfc7e..344a3c51a8 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+/**
+ * refs_resolve_ref_unsafe_with_errno() is like
+ * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * lead to a failure. We guarantee that errno is set to a meaningful
+ * value on non-zero return.
+ */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index e6b1cd9ff7..5acd636ba9 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -935,6 +935,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	int mustexist = (old_oid && !is_null_oid(old_oid));
 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
 	int resolved;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -947,19 +948,19 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
 	files_ref_path(refs, &ref_file, refname);
-	resolved = !!refs_resolve_ref_unsafe(&refs->base,
-					     refname, resolve_flags,
-					     &lock->old_oid, type);
+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+							resolve_flags,
+							&lock->old_oid, type,
+							&resolve_errno);
 	if (!resolved &&
-	    (errno != ENOTDIR ||
+	    (resolve_errno != ENOTDIR ||
 	     /* in case of D/F conflict, try to generate a better error
 	      * message. If that fails, fall back to strerror(ENOTDIR).
 	      */
 	     !refs_verify_refname_available(&refs->base, refname, extras,
 					    skip, err))) {
 		strbuf_addf(err, "unable to resolve reference '%s': %s",
-			    refname, strerror(errno));
-
+			    refname, strerror(resolve_errno));
 		goto error_return;
 	}
 
-- 
2.32.0.851.g0fc62a9785


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 0/7] refs: cleanup errno sideband ref related functions
  2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                               ` (5 preceding siblings ...)
  2021-07-14 11:43             ` [PATCH v7 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22             ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
                                 ` (7 more replies)
  6 siblings, 8 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

A v8 re-roll of v7 of this, covered in detail at:
https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/

This topic relies on the now-re-rolled and bigger dependency topic to
cleanup the use of the reflog API, which is 1/4 callers (and the
complex one) that need an API being simplified here, see
https://lore.kernel.org/git/cover-00.11-00000000000-20210716T140631Z-avarab@gmail.com

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

Ævar Arnfjörð Bjarmason (1):
  refs file backend: move raceproof_create_file() here

 cache.h               |  43 ---------
 object-file.c         |  68 --------------
 refs.c                |  69 +++++++++-----
 refs.h                |  11 +++
 refs/debug.c          |   4 +-
 refs/files-backend.c  | 207 +++++++++++++++++++++++++++++++++---------
 refs/packed-backend.c |  15 +--
 refs/refs-internal.h  |  32 ++++---
 8 files changed, 250 insertions(+), 199 deletions(-)

Range-diff against v7:
-:  ----------- > 1:  ce1ca2cf30f refs file backend: move raceproof_create_file() here
1:  4beba4443ab = 2:  2a69bbea821 refs: remove EINVAL errno output from specification of read_raw_ref_fn
2:  fd8e3561851 ! 3:  a3f80c6d2f7 refs/files-backend: stop setting errno from lock_ref_oid_basic
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      	struct strbuf ref_file = STRBUF_INIT;
      	struct ref_lock *lock;
     -	int last_errno = 0;
    - 	int mustexist = (old_oid && !is_null_oid(old_oid));
    - 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
    - 	int resolved;
    + 
    + 	files_assert_main_repository(refs, "lock_ref_oid_basic");
    + 	assert(err);
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 	resolved = !!refs_resolve_ref_unsafe(&refs->base,
    - 					     refname, resolve_flags,
    - 					     &lock->old_oid, type);
    --	if (!resolved) {
    + 	files_ref_path(refs, &ref_file, refname);
    + 	if (!refs_resolve_ref_unsafe(&refs->base, refname,
    + 				     RESOLVE_REF_NO_RECURSE,
    +-				     &lock->old_oid, type)) {
     -		last_errno = errno;
     -		if (last_errno != ENOTDIR ||
     -		    !refs_verify_refname_available(&refs->base, refname,
    --						   extras, skip, err))
    +-						   NULL, NULL, err))
     -			strbuf_addf(err, "unable to resolve reference '%s': %s",
     -				    refname, strerror(last_errno));
    -+	if (!resolved &&
    +-
    ++				     &lock->old_oid, type) &&
     +	    (errno != ENOTDIR ||
     +	     /* in case of D/F conflict, try to generate a better error
     +	      * message. If that fails, fall back to strerror(ENOTDIR).
     +	      */
    -+	     !refs_verify_refname_available(&refs->base, refname, extras,
    -+					    skip, err))) {
    ++	     !refs_verify_refname_available(&refs->base, refname, NULL,
    ++					    NULL, err))) {
     +		strbuf_addf(err, "unable to resolve reference '%s': %s",
     +			    refname, strerror(errno));
    - 
      		goto error_return;
      	}
    + 
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    + 	 */
      	if (is_null_oid(&lock->old_oid) &&
      	    refs_verify_refname_available(refs->packed_ref_store, refname,
    - 					  extras, skip, err)) {
    +-					  NULL, NULL, err)) {
     -		last_errno = ENOTDIR;
    ++					  NULL, NULL, err))
      		goto error_return;
    - 	}
    +-	}
      
      	lock->ref_name = xstrdup(refname);
      
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      		unable_to_lock_message(ref_file.buf, errno, err);
      		goto error_return;
      	}
    - 
    - 	if (verify_lock(&refs->base, lock, old_oid, mustexist, err)) {
    --		last_errno = errno;
    - 		goto error_return;
    - 	}
    - 	goto out;
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      
       out:
3:  913e72e9ffd = 4:  147058c8c3d refs: make errno output explicit for read_raw_ref_fn
4:  dd191768f66 = 5:  b42a7474f18 refs: add failure_errno to refs_read_raw_ref() signature
5:  77c53dc0d93 = 6:  93b770c8bea refs: explicitly return failure_errno from parse_loose_ref_contents
6:  5d4b1b4ddff ! 7:  cb32b5c0526 refs: make errno output explicit for refs_resolve_ref_unsafe
    @@ refs.h: const char *refs_resolve_ref_unsafe(struct ref_store *refs,
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 	int mustexist = (old_oid && !is_null_oid(old_oid));
    - 	int resolve_flags = RESOLVE_REF_NO_RECURSE;
    - 	int resolved;
    + {
    + 	struct strbuf ref_file = STRBUF_INIT;
    + 	struct ref_lock *lock;
     +	int resolve_errno = 0;
      
      	files_assert_main_repository(refs, "lock_ref_oid_basic");
      	assert(err);
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 		resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
    + 	CALLOC_ARRAY(lock, 1);
      
      	files_ref_path(refs, &ref_file, refname);
    --	resolved = !!refs_resolve_ref_unsafe(&refs->base,
    --					     refname, resolve_flags,
    --					     &lock->old_oid, type);
    -+	resolved = !!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
    -+							resolve_flags,
    -+							&lock->old_oid, type,
    -+							&resolve_errno);
    - 	if (!resolved &&
    +-	if (!refs_resolve_ref_unsafe(&refs->base, refname,
    +-				     RESOLVE_REF_NO_RECURSE,
    +-				     &lock->old_oid, type) &&
     -	    (errno != ENOTDIR ||
    ++	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
    ++						RESOLVE_REF_NO_RECURSE,
    ++						&lock->old_oid, type,
    ++						&resolve_errno) &&
     +	    (resolve_errno != ENOTDIR ||
      	     /* in case of D/F conflict, try to generate a better error
      	      * message. If that fails, fall back to strerror(ENOTDIR).
      	      */
    - 	     !refs_verify_refname_available(&refs->base, refname, extras,
    - 					    skip, err))) {
    + 	     !refs_verify_refname_available(&refs->base, refname, NULL,
    + 					    NULL, err))) {
      		strbuf_addf(err, "unable to resolve reference '%s': %s",
     -			    refname, strerror(errno));
    --
     +			    refname, strerror(resolve_errno));
      		goto error_return;
      	}
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v8 1/7] refs file backend: move raceproof_create_file() here
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                 ` (6 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

Move the raceproof_create_file() API added to cache.h and
object-file.c in 177978f56ad (raceproof_create_file(): new function,
2017-01-06) to its only user, refs/files-backend.c.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 cache.h              |  43 -----------------
 object-file.c        |  68 ---------------------------
 refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 109 insertions(+), 111 deletions(-)

diff --git a/cache.h b/cache.h
index ba04ff8bd36..eb4dfe6381f 100644
--- a/cache.h
+++ b/cache.h
@@ -1202,49 +1202,6 @@ enum scld_error safe_create_leading_directories(char *path);
 enum scld_error safe_create_leading_directories_const(const char *path);
 enum scld_error safe_create_leading_directories_no_share(char *path);
 
-/*
- * Callback function for raceproof_create_file(). This function is
- * expected to do something that makes dirname(path) permanent despite
- * the fact that other processes might be cleaning up empty
- * directories at the same time. Usually it will create a file named
- * path, but alternatively it could create another file in that
- * directory, or even chdir() into that directory. The function should
- * return 0 if the action was completed successfully. On error, it
- * should return a nonzero result and set errno.
- * raceproof_create_file() treats two errno values specially:
- *
- * - ENOENT -- dirname(path) does not exist. In this case,
- *             raceproof_create_file() tries creating dirname(path)
- *             (and any parent directories, if necessary) and calls
- *             the function again.
- *
- * - EISDIR -- the file already exists and is a directory. In this
- *             case, raceproof_create_file() removes the directory if
- *             it is empty (and recursively any empty directories that
- *             it contains) and calls the function again.
- *
- * Any other errno causes raceproof_create_file() to fail with the
- * callback's return value and errno.
- *
- * Obviously, this function should be OK with being called again if it
- * fails with ENOENT or EISDIR. In other scenarios it will not be
- * called again.
- */
-typedef int create_file_fn(const char *path, void *cb);
-
-/*
- * Create a file in dirname(path) by calling fn, creating leading
- * directories if necessary. Retry a few times in case we are racing
- * with another process that is trying to clean up the directory that
- * contains path. See the documentation for create_file_fn for more
- * details.
- *
- * Return the value and set the errno that resulted from the most
- * recent call of fn. fn is always called at least once, and will be
- * called more than once if it returns ENOENT or EISDIR.
- */
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);
-
 int mkdir_in_gitdir(const char *path);
 char *expand_user_path(const char *path, int real_home);
 const char *enter_repo(const char *path, int strict);
diff --git a/object-file.c b/object-file.c
index f233b440b22..ae766e18b4e 100644
--- a/object-file.c
+++ b/object-file.c
@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)
 	return result;
 }
 
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
-{
-	/*
-	 * The number of times we will try to remove empty directories
-	 * in the way of path. This is only 1 because if another
-	 * process is racily creating directories that conflict with
-	 * us, we don't want to fight against them.
-	 */
-	int remove_directories_remaining = 1;
-
-	/*
-	 * The number of times that we will try to create the
-	 * directories containing path. We are willing to attempt this
-	 * more than once, because another process could be trying to
-	 * clean up empty directories at the same time as we are
-	 * trying to create them.
-	 */
-	int create_directories_remaining = 3;
-
-	/* A scratch copy of path, filled lazily if we need it: */
-	struct strbuf path_copy = STRBUF_INIT;
-
-	int ret, save_errno;
-
-	/* Sanity check: */
-	assert(*path);
-
-retry_fn:
-	ret = fn(path, cb);
-	save_errno = errno;
-	if (!ret)
-		goto out;
-
-	if (errno == EISDIR && remove_directories_remaining-- > 0) {
-		/*
-		 * A directory is in the way. Maybe it is empty; try
-		 * to remove it:
-		 */
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
-			goto retry_fn;
-	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
-		/*
-		 * Maybe the containing directory didn't exist, or
-		 * maybe it was just deleted by a process that is
-		 * racing with us to clean up empty directories. Try
-		 * to create it:
-		 */
-		enum scld_error scld_result;
-
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		do {
-			scld_result = safe_create_leading_directories(path_copy.buf);
-			if (scld_result == SCLD_OK)
-				goto retry_fn;
-		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
-	}
-
-out:
-	strbuf_release(&path_copy);
-	errno = save_errno;
-	return ret;
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index bd99bc570de..36af99af844 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(
 	return ref_iterator;
 }
 
+/*
+ * Callback function for raceproof_create_file(). This function is
+ * expected to do something that makes dirname(path) permanent despite
+ * the fact that other processes might be cleaning up empty
+ * directories at the same time. Usually it will create a file named
+ * path, but alternatively it could create another file in that
+ * directory, or even chdir() into that directory. The function should
+ * return 0 if the action was completed successfully. On error, it
+ * should return a nonzero result and set errno.
+ * raceproof_create_file() treats two errno values specially:
+ *
+ * - ENOENT -- dirname(path) does not exist. In this case,
+ *             raceproof_create_file() tries creating dirname(path)
+ *             (and any parent directories, if necessary) and calls
+ *             the function again.
+ *
+ * - EISDIR -- the file already exists and is a directory. In this
+ *             case, raceproof_create_file() removes the directory if
+ *             it is empty (and recursively any empty directories that
+ *             it contains) and calls the function again.
+ *
+ * Any other errno causes raceproof_create_file() to fail with the
+ * callback's return value and errno.
+ *
+ * Obviously, this function should be OK with being called again if it
+ * fails with ENOENT or EISDIR. In other scenarios it will not be
+ * called again.
+ */
+typedef int create_file_fn(const char *path, void *cb);
+
+/*
+ * Create a file in dirname(path) by calling fn, creating leading
+ * directories if necessary. Retry a few times in case we are racing
+ * with another process that is trying to clean up the directory that
+ * contains path. See the documentation for create_file_fn for more
+ * details.
+ *
+ * Return the value and set the errno that resulted from the most
+ * recent call of fn. fn is always called at least once, and will be
+ * called more than once if it returns ENOENT or EISDIR.
+ */
+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
+{
+	/*
+	 * The number of times we will try to remove empty directories
+	 * in the way of path. This is only 1 because if another
+	 * process is racily creating directories that conflict with
+	 * us, we don't want to fight against them.
+	 */
+	int remove_directories_remaining = 1;
+
+	/*
+	 * The number of times that we will try to create the
+	 * directories containing path. We are willing to attempt this
+	 * more than once, because another process could be trying to
+	 * clean up empty directories at the same time as we are
+	 * trying to create them.
+	 */
+	int create_directories_remaining = 3;
+
+	/* A scratch copy of path, filled lazily if we need it: */
+	struct strbuf path_copy = STRBUF_INIT;
+
+	int ret, save_errno;
+
+	/* Sanity check: */
+	assert(*path);
+
+retry_fn:
+	ret = fn(path, cb);
+	save_errno = errno;
+	if (!ret)
+		goto out;
+
+	if (errno == EISDIR && remove_directories_remaining-- > 0) {
+		/*
+		 * A directory is in the way. Maybe it is empty; try
+		 * to remove it:
+		 */
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
+			goto retry_fn;
+	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
+		/*
+		 * Maybe the containing directory didn't exist, or
+		 * maybe it was just deleted by a process that is
+		 * racing with us to clean up empty directories. Try
+		 * to create it:
+		 */
+		enum scld_error scld_result;
+
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		do {
+			scld_result = safe_create_leading_directories(path_copy.buf);
+			if (scld_result == SCLD_OK)
+				goto retry_fn;
+		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
+	}
+
+out:
+	strbuf_release(&path_copy);
+	errno = save_errno;
+	return ret;
+}
+
 static int remove_empty_directories(struct strbuf *path)
 {
 	/*
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
                                 ` (5 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 467f4b3c936..f4445e32904 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -619,9 +619,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                 ` (4 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

In the case of the "errno != ENOTDIR" case that originates in 5b2d8d6f218
(lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
2015-05-11), there the "last_errno" was saved away to return it from
lock_ref_oid_basic(), now that we're no longer doing that we can skip
that entirely and use "errno" directly. A follow-up change will
extract the specific errno we want earlier in this function.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 36af99af844..a6a9f2b99fa 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -991,7 +990,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1001,14 +999,15 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	files_ref_path(refs, &ref_file, refname);
 	if (!refs_resolve_ref_unsafe(&refs->base, refname,
 				     RESOLVE_REF_NO_RECURSE,
-				     &lock->old_oid, type)) {
-		last_errno = errno;
-		if (last_errno != ENOTDIR ||
-		    !refs_verify_refname_available(&refs->base, refname,
-						   NULL, NULL, err))
-			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
-
+				     &lock->old_oid, type) &&
+	    (errno != ENOTDIR ||
+	     /* in case of D/F conflict, try to generate a better error
+	      * message. If that fails, fall back to strerror(ENOTDIR).
+	      */
+	     !refs_verify_refname_available(&refs->base, refname, NULL,
+					    NULL, err))) {
+		strbuf_addf(err, "unable to resolve reference '%s': %s",
+			    refname, strerror(errno));
 		goto error_return;
 	}
 
@@ -1020,15 +1019,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	 */
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
-					  NULL, NULL, err)) {
-		last_errno = ENOTDIR;
+					  NULL, NULL, err))
 		goto error_return;
-	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
@@ -1045,7 +1041,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 4/7] refs: make errno output explicit for read_raw_ref_fn
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                 ` (2 preceding siblings ...)
  2021-07-16 14:22               ` [PATCH v8 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
                                 ` (3 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                |  2 +-
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 29 +++++++++++++++--------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/refs.c b/refs.c
index 5e2330bf78e..136e2e4c78a 100644
--- a/refs.c
+++ b/refs.c
@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 18fd9bca595..d0aaea6d918 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index a6a9f2b99fa..70970f6f770 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	if (hold_lock_file_for_update_timeout(
 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
 			    get_files_ref_lock_timeout_ms()) < 0) {
-		if (errno == ENOENT && --attempts_remaining > 0) {
+		int myerr = errno;
+		errno = 0;
+		if (myerr == ENOENT && --attempts_remaining > 0) {
 			/*
 			 * Maybe somebody just deleted one of the
 			 * directories leading to ref_file.  Try
@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 			 */
 			goto retry;
 		} else {
-			unable_to_lock_message(ref_file.buf, errno, err);
+			unable_to_lock_message(ref_file.buf, myerr, err);
 			goto error_return;
 		}
 	}
@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 53d1ec27b60..1cb7f8e8f70 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f4445e32904..79dfb3af484 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -617,11 +617,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -635,9 +639,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 5/7] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                 ` (3 preceding siblings ...)
  2021-07-16 14:22               ` [PATCH v8 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
                                 ` (2 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Some of our callers explicitly do not care about the errno, rather
than understanding NULL let's have them declare that they don't care
by passing in an "ignore_errno". There's only three of them, and using
that pattern will make it more obvious that they want to throw away
data, let's also add a comment to one of the callers about why we'd
like to ignore the errno.

Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
has a large set of legacy callers, so we're faking up the old "errno"
behavior for it. We can convert those callers to
refs_resolve_ref_unsafe_with_errno() later.

We are leaving out out the refs_read_special_head() in
refs_read_raw_ref() for now, as noted in the next commit moving it to
"failure_errno" will require some special consideration.

We're intentionally mis-indenting the argument list of the new
refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
subsequent commit, doing it this way makes that diff smaller.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 61 ++++++++++++++++++++++++++++++-------------
 refs/files-backend.c  | 10 ++++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index 136e2e4c78a..b451f917d39 100644
--- a/refs.c
+++ b/refs.c
@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
+	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type, &errno);
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
 	int symref_count;
 
+	assert(failure_errno);
+
 	if (!oid)
 		oid = &unused_oid;
 	if (!flags)
@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, failure_errno)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (*failure_errno != ENOENT &&
+			    *failure_errno != EISDIR &&
+			    *failure_errno != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,
 
 	strbuf_grow(&dirname, strlen(refname) + 1);
 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
+		/*
+		 * Just saying "Is a directory" when we e.g. can't
+		 * lock some multi-level ref isn't very informative,
+		 * the user won't be told *what* is a directory, so
+		 * let's not use strerror() below.
+		 */
+		int ignore_errno;
 		/* Expand dirname to the new prefix, not including the trailing slash: */
 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
 
@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, &ignore_errno)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 70970f6f770..25bb225d92a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
+		int ignore_errno;
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	/* Is it a directory? */
 	if (S_ISDIR(st.st_mode)) {
+		int ignore_errno;
 		/*
 		 * Even though there is a directory where the loose
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 1cb7f8e8f70..a4bf3d22d77 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 79dfb3af484..54f57c6a2df 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                 ` (4 preceding siblings ...)
  2021-07-16 14:22               ` [PATCH v8 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-16 14:22               ` [PATCH v8 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

In untangling this we discovered a tricky edge case. The
refs_read_special_head() function was relying on
parse_loose_ref_contents() setting EINVAL.

By converting it to use "saved_errno" we can migrate away from "errno"
in this part of the code entirely, and do away with an existing
"save_errno" pattern, its only purpose was to not clobber the "errno"
we previously needed at the end of files_read_raw_ref().

Let's assert that we can do that by not having files_read_raw_ref()
itself operate on *failure_errno in addition to passing it on. Instead
we'll assert that if we return non-zero we actually do set errno, thus
assuring ourselves and callers that they can trust the resulting
"failure_errno".

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 30 +++++++++++++++++++-----------
 refs/refs-internal.h |  6 ++++--
 3 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index b451f917d39..ceaaccc1680 100644
--- a/refs.c
+++ b/refs.c
@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 25bb225d92a..45d7c346dea 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	int fd;
 	int ret = -1;
 	int remaining_retries = 3;
+	int myerr = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (lstat(path, &st) < 0) {
 		int ignore_errno;
-		if (errno != ENOENT)
+		myerr = errno;
+		errno = 0;
+		if (myerr != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = ENOENT;
+			myerr = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			myerr = errno;
+			errno = 0;
+			if (myerr == ENOENT || myerr == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = EISDIR;
+			myerr = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		myerr = errno;
+		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
 
 out:
-	*failure_errno = errno;
+	if (ret && !myerr)
+		BUG("returning non-zero %d, should have set myerr!", ret);
+	*failure_errno = myerr;
+
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 54f57c6a2df..bf581e70cf6 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -689,10 +689,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v8 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                 ` (5 preceding siblings ...)
  2021-07-16 14:22               ` [PATCH v8 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-07-16 14:22               ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16 14:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  2 +-
 refs.h               | 11 +++++++++++
 refs/files-backend.c | 12 +++++++-----
 3 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index ceaaccc1680..9e2592ee28a 100644
--- a/refs.c
+++ b/refs.c
@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
diff --git a/refs.h b/refs.h
index c009707438d..ba09ba0687b 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+/**
+ * refs_resolve_ref_unsafe_with_errno() is like
+ * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * lead to a failure. We guarantee that errno is set to a meaningful
+ * value on non-zero return.
+ */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 45d7c346dea..be5f386f64d 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1008,17 +1009,18 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	CALLOC_ARRAY(lock, 1);
 
 	files_ref_path(refs, &ref_file, refname);
-	if (!refs_resolve_ref_unsafe(&refs->base, refname,
-				     RESOLVE_REF_NO_RECURSE,
-				     &lock->old_oid, type) &&
-	    (errno != ENOTDIR ||
+	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+						RESOLVE_REF_NO_RECURSE,
+						&lock->old_oid, type,
+						&resolve_errno) &&
+	    (resolve_errno != ENOTDIR ||
 	     /* in case of D/F conflict, try to generate a better error
 	      * message. If that fails, fall back to strerror(ENOTDIR).
 	      */
 	     !refs_verify_refname_available(&refs->base, refname, NULL,
 					    NULL, err))) {
 		strbuf_addf(err, "unable to resolve reference '%s': %s",
-			    refname, strerror(errno));
+			    refname, strerror(resolve_errno));
 		goto error_return;
 	}
 
-- 
2.32.0.874.gfa1990a4f10


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 0/7] refs: cleanup errno sideband ref related functions
  2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                 ` (6 preceding siblings ...)
  2021-07-16 14:22               ` [PATCH v8 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33               ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
                                   ` (8 more replies)
  7 siblings, 9 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

A v9 re-roll of the v8, see
https://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/
and more imporantly the real summary of what this is in v7 at
https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/

As noted there this topic builds on my just-re-rolled v3 removing of
dead code + fixing a gc race in the refs API, at:
https://lore.kernel.org/git/cover-00.12-00000000000-20210720T102051Z-avarab@gmail.com/

The changes in this v8 are merely to re-roll on top of that. There was
a stray comment that didn't apply anymore with its new 12th patch. I
also removed the "in case of D/F conflict..." comment the 3rd patch
previously added, with the changes in the base topic I think this
codepath has become easily understood enough to not need an new
comment anymore (the initial version was more complex, see
https://lore.kernel.org/git/95025080c16f535599826ed4f013845d712b0e8d.1625684869.git.gitgitgadget@gmail.com/).

Han-Wen Nienhuys (6):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

Ævar Arnfjörð Bjarmason (1):
  refs file backend: move raceproof_create_file() here

 cache.h               |  43 ----------
 object-file.c         |  68 ---------------
 refs.c                |  69 ++++++++++-----
 refs.h                |  11 +++
 refs/debug.c          |   4 +-
 refs/files-backend.c  | 195 +++++++++++++++++++++++++++++++++---------
 refs/packed-backend.c |  15 ++--
 refs/refs-internal.h  |  32 ++++---
 8 files changed, 243 insertions(+), 194 deletions(-)

Range-diff against v8:
1:  ce1ca2cf30f = 1:  b7063c5af89 refs file backend: move raceproof_create_file() here
2:  2a69bbea821 = 2:  5a63b64f53f refs: remove EINVAL errno output from specification of read_raw_ref_fn
3:  a3f80c6d2f7 ! 3:  0dd8a4c1209 refs/files-backend: stop setting errno from lock_ref_oid_basic
    @@ Commit message
         refs_reflog_exists() (which calls a function in a vtable that is not
         documented to use and/or preserve errno)
     
    -    In the case of the "errno != ENOTDIR" case that originates in 5b2d8d6f218
    -    (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts,
    -    2015-05-11), there the "last_errno" was saved away to return it from
    -    lock_ref_oid_basic(), now that we're no longer doing that we can skip
    -    that entirely and use "errno" directly. A follow-up change will
    -    extract the specific errno we want earlier in this function.
    -
         Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      	files_assert_main_repository(refs, "lock_ref_oid_basic");
      	assert(err);
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 	files_ref_path(refs, &ref_file, refname);
      	if (!refs_resolve_ref_unsafe(&refs->base, refname,
      				     RESOLVE_REF_NO_RECURSE,
    --				     &lock->old_oid, type)) {
    + 				     &lock->old_oid, type)) {
     -		last_errno = errno;
    --		if (last_errno != ENOTDIR ||
    --		    !refs_verify_refname_available(&refs->base, refname,
    --						   NULL, NULL, err))
    --			strbuf_addf(err, "unable to resolve reference '%s': %s",
    + 		if (!refs_verify_refname_available(&refs->base, refname,
    + 						   NULL, NULL, err))
    + 			strbuf_addf(err, "unable to resolve reference '%s': %s",
     -				    refname, strerror(last_errno));
    --
    -+				     &lock->old_oid, type) &&
    -+	    (errno != ENOTDIR ||
    -+	     /* in case of D/F conflict, try to generate a better error
    -+	      * message. If that fails, fall back to strerror(ENOTDIR).
    -+	      */
    -+	     !refs_verify_refname_available(&refs->base, refname, NULL,
    -+					    NULL, err))) {
    -+		strbuf_addf(err, "unable to resolve reference '%s': %s",
    -+			    refname, strerror(errno));
    ++				    refname, strerror(errno));
    + 
      		goto error_return;
      	}
    - 
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      	 */
      	if (is_null_oid(&lock->old_oid) &&
4:  147058c8c3d = 4:  c54fb99714a refs: make errno output explicit for read_raw_ref_fn
5:  b42a7474f18 = 5:  18825efce7d refs: add failure_errno to refs_read_raw_ref() signature
6:  93b770c8bea = 6:  57e3f246f4f refs: explicitly return failure_errno from parse_loose_ref_contents
7:  cb32b5c0526 ! 7:  4b5e168b978 refs: make errno output explicit for refs_resolve_ref_unsafe
    @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_re
      	files_ref_path(refs, &ref_file, refname);
     -	if (!refs_resolve_ref_unsafe(&refs->base, refname,
     -				     RESOLVE_REF_NO_RECURSE,
    --				     &lock->old_oid, type) &&
    --	    (errno != ENOTDIR ||
    +-				     &lock->old_oid, type)) {
     +	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
     +						RESOLVE_REF_NO_RECURSE,
     +						&lock->old_oid, type,
    -+						&resolve_errno) &&
    -+	    (resolve_errno != ENOTDIR ||
    - 	     /* in case of D/F conflict, try to generate a better error
    - 	      * message. If that fails, fall back to strerror(ENOTDIR).
    - 	      */
    - 	     !refs_verify_refname_available(&refs->base, refname, NULL,
    - 					    NULL, err))) {
    - 		strbuf_addf(err, "unable to resolve reference '%s': %s",
    --			    refname, strerror(errno));
    -+			    refname, strerror(resolve_errno));
    ++						&resolve_errno)) {
    + 		if (!refs_verify_refname_available(&refs->base, refname,
    + 						   NULL, NULL, err))
    + 			strbuf_addf(err, "unable to resolve reference '%s': %s",
    +-				    refname, strerror(errno));
    ++				    refname, strerror(resolve_errno));
    + 
      		goto error_return;
      	}
    - 
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v9 1/7] refs file backend: move raceproof_create_file() here
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

Move the raceproof_create_file() API added to cache.h and
object-file.c in 177978f56ad (raceproof_create_file(): new function,
2017-01-06) to its only user, refs/files-backend.c.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 cache.h              |  43 -----------------
 object-file.c        |  68 ---------------------------
 refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 109 insertions(+), 111 deletions(-)

diff --git a/cache.h b/cache.h
index ba04ff8bd36..eb4dfe6381f 100644
--- a/cache.h
+++ b/cache.h
@@ -1202,49 +1202,6 @@ enum scld_error safe_create_leading_directories(char *path);
 enum scld_error safe_create_leading_directories_const(const char *path);
 enum scld_error safe_create_leading_directories_no_share(char *path);
 
-/*
- * Callback function for raceproof_create_file(). This function is
- * expected to do something that makes dirname(path) permanent despite
- * the fact that other processes might be cleaning up empty
- * directories at the same time. Usually it will create a file named
- * path, but alternatively it could create another file in that
- * directory, or even chdir() into that directory. The function should
- * return 0 if the action was completed successfully. On error, it
- * should return a nonzero result and set errno.
- * raceproof_create_file() treats two errno values specially:
- *
- * - ENOENT -- dirname(path) does not exist. In this case,
- *             raceproof_create_file() tries creating dirname(path)
- *             (and any parent directories, if necessary) and calls
- *             the function again.
- *
- * - EISDIR -- the file already exists and is a directory. In this
- *             case, raceproof_create_file() removes the directory if
- *             it is empty (and recursively any empty directories that
- *             it contains) and calls the function again.
- *
- * Any other errno causes raceproof_create_file() to fail with the
- * callback's return value and errno.
- *
- * Obviously, this function should be OK with being called again if it
- * fails with ENOENT or EISDIR. In other scenarios it will not be
- * called again.
- */
-typedef int create_file_fn(const char *path, void *cb);
-
-/*
- * Create a file in dirname(path) by calling fn, creating leading
- * directories if necessary. Retry a few times in case we are racing
- * with another process that is trying to clean up the directory that
- * contains path. See the documentation for create_file_fn for more
- * details.
- *
- * Return the value and set the errno that resulted from the most
- * recent call of fn. fn is always called at least once, and will be
- * called more than once if it returns ENOENT or EISDIR.
- */
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);
-
 int mkdir_in_gitdir(const char *path);
 char *expand_user_path(const char *path, int real_home);
 const char *enter_repo(const char *path, int strict);
diff --git a/object-file.c b/object-file.c
index ecca5a8da00..231a02997ba 100644
--- a/object-file.c
+++ b/object-file.c
@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)
 	return result;
 }
 
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
-{
-	/*
-	 * The number of times we will try to remove empty directories
-	 * in the way of path. This is only 1 because if another
-	 * process is racily creating directories that conflict with
-	 * us, we don't want to fight against them.
-	 */
-	int remove_directories_remaining = 1;
-
-	/*
-	 * The number of times that we will try to create the
-	 * directories containing path. We are willing to attempt this
-	 * more than once, because another process could be trying to
-	 * clean up empty directories at the same time as we are
-	 * trying to create them.
-	 */
-	int create_directories_remaining = 3;
-
-	/* A scratch copy of path, filled lazily if we need it: */
-	struct strbuf path_copy = STRBUF_INIT;
-
-	int ret, save_errno;
-
-	/* Sanity check: */
-	assert(*path);
-
-retry_fn:
-	ret = fn(path, cb);
-	save_errno = errno;
-	if (!ret)
-		goto out;
-
-	if (errno == EISDIR && remove_directories_remaining-- > 0) {
-		/*
-		 * A directory is in the way. Maybe it is empty; try
-		 * to remove it:
-		 */
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
-			goto retry_fn;
-	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
-		/*
-		 * Maybe the containing directory didn't exist, or
-		 * maybe it was just deleted by a process that is
-		 * racing with us to clean up empty directories. Try
-		 * to create it:
-		 */
-		enum scld_error scld_result;
-
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		do {
-			scld_result = safe_create_leading_directories(path_copy.buf);
-			if (scld_result == SCLD_OK)
-				goto retry_fn;
-		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
-	}
-
-out:
-	strbuf_release(&path_copy);
-	errno = save_errno;
-	return ret;
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 443182da102..9fc596fc75a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(
 	return ref_iterator;
 }
 
+/*
+ * Callback function for raceproof_create_file(). This function is
+ * expected to do something that makes dirname(path) permanent despite
+ * the fact that other processes might be cleaning up empty
+ * directories at the same time. Usually it will create a file named
+ * path, but alternatively it could create another file in that
+ * directory, or even chdir() into that directory. The function should
+ * return 0 if the action was completed successfully. On error, it
+ * should return a nonzero result and set errno.
+ * raceproof_create_file() treats two errno values specially:
+ *
+ * - ENOENT -- dirname(path) does not exist. In this case,
+ *             raceproof_create_file() tries creating dirname(path)
+ *             (and any parent directories, if necessary) and calls
+ *             the function again.
+ *
+ * - EISDIR -- the file already exists and is a directory. In this
+ *             case, raceproof_create_file() removes the directory if
+ *             it is empty (and recursively any empty directories that
+ *             it contains) and calls the function again.
+ *
+ * Any other errno causes raceproof_create_file() to fail with the
+ * callback's return value and errno.
+ *
+ * Obviously, this function should be OK with being called again if it
+ * fails with ENOENT or EISDIR. In other scenarios it will not be
+ * called again.
+ */
+typedef int create_file_fn(const char *path, void *cb);
+
+/*
+ * Create a file in dirname(path) by calling fn, creating leading
+ * directories if necessary. Retry a few times in case we are racing
+ * with another process that is trying to clean up the directory that
+ * contains path. See the documentation for create_file_fn for more
+ * details.
+ *
+ * Return the value and set the errno that resulted from the most
+ * recent call of fn. fn is always called at least once, and will be
+ * called more than once if it returns ENOENT or EISDIR.
+ */
+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
+{
+	/*
+	 * The number of times we will try to remove empty directories
+	 * in the way of path. This is only 1 because if another
+	 * process is racily creating directories that conflict with
+	 * us, we don't want to fight against them.
+	 */
+	int remove_directories_remaining = 1;
+
+	/*
+	 * The number of times that we will try to create the
+	 * directories containing path. We are willing to attempt this
+	 * more than once, because another process could be trying to
+	 * clean up empty directories at the same time as we are
+	 * trying to create them.
+	 */
+	int create_directories_remaining = 3;
+
+	/* A scratch copy of path, filled lazily if we need it: */
+	struct strbuf path_copy = STRBUF_INIT;
+
+	int ret, save_errno;
+
+	/* Sanity check: */
+	assert(*path);
+
+retry_fn:
+	ret = fn(path, cb);
+	save_errno = errno;
+	if (!ret)
+		goto out;
+
+	if (errno == EISDIR && remove_directories_remaining-- > 0) {
+		/*
+		 * A directory is in the way. Maybe it is empty; try
+		 * to remove it:
+		 */
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
+			goto retry_fn;
+	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
+		/*
+		 * Maybe the containing directory didn't exist, or
+		 * maybe it was just deleted by a process that is
+		 * racing with us to clean up empty directories. Try
+		 * to create it:
+		 */
+		enum scld_error scld_result;
+
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		do {
+			scld_result = safe_create_leading_directories(path_copy.buf);
+			if (scld_result == SCLD_OK)
+				goto retry_fn;
+		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
+	}
+
+out:
+	strbuf_release(&path_copy);
+	errno = save_errno;
+	return ret;
+}
+
 static int remove_empty_directories(struct strbuf *path)
 {
 	/*
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
                                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3155708345f..dc0d826c3ac 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -622,9 +622,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 9fc596fc75a..9e00da76121 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname,
@@ -991,7 +990,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1002,11 +1000,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (!refs_resolve_ref_unsafe(&refs->base, refname,
 				     RESOLVE_REF_NO_RECURSE,
 				     &lock->old_oid, type)) {
-		last_errno = errno;
 		if (!refs_verify_refname_available(&refs->base, refname,
 						   NULL, NULL, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(errno));
 
 		goto error_return;
 	}
@@ -1019,15 +1016,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	 */
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
-					  NULL, NULL, err)) {
-		last_errno = ENOTDIR;
+					  NULL, NULL, err))
 		goto error_return;
-	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
@@ -1044,7 +1038,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (2 preceding siblings ...)
  2021-07-20 10:33                 ` [PATCH v9 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-08-16 13:00                   ` Han-Wen Nienhuys
  2021-07-20 10:33                 ` [PATCH v9 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
                                   ` (4 subsequent siblings)
  8 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                |  2 +-
 refs/debug.c          |  4 ++--
 refs/files-backend.c  | 29 +++++++++++++++--------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 34 insertions(+), 29 deletions(-)

diff --git a/refs.c b/refs.c
index d9635436759..fc6c0ddffae 100644
--- a/refs.c
+++ b/refs.c
@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index 18fd9bca595..d0aaea6d918 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
@@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	oidcpy(oid, null_oid());
 	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 9e00da76121..85b4d791444 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	if (hold_lock_file_for_update_timeout(
 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
 			    get_files_ref_lock_timeout_ms()) < 0) {
-		if (errno == ENOENT && --attempts_remaining > 0) {
+		int myerr = errno;
+		errno = 0;
+		if (myerr == ENOENT && --attempts_remaining > 0) {
 			/*
 			 * Maybe somebody just deleted one of the
 			 * directories leading to ref_file.  Try
@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 			 */
 			goto retry;
 		} else {
-			unable_to_lock_message(ref_file.buf, errno, err);
+			unable_to_lock_message(ref_file.buf, myerr, err);
 			goto error_return;
 		}
 	}
@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 24a360b719f..159ac776240 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index dc0d826c3ac..33a31d5c236 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -620,11 +620,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -638,9 +642,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 5/7] refs: add failure_errno to refs_read_raw_ref() signature
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (3 preceding siblings ...)
  2021-07-20 10:33                 ` [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-07-20 10:33                 ` [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
                                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Some of our callers explicitly do not care about the errno, rather
than understanding NULL let's have them declare that they don't care
by passing in an "ignore_errno". There's only three of them, and using
that pattern will make it more obvious that they want to throw away
data, let's also add a comment to one of the callers about why we'd
like to ignore the errno.

Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
has a large set of legacy callers, so we're faking up the old "errno"
behavior for it. We can convert those callers to
refs_resolve_ref_unsafe_with_errno() later.

We are leaving out out the refs_read_special_head() in
refs_read_raw_ref() for now, as noted in the next commit moving it to
"failure_errno" will require some special consideration.

We're intentionally mis-indenting the argument list of the new
refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
subsequent commit, doing it this way makes that diff smaller.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 61 ++++++++++++++++++++++++++++++-------------
 refs/files-backend.c  | 10 ++++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index fc6c0ddffae..728285c9220 100644
--- a/refs.c
+++ b/refs.c
@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
+	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type, &errno);
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
 	int symref_count;
 
+	assert(failure_errno);
+
 	if (!oid)
 		oid = &unused_oid;
 	if (!flags)
@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, failure_errno)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (*failure_errno != ENOENT &&
+			    *failure_errno != EISDIR &&
+			    *failure_errno != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,
 
 	strbuf_grow(&dirname, strlen(refname) + 1);
 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
+		/*
+		 * Just saying "Is a directory" when we e.g. can't
+		 * lock some multi-level ref isn't very informative,
+		 * the user won't be told *what* is a directory, so
+		 * let's not use strerror() below.
+		 */
+		int ignore_errno;
 		/* Expand dirname to the new prefix, not including the trailing slash: */
 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
 
@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, &ignore_errno)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 85b4d791444..bbd8caec624 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
+		int ignore_errno;
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	/* Is it a directory? */
 	if (S_ISDIR(st.st_mode)) {
+		int ignore_errno;
 		/*
 		 * Even though there is a directory where the loose
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 159ac776240..923b9ad89de 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 33a31d5c236..7beb38f79cc 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (4 preceding siblings ...)
  2021-07-20 10:33                 ` [PATCH v9 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-08-13 20:54                   ` Jonathan Tan
  2021-07-20 10:33                 ` [PATCH v9 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
                                   ` (2 subsequent siblings)
  8 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

In untangling this we discovered a tricky edge case. The
refs_read_special_head() function was relying on
parse_loose_ref_contents() setting EINVAL.

By converting it to use "saved_errno" we can migrate away from "errno"
in this part of the code entirely, and do away with an existing
"save_errno" pattern, its only purpose was to not clobber the "errno"
we previously needed at the end of files_read_raw_ref().

Let's assert that we can do that by not having files_read_raw_ref()
itself operate on *failure_errno in addition to passing it on. Instead
we'll assert that if we return non-zero we actually do set errno, thus
assuring ourselves and callers that they can trust the resulting
"failure_errno".

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 30 +++++++++++++++++++-----------
 refs/refs-internal.h |  6 ++++--
 3 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index 728285c9220..b31dbdd0fa2 100644
--- a/refs.c
+++ b/refs.c
@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index bbd8caec624..746831c86cb 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	int fd;
 	int ret = -1;
 	int remaining_retries = 3;
+	int myerr = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (lstat(path, &st) < 0) {
 		int ignore_errno;
-		if (errno != ENOENT)
+		myerr = errno;
+		errno = 0;
+		if (myerr != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = ENOENT;
+			myerr = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			myerr = errno;
+			errno = 0;
+			if (myerr == ENOENT || myerr == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = EISDIR;
+			myerr = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		myerr = errno;
+		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
 
 out:
-	*failure_errno = errno;
+	if (ret && !myerr)
+		BUG("returning non-zero %d, should have set myerr!", ret);
+	*failure_errno = myerr;
+
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -486,7 +494,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 7beb38f79cc..9aa4af81836 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -692,10 +692,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v9 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (5 preceding siblings ...)
  2021-07-20 10:33                 ` [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-07-20 10:33                 ` Ævar Arnfjörð Bjarmason
  2021-07-26 23:49                 ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
  8 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-20 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  2 +-
 refs.h               | 11 +++++++++++
 refs/files-backend.c | 10 ++++++----
 3 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index b31dbdd0fa2..9497b51e0a1 100644
--- a/refs.c
+++ b/refs.c
@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
diff --git a/refs.h b/refs.h
index c009707438d..ba09ba0687b 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+/**
+ * refs_resolve_ref_unsafe_with_errno() is like
+ * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * lead to a failure. We guarantee that errno is set to a meaningful
+ * value on non-zero return.
+ */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 746831c86cb..68182296c1b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1008,13 +1009,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	CALLOC_ARRAY(lock, 1);
 
 	files_ref_path(refs, &ref_file, refname);
-	if (!refs_resolve_ref_unsafe(&refs->base, refname,
-				     RESOLVE_REF_NO_RECURSE,
-				     &lock->old_oid, type)) {
+	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+						RESOLVE_REF_NO_RECURSE,
+						&lock->old_oid, type,
+						&resolve_errno)) {
 		if (!refs_verify_refname_available(&refs->base, refname,
 						   NULL, NULL, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
-- 
2.32.0.874.ge7a9d58bfcf


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* Re: [PATCH v9 0/7] refs: cleanup errno sideband ref related functions
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (6 preceding siblings ...)
  2021-07-20 10:33                 ` [PATCH v9 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
@ 2021-07-26 23:49                 ` Ævar Arnfjörð Bjarmason
  2021-07-27  0:18                   ` Junio C Hamano
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
  8 siblings, 1 reply; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-26 23:49 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason


On Tue, Jul 20 2021, Ævar Arnfjörð Bjarmason wrote:

> A v9 re-roll of the v8, see
> https://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/
> and more imporantly the real summary of what this is in v7 at
> https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/

FYI I did not re-roll a v10 of this on top of the just re-rolled base
topic:
https://lore.kernel.org/git/cover-00.11-0000000000-20210726T234237Z-avarab@gmail.com/#t

The range-diff I have locally between the two (this version, and my
rebased "should I need a re-roll" version) shows no changes, so this
still cleanly applies on top.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v9 0/7] refs: cleanup errno sideband ref related functions
  2021-07-26 23:49                 ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-07-27  0:18                   ` Junio C Hamano
  0 siblings, 0 replies; 148+ messages in thread
From: Junio C Hamano @ 2021-07-27  0:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Han-Wen Nienhuys, Michael Haggerty, Jonathan Tan

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Tue, Jul 20 2021, Ævar Arnfjörð Bjarmason wrote:
>
>> A v9 re-roll of the v8, see
>> https://lore.kernel.org/git/cover-0.7-00000000000-20210716T142032Z-avarab@gmail.com/
>> and more imporantly the real summary of what this is in v7 at
>> https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/
>
> FYI I did not re-roll a v10 of this on top of the just re-rolled base
> topic:
> https://lore.kernel.org/git/cover-00.11-0000000000-20210726T234237Z-avarab@gmail.com/#t
>
> The range-diff I have locally between the two (this version, and my
> rebased "should I need a re-roll" version) shows no changes, so this
> still cleanly applies on top.

Thanks.  I just rebased it on top of the base topic locally.  No
need to resend.


^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-07-20 10:33                 ` [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-08-13 20:54                   ` Jonathan Tan
  0 siblings, 0 replies; 148+ messages in thread
From: Jonathan Tan @ 2021-08-13 20:54 UTC (permalink / raw)
  To: avarab; +Cc: git, gitster, peff, hanwen, mhagger, jonathantanmy

> @@ -448,26 +454,28 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
>  	}
>  	strbuf_reset(&sb_contents);
>  	if (strbuf_read(&sb_contents, fd, 256) < 0) {
> -		int save_errno = errno;
>  		close(fd);
> -		errno = save_errno;
>  		goto out;
>  	}

[snip unrelated code]

>  out:
> -	*failure_errno = errno;
> +	if (ret && !myerr)
> +		BUG("returning non-zero %d, should have set myerr!", ret);

At $DAYJOB, some people have observed this BUG triggering. Right now I
don't have a consistent reproduction recipe, but we noticed that the
block starting with "if (strbuf_read" quoted above does not set myerr
upon an error, and instead immediately jumps to "out" with ret being -1.

^ permalink raw reply	[flat|nested] 148+ messages in thread

* Re: [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn
  2021-07-20 10:33                 ` [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-08-16 13:00                   ` Han-Wen Nienhuys
  0 siblings, 0 replies; 148+ messages in thread
From: Han-Wen Nienhuys @ 2021-08-16 13:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Michael Haggerty, Jonathan Tan

On Tue, Jul 20, 2021 at 12:33 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> --- a/refs/debug.c
> +++ b/refs/debug.c
> @@ -238,7 +238,7 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
>
>  static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
>                               struct object_id *oid, struct strbuf *referent,
> -                             unsigned int *type)
> +                             unsigned int *type, int *failure_errno)

the code in debug.c needs some tweaking:

>  {
>         struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
>         int res = 0;
> @@ -246,7 +246,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
>         oidcpy(oid, null_oid());
>         errno = 0;

this line can go

>         res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
> -                                           type);
> +                                           type, failure_errno);
>
>         if (res == 0) {
>                 trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",

A bit below, failure_errno (rather than errno) should be printed.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v10 0/8] refs: cleanup errno sideband ref related functions
  2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
                                   ` (7 preceding siblings ...)
  2021-07-26 23:49                 ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                 ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 1/8] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
                                     ` (7 more replies)
  8 siblings, 8 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

A v10 re-roll of the v9, see
https://lore.kernel.org/git/cover-0.7-00000000000-20210720T102644Z-avarab@gmail.com/
and more imporantly the real summary of what this is in v7 at
https://lore.kernel.org/git/cover-0.6-0000000000-20210714T114301Z-avarab@gmail.com/

This topic builds on my just-re-rolled v5 removing of dead code +
fixing a gc race in the refs API, at:
https://lore.kernel.org/git/cover-v5-00.13-00000000000-20210823T113115Z-avarab@gmail.com/

The changes since v9 are:

 * Rebased on v5 of its "base" topic, see:
   https://lore.kernel.org/git/cover-v5-00.13-00000000000-20210823T113115Z-avarab@gmail.com/

 * Incorporate the test + fix Han-Wen submitted as
   https://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com/;
   that's here as 6/8.

   I fixed up the test code a bit, added a SYMLINKS prerequisite as
   suggested by Junio, used a "test_when_finished" instead of a "rm
   -rf" at the start, used test_cmp over "test", and other minor style
   changes. The actual BUG() fix is then squashed into 7/8.

 * Fixed issues in refs/debug.c, noted in
   https://lore.kernel.org/git/CAFQ2z_M0LNmZn2xW_GWdwZOCi20xc9t3EnMzMzHP8ZcmWrW9EA@mail.gmail.com/

Han-Wen Nienhuys (7):
  refs: remove EINVAL errno output from specification of read_raw_ref_fn
  refs/files-backend: stop setting errno from lock_ref_oid_basic
  refs: make errno output explicit for read_raw_ref_fn
  refs: add failure_errno to refs_read_raw_ref() signature
  branch tests: test for errno propagating on failing read
  refs: explicitly return failure_errno from parse_loose_ref_contents
  refs: make errno output explicit for refs_resolve_ref_unsafe

Ævar Arnfjörð Bjarmason (1):
  refs file backend: move raceproof_create_file() here

 cache.h               |  43 ---------
 object-file.c         |  68 ---------------
 refs.c                |  69 ++++++++++-----
 refs.h                |  11 +++
 refs/debug.c          |   7 +-
 refs/files-backend.c  | 196 +++++++++++++++++++++++++++++++++---------
 refs/packed-backend.c |  15 ++--
 refs/refs-internal.h  |  32 ++++---
 t/t3200-branch.sh     |  21 +++++
 9 files changed, 266 insertions(+), 196 deletions(-)

Range-diff against v9:
1:  b7063c5af89 = 1:  f06b054e861 refs file backend: move raceproof_create_file() here
2:  5a63b64f53f = 2:  ba0f5f5fb0a refs: remove EINVAL errno output from specification of read_raw_ref_fn
3:  0dd8a4c1209 ! 3:  2c4c30e8e06 refs/files-backend: stop setting errno from lock_ref_oid_basic
    @@ refs/files-backend.c: static int create_reflock(const char *path, void *cb)
     - * On failure errno is set to something meaningful.
       */
      static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
    - 					   const char *refname,
    + 					   const char *refname, int *type,
     @@ refs/files-backend.c: static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
      {
      	struct strbuf ref_file = STRBUF_INIT;
4:  c54fb99714a ! 4:  05f3a346e3b refs: make errno output explicit for read_raw_ref_fn
    @@ refs/debug.c: debug_ref_iterator_begin(struct ref_store *ref_store, const char *
      {
      	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
      	int res = 0;
    -@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 
      	oidcpy(oid, null_oid());
    - 	errno = 0;
    +-	errno = 0;
      	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
     -					    type);
     +					    type, failure_errno);
      
      	if (res == 0) {
      		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
    +@@ refs/debug.c: static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
    + 	} else {
    + 		trace_printf_key(&trace_refs,
    + 				 "read_raw_ref: %s: %d (errno %d)\n", refname,
    +-				 res, errno);
    ++				 res, *failure_errno);
    + 	}
    + 	return res;
    + }
     
      ## refs/files-backend.c ##
     @@ refs/files-backend.c: static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
5:  18825efce7d = 5:  fa9260f25fa refs: add failure_errno to refs_read_raw_ref() signature
-:  ----------- > 6:  6dae8b643ad branch tests: test for errno propagating on failing read
6:  57e3f246f4f ! 7:  18bf4a0e97c refs: explicitly return failure_errno from parse_loose_ref_contents
    @@ refs/files-backend.c: static int files_read_raw_ref(struct ref_store *ref_store,
      	strbuf_reset(&sb_contents);
      	if (strbuf_read(&sb_contents, fd, 256) < 0) {
     -		int save_errno = errno;
    ++		myerr = errno;
      		close(fd);
     -		errno = save_errno;
      		goto out;
    @@ refs/refs-internal.h: struct ref_store {
      
      /*
       * Fill in the generic part of refs and add it to our collection of
    +
    + ## t/t3200-branch.sh ##
    +@@ t/t3200-branch.sh: test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
    + 	) &&
    + 	git --git-dir subdir/.git/ branch rename-src &&
    + 	git rev-parse rename-src >expect &&
    ++	# Tests a BUG() assertion in files_read_raw_ref()
    + 	git --git-dir subdir/.git/ branch -m rename-src rename-dest &&
    + 	git rev-parse rename-dest >actual &&
    + 	test_cmp expect actual &&
7:  4b5e168b978 = 8:  7d94a32af83 refs: make errno output explicit for refs_resolve_ref_unsafe
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply	[flat|nested] 148+ messages in thread

* [PATCH v10 1/8] refs file backend: move raceproof_create_file() here
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 2/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

Move the raceproof_create_file() API added to cache.h and
object-file.c in 177978f56ad (raceproof_create_file(): new function,
2017-01-06) to its only user, refs/files-backend.c.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 cache.h              |  43 -----------------
 object-file.c        |  68 ---------------------------
 refs/files-backend.c | 109 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 109 insertions(+), 111 deletions(-)

diff --git a/cache.h b/cache.h
index bd4869beee4..1e838303654 100644
--- a/cache.h
+++ b/cache.h
@@ -1211,49 +1211,6 @@ enum scld_error safe_create_leading_directories(char *path);
 enum scld_error safe_create_leading_directories_const(const char *path);
 enum scld_error safe_create_leading_directories_no_share(char *path);
 
-/*
- * Callback function for raceproof_create_file(). This function is
- * expected to do something that makes dirname(path) permanent despite
- * the fact that other processes might be cleaning up empty
- * directories at the same time. Usually it will create a file named
- * path, but alternatively it could create another file in that
- * directory, or even chdir() into that directory. The function should
- * return 0 if the action was completed successfully. On error, it
- * should return a nonzero result and set errno.
- * raceproof_create_file() treats two errno values specially:
- *
- * - ENOENT -- dirname(path) does not exist. In this case,
- *             raceproof_create_file() tries creating dirname(path)
- *             (and any parent directories, if necessary) and calls
- *             the function again.
- *
- * - EISDIR -- the file already exists and is a directory. In this
- *             case, raceproof_create_file() removes the directory if
- *             it is empty (and recursively any empty directories that
- *             it contains) and calls the function again.
- *
- * Any other errno causes raceproof_create_file() to fail with the
- * callback's return value and errno.
- *
- * Obviously, this function should be OK with being called again if it
- * fails with ENOENT or EISDIR. In other scenarios it will not be
- * called again.
- */
-typedef int create_file_fn(const char *path, void *cb);
-
-/*
- * Create a file in dirname(path) by calling fn, creating leading
- * directories if necessary. Retry a few times in case we are racing
- * with another process that is trying to clean up the directory that
- * contains path. See the documentation for create_file_fn for more
- * details.
- *
- * Return the value and set the errno that resulted from the most
- * recent call of fn. fn is always called at least once, and will be
- * called more than once if it returns ENOENT or EISDIR.
- */
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb);
-
 int mkdir_in_gitdir(const char *path);
 char *expand_user_path(const char *path, int real_home);
 const char *enter_repo(const char *path, int strict);
diff --git a/object-file.c b/object-file.c
index a8be8994814..9b318eecb19 100644
--- a/object-file.c
+++ b/object-file.c
@@ -414,74 +414,6 @@ enum scld_error safe_create_leading_directories_const(const char *path)
 	return result;
 }
 
-int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
-{
-	/*
-	 * The number of times we will try to remove empty directories
-	 * in the way of path. This is only 1 because if another
-	 * process is racily creating directories that conflict with
-	 * us, we don't want to fight against them.
-	 */
-	int remove_directories_remaining = 1;
-
-	/*
-	 * The number of times that we will try to create the
-	 * directories containing path. We are willing to attempt this
-	 * more than once, because another process could be trying to
-	 * clean up empty directories at the same time as we are
-	 * trying to create them.
-	 */
-	int create_directories_remaining = 3;
-
-	/* A scratch copy of path, filled lazily if we need it: */
-	struct strbuf path_copy = STRBUF_INIT;
-
-	int ret, save_errno;
-
-	/* Sanity check: */
-	assert(*path);
-
-retry_fn:
-	ret = fn(path, cb);
-	save_errno = errno;
-	if (!ret)
-		goto out;
-
-	if (errno == EISDIR && remove_directories_remaining-- > 0) {
-		/*
-		 * A directory is in the way. Maybe it is empty; try
-		 * to remove it:
-		 */
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
-			goto retry_fn;
-	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
-		/*
-		 * Maybe the containing directory didn't exist, or
-		 * maybe it was just deleted by a process that is
-		 * racing with us to clean up empty directories. Try
-		 * to create it:
-		 */
-		enum scld_error scld_result;
-
-		if (!path_copy.len)
-			strbuf_addstr(&path_copy, path);
-
-		do {
-			scld_result = safe_create_leading_directories(path_copy.buf);
-			if (scld_result == SCLD_OK)
-				goto retry_fn;
-		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
-	}
-
-out:
-	strbuf_release(&path_copy);
-	errno = save_errno;
-	return ret;
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index ab666af4b75..950f9738ae0 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -852,6 +852,115 @@ static struct ref_iterator *files_ref_iterator_begin(
 	return ref_iterator;
 }
 
+/*
+ * Callback function for raceproof_create_file(). This function is
+ * expected to do something that makes dirname(path) permanent despite
+ * the fact that other processes might be cleaning up empty
+ * directories at the same time. Usually it will create a file named
+ * path, but alternatively it could create another file in that
+ * directory, or even chdir() into that directory. The function should
+ * return 0 if the action was completed successfully. On error, it
+ * should return a nonzero result and set errno.
+ * raceproof_create_file() treats two errno values specially:
+ *
+ * - ENOENT -- dirname(path) does not exist. In this case,
+ *             raceproof_create_file() tries creating dirname(path)
+ *             (and any parent directories, if necessary) and calls
+ *             the function again.
+ *
+ * - EISDIR -- the file already exists and is a directory. In this
+ *             case, raceproof_create_file() removes the directory if
+ *             it is empty (and recursively any empty directories that
+ *             it contains) and calls the function again.
+ *
+ * Any other errno causes raceproof_create_file() to fail with the
+ * callback's return value and errno.
+ *
+ * Obviously, this function should be OK with being called again if it
+ * fails with ENOENT or EISDIR. In other scenarios it will not be
+ * called again.
+ */
+typedef int create_file_fn(const char *path, void *cb);
+
+/*
+ * Create a file in dirname(path) by calling fn, creating leading
+ * directories if necessary. Retry a few times in case we are racing
+ * with another process that is trying to clean up the directory that
+ * contains path. See the documentation for create_file_fn for more
+ * details.
+ *
+ * Return the value and set the errno that resulted from the most
+ * recent call of fn. fn is always called at least once, and will be
+ * called more than once if it returns ENOENT or EISDIR.
+ */
+static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
+{
+	/*
+	 * The number of times we will try to remove empty directories
+	 * in the way of path. This is only 1 because if another
+	 * process is racily creating directories that conflict with
+	 * us, we don't want to fight against them.
+	 */
+	int remove_directories_remaining = 1;
+
+	/*
+	 * The number of times that we will try to create the
+	 * directories containing path. We are willing to attempt this
+	 * more than once, because another process could be trying to
+	 * clean up empty directories at the same time as we are
+	 * trying to create them.
+	 */
+	int create_directories_remaining = 3;
+
+	/* A scratch copy of path, filled lazily if we need it: */
+	struct strbuf path_copy = STRBUF_INIT;
+
+	int ret, save_errno;
+
+	/* Sanity check: */
+	assert(*path);
+
+retry_fn:
+	ret = fn(path, cb);
+	save_errno = errno;
+	if (!ret)
+		goto out;
+
+	if (errno == EISDIR && remove_directories_remaining-- > 0) {
+		/*
+		 * A directory is in the way. Maybe it is empty; try
+		 * to remove it:
+		 */
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY))
+			goto retry_fn;
+	} else if (errno == ENOENT && create_directories_remaining-- > 0) {
+		/*
+		 * Maybe the containing directory didn't exist, or
+		 * maybe it was just deleted by a process that is
+		 * racing with us to clean up empty directories. Try
+		 * to create it:
+		 */
+		enum scld_error scld_result;
+
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		do {
+			scld_result = safe_create_leading_directories(path_copy.buf);
+			if (scld_result == SCLD_OK)
+				goto retry_fn;
+		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
+	}
+
+out:
+	strbuf_release(&path_copy);
+	errno = save_errno;
+	return ret;
+}
+
 static int remove_empty_directories(struct strbuf *path)
 {
 	/*
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 2/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 1/8] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 3/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
                                     ` (5 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This commit does not change code; it documents the fact that an alternate ref
backend does not need to return EINVAL from read_raw_ref_fn to function
properly.

This is correct, because refs_read_raw_ref is only called from;

* resolve_ref_unsafe(), which does not care for the EINVAL errno result.

* refs_verify_refname_available(), which does not inspect errno.

* files-backend.c, where errno is overwritten on failure.

* packed-backend.c (is_packed_transaction_needed), which calls it for the
  packed ref backend, which never emits EINVAL.

A grep for EINVAL */*c reveals that no code checks errno against EINVAL after
reading references. In addition, the refs.h file does not mention errno at all.

A grep over resolve_ref_unsafe() turned up the following callers that inspect
errno:

* sequencer.c::print_commit_summary, which uses it for die_errno

* lock_ref_oid_basic(), which only treats EISDIR and ENOTDIR specially.

The files ref backend does use EINVAL. The files backend does not call into
the generic API (refs_read_raw), but into the files-specific function
(files_read_raw_ref), which we are not changing in this commit.

As the errno sideband is unintuitive and error-prone, remove EINVAL
value, as a step towards getting rid of the errno sideband altogether.

Spotted by Ævar Arnfjörð Bjarmason <avarab@gmail.com>.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/refs-internal.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index d496c5ed52d..1c6e5ab51d7 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -622,9 +622,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  *
  * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
  * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, set errno to
- * EINVAL, and return -1. If there is another error reading the ref,
- * set errno appropriately and return -1.
+ * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
+ * (errno should not be ENOENT) If there is another error reading the
+ * ref, set errno appropriately and return -1.
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 3/8] refs/files-backend: stop setting errno from lock_ref_oid_basic
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 1/8] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 2/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 4/8] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
                                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

refs/files-backend.c::lock_ref_oid_basic() tries to signal how it failed
to its callers using errno.

It is safe to stop setting errno here, because the callers of this
file-scope static function are

* files_copy_or_rename_ref()
* files_create_symref()
* files_reflog_expire()

None of them looks at errno after seeing a negative return from
lock_ref_oid_basic() to make any decision, and no caller of these three
functions looks at errno after they signal a failure by returning a
negative value. In particular,

* files_copy_or_rename_ref() - here, calls are followed by error()
(which performs I/O) or write_ref_to_lockfile() (which calls
parse_object() which may perform I/O)

* files_create_symref() - here, calls are followed by error() or
create_symref_locked() (which performs I/O and does not inspect
errno)

* files_reflog_expire() - here, calls are followed by error() or
refs_reflog_exists() (which calls a function in a vtable that is not
documented to use and/or preserve errno)

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs/files-backend.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 950f9738ae0..c1794dcd295 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -982,7 +982,6 @@ static int create_reflock(const char *path, void *cb)
 
 /*
  * Locks a ref returning the lock on success and NULL on failure.
- * On failure errno is set to something meaningful.
  */
 static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 					   const char *refname, int *type,
@@ -990,7 +989,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
-	int last_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1001,11 +999,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	if (!refs_resolve_ref_unsafe(&refs->base, refname,
 				     RESOLVE_REF_NO_RECURSE,
 				     &lock->old_oid, type)) {
-		last_errno = errno;
 		if (!refs_verify_refname_available(&refs->base, refname,
 						   NULL, NULL, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(last_errno));
+				    refname, strerror(errno));
 
 		goto error_return;
 	}
@@ -1018,15 +1015,12 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	 */
 	if (is_null_oid(&lock->old_oid) &&
 	    refs_verify_refname_available(refs->packed_ref_store, refname,
-					  NULL, NULL, err)) {
-		last_errno = ENOTDIR;
+					  NULL, NULL, err))
 		goto error_return;
-	}
 
 	lock->ref_name = xstrdup(refname);
 
 	if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) {
-		last_errno = errno;
 		unable_to_lock_message(ref_file.buf, errno, err);
 		goto error_return;
 	}
@@ -1043,7 +1037,6 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 
  out:
 	strbuf_release(&ref_file);
-	errno = last_errno;
 	return lock;
 }
 
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 4/8] refs: make errno output explicit for read_raw_ref_fn
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
                                     ` (2 preceding siblings ...)
  2021-08-23 11:52                   ` [PATCH v10 3/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 5/8] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
                                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This makes it explicit how alternative ref backends should report errors in
read_raw_ref_fn.

read_raw_ref_fn needs to supply a credible errno for a number of cases. These
are primarily:

1) The files backend calls read_raw_ref from lock_raw_ref, and uses the
resulting error codes to create/remove directories as needed.

2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set,
returning the last successfully resolved symref. This is necessary so
read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch
du-jour), and we know on which branch to create the first commit.

Make this information flow explicit by adding a failure_errno to the signature
of read_raw_ref. All errnos from the files backend are still propagated
unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are
relevant.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                |  2 +-
 refs/debug.c          |  7 +++----
 refs/files-backend.c  | 29 +++++++++++++++--------------
 refs/packed-backend.c |  8 ++++----
 refs/refs-internal.h  | 20 ++++++++++++--------
 5 files changed, 35 insertions(+), 31 deletions(-)

diff --git a/refs.c b/refs.c
index 05944d8e725..da0cc82ca66 100644
--- a/refs.c
+++ b/refs.c
@@ -1682,7 +1682,7 @@ int refs_read_raw_ref(struct ref_store *ref_store,
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type);
+					   type, &errno);
 }
 
 /* This function needs to return a meaningful errno on failure */
diff --git a/refs/debug.c b/refs/debug.c
index bf4a82bccb6..8667c640237 100644
--- a/refs/debug.c
+++ b/refs/debug.c
@@ -239,15 +239,14 @@ debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
 
 static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 			      struct object_id *oid, struct strbuf *referent,
-			      unsigned int *type)
+			      unsigned int *type, int *failure_errno)
 {
 	struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store;
 	int res = 0;
 
 	oidcpy(oid, null_oid());
-	errno = 0;
 	res = drefs->refs->be->read_raw_ref(drefs->refs, refname, oid, referent,
-					    type);
+					    type, failure_errno);
 
 	if (res == 0) {
 		trace_printf_key(&trace_refs, "read_raw_ref: %s: %s (=> %s) type %x: %d\n",
@@ -255,7 +254,7 @@ static int debug_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	} else {
 		trace_printf_key(&trace_refs,
 				 "read_raw_ref: %s: %d (errno %d)\n", refname,
-				 res, errno);
+				 res, *failure_errno);
 	}
 	return res;
 }
diff --git a/refs/files-backend.c b/refs/files-backend.c
index c1794dcd295..b3de3633969 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -341,9 +341,9 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs)
 	return refs->loose;
 }
 
-static int files_read_raw_ref(struct ref_store *ref_store,
-			      const char *refname, struct object_id *oid,
-			      struct strbuf *referent, unsigned int *type)
+static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			      struct object_id *oid, struct strbuf *referent,
+			      unsigned int *type, int *failure_errno)
 {
 	struct files_ref_store *refs =
 		files_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -354,7 +354,6 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	struct stat st;
 	int fd;
 	int ret = -1;
-	int save_errno;
 	int remaining_retries = 3;
 
 	*type = 0;
@@ -459,10 +458,9 @@ static int files_read_raw_ref(struct ref_store *ref_store,
 	ret = parse_loose_ref_contents(buf, oid, referent, type);
 
 out:
-	save_errno = errno;
+	*failure_errno = errno;
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
-	errno = save_errno;
 	return ret;
 }
 
@@ -540,6 +538,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	struct strbuf ref_file = STRBUF_INIT;
 	int attempts_remaining = 3;
 	int ret = TRANSACTION_GENERIC_ERROR;
+	int failure_errno;
 
 	assert(err);
 	files_assert_main_repository(refs, "lock_raw_ref");
@@ -610,7 +609,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	if (hold_lock_file_for_update_timeout(
 			    &lock->lk, ref_file.buf, LOCK_NO_DEREF,
 			    get_files_ref_lock_timeout_ms()) < 0) {
-		if (errno == ENOENT && --attempts_remaining > 0) {
+		int myerr = errno;
+		errno = 0;
+		if (myerr == ENOENT && --attempts_remaining > 0) {
 			/*
 			 * Maybe somebody just deleted one of the
 			 * directories leading to ref_file.  Try
@@ -618,7 +619,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 			 */
 			goto retry;
 		} else {
-			unable_to_lock_message(ref_file.buf, errno, err);
+			unable_to_lock_message(ref_file.buf, myerr, err);
 			goto error_return;
 		}
 	}
@@ -628,9 +629,9 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	 * fear that its value will change.
 	 */
 
-	if (files_read_raw_ref(&refs->base, refname,
-			       &lock->old_oid, referent, type)) {
-		if (errno == ENOENT) {
+	if (files_read_raw_ref(&refs->base, refname, &lock->old_oid, referent,
+			       type, &failure_errno)) {
+		if (failure_errno == ENOENT) {
 			if (mustexist) {
 				/* Garden variety missing reference. */
 				strbuf_addf(err, "unable to resolve reference '%s'",
@@ -654,7 +655,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 				 *   reference named "refs/foo/bar/baz".
 				 */
 			}
-		} else if (errno == EISDIR) {
+		} else if (failure_errno == EISDIR) {
 			/*
 			 * There is a directory in the way. It might have
 			 * contained references that have been deleted. If
@@ -692,13 +693,13 @@ static int lock_raw_ref(struct files_ref_store *refs,
 					goto error_return;
 				}
 			}
-		} else if (errno == EINVAL && (*type & REF_ISBROKEN)) {
+		} else if (failure_errno == EINVAL && (*type & REF_ISBROKEN)) {
 			strbuf_addf(err, "unable to resolve reference '%s': "
 				    "reference broken", refname);
 			goto error_return;
 		} else {
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(failure_errno));
 			goto error_return;
 		}
 
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 65ba4214b8d..47247a14917 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -724,9 +724,9 @@ static struct snapshot *get_snapshot(struct packed_ref_store *refs)
 	return refs->snapshot;
 }
 
-static int packed_read_raw_ref(struct ref_store *ref_store,
-			       const char *refname, struct object_id *oid,
-			       struct strbuf *referent, unsigned int *type)
+static int packed_read_raw_ref(struct ref_store *ref_store, const char *refname,
+			       struct object_id *oid, struct strbuf *referent,
+			       unsigned int *type, int *failure_errno)
 {
 	struct packed_ref_store *refs =
 		packed_downcast(ref_store, REF_STORE_READ, "read_raw_ref");
@@ -739,7 +739,7 @@ static int packed_read_raw_ref(struct ref_store *ref_store,
 
 	if (!rec) {
 		/* refname is not a packed reference. */
-		errno = ENOENT;
+		*failure_errno = ENOENT;
 		return -1;
 	}
 
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 1c6e5ab51d7..ddd6d7f8ebd 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -620,11 +620,15 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * properly-formatted or even safe reference name. NEITHER INPUT NOR
  * OUTPUT REFERENCE NAMES ARE VALIDATED WITHIN THIS FUNCTION.
  *
- * Return 0 on success. If the ref doesn't exist, set errno to ENOENT
- * and return -1. If the ref exists but is neither a symbolic ref nor
- * an object ID, it is broken; set REF_ISBROKEN in type, and return -1
- * (errno should not be ENOENT) If there is another error reading the
- * ref, set errno appropriately and return -1.
+ * Return 0 on success, or -1 on failure. If the ref exists but is neither a
+ * symbolic ref nor an object ID, it is broken. In this case set REF_ISBROKEN in
+ * type, and return -1 (failure_errno should not be ENOENT)
+ *
+ * failure_errno provides errno codes that are interpreted beyond error
+ * reporting. The following error codes have special meaning:
+ *    * ENOENT: the ref doesn't exist
+ *    * EISDIR: ref name is a directory
+ *    * ENOTDIR: ref prefix is not a directory
  *
  * Backend-specific flags might be set in type as well, regardless of
  * outcome.
@@ -638,9 +642,9 @@ typedef int reflog_expire_fn(struct ref_store *ref_store,
  * - in all other cases, referent will be untouched, and therefore
  *   refname will still be valid and unchanged.
  */
-typedef int read_raw_ref_fn(struct ref_store *ref_store,
-			    const char *refname, struct object_id *oid,
-			    struct strbuf *referent, unsigned int *type);
+typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname,
+			    struct object_id *oid, struct strbuf *referent,
+			    unsigned int *type, int *failure_errno);
 
 struct ref_storage_be {
 	struct ref_storage_be *next;
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 5/8] refs: add failure_errno to refs_read_raw_ref() signature
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
                                     ` (3 preceding siblings ...)
  2021-08-23 11:52                   ` [PATCH v10 4/8] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 6/8] branch tests: test for errno propagating on failing read Ævar Arnfjörð Bjarmason
                                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This lets us use the explicit errno output parameter in refs_resolve_ref_unsafe.

Some of our callers explicitly do not care about the errno, rather
than understanding NULL let's have them declare that they don't care
by passing in an "ignore_errno". There's only three of them, and using
that pattern will make it more obvious that they want to throw away
data, let's also add a comment to one of the callers about why we'd
like to ignore the errno.

Let's not extend that to refs_resolve_ref_unsafe() itself for now, it
has a large set of legacy callers, so we're faking up the old "errno"
behavior for it. We can convert those callers to
refs_resolve_ref_unsafe_with_errno() later.

We are leaving out out the refs_read_special_head() in
refs_read_raw_ref() for now, as noted in the next commit moving it to
"failure_errno" will require some special consideration.

We're intentionally mis-indenting the argument list of the new
refs_resolve_ref_unsafe_with_errno(), it will be non-static in a
subsequent commit, doing it this way makes that diff smaller.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c                | 61 ++++++++++++++++++++++++++++++-------------
 refs/files-backend.c  | 10 ++++---
 refs/packed-backend.c |  7 ++---
 refs/refs-internal.h  |  6 ++---
 4 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/refs.c b/refs.c
index da0cc82ca66..4c782fa2978 100644
--- a/refs.c
+++ b/refs.c
@@ -1672,30 +1672,33 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	return result;
 }
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type)
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno)
 {
+	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
 					      type);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
-					   type, &errno);
+					   type, failure_errno);
 }
 
-/* This function needs to return a meaningful errno on failure */
-const char *refs_resolve_ref_unsafe(struct ref_store *refs,
-				    const char *refname,
-				    int resolve_flags,
-				    struct object_id *oid, int *flags)
+static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno)
 {
 	static struct strbuf sb_refname = STRBUF_INIT;
 	struct object_id unused_oid;
 	int unused_flags;
 	int symref_count;
 
+	assert(failure_errno);
+
 	if (!oid)
 		oid = &unused_oid;
 	if (!flags)
@@ -1706,7 +1709,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 		if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 		    !refname_is_safe(refname)) {
-			errno = EINVAL;
+			*failure_errno = EINVAL;
 			return NULL;
 		}
 
@@ -1724,8 +1727,8 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 	for (symref_count = 0; symref_count < SYMREF_MAXDEPTH; symref_count++) {
 		unsigned int read_flags = 0;
 
-		if (refs_read_raw_ref(refs, refname,
-				      oid, &sb_refname, &read_flags)) {
+		if (refs_read_raw_ref(refs, refname, oid, &sb_refname,
+				      &read_flags, failure_errno)) {
 			*flags |= read_flags;
 
 			/* In reading mode, refs must eventually resolve */
@@ -1737,9 +1740,9 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 			 * may show errors besides ENOENT if there are
 			 * similarly-named refs.
 			 */
-			if (errno != ENOENT &&
-			    errno != EISDIR &&
-			    errno != ENOTDIR)
+			if (*failure_errno != ENOENT &&
+			    *failure_errno != EISDIR &&
+			    *failure_errno != ENOTDIR)
 				return NULL;
 
 			oidclr(oid);
@@ -1766,7 +1769,7 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
 			if (!(resolve_flags & RESOLVE_REF_ALLOW_BAD_NAME) ||
 			    !refname_is_safe(refname)) {
-				errno = EINVAL;
+				*failure_errno = EINVAL;
 				return NULL;
 			}
 
@@ -1774,10 +1777,24 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 		}
 	}
 
-	errno = ELOOP;
+	*failure_errno = ELOOP;
 	return NULL;
 }
 
+const char *refs_resolve_ref_unsafe(struct ref_store *refs, const char *refname,
+				    int resolve_flags, struct object_id *oid,
+				    int *flags)
+{
+	int failure_errno = 0;
+	const char *refn;
+	refn = refs_resolve_ref_unsafe_with_errno(refs, refname, resolve_flags,
+						  oid, flags, &failure_errno);
+	if (!refn)
+		/* For unmigrated legacy callers */
+		errno = failure_errno;
+	return refn;
+}
+
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
@@ -2228,6 +2245,13 @@ int refs_verify_refname_available(struct ref_store *refs,
 
 	strbuf_grow(&dirname, strlen(refname) + 1);
 	for (slash = strchr(refname, '/'); slash; slash = strchr(slash + 1, '/')) {
+		/*
+		 * Just saying "Is a directory" when we e.g. can't
+		 * lock some multi-level ref isn't very informative,
+		 * the user won't be told *what* is a directory, so
+		 * let's not use strerror() below.
+		 */
+		int ignore_errno;
 		/* Expand dirname to the new prefix, not including the trailing slash: */
 		strbuf_add(&dirname, refname + dirname.len, slash - refname - dirname.len);
 
@@ -2239,7 +2263,8 @@ int refs_verify_refname_available(struct ref_store *refs,
 		if (skip && string_list_has_string(skip, dirname.buf))
 			continue;
 
-		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent, &type)) {
+		if (!refs_read_raw_ref(refs, dirname.buf, &oid, &referent,
+				       &type, &ignore_errno)) {
 			strbuf_addf(err, _("'%s' exists; cannot create '%s'"),
 				    dirname.buf, refname);
 			goto cleanup;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index b3de3633969..f962495e456 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -381,10 +381,11 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		goto out;
 
 	if (lstat(path, &st) < 0) {
+		int ignore_errno;
 		if (errno != ENOENT)
 			goto out;
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = ENOENT;
 			goto out;
 		}
@@ -418,13 +419,14 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	/* Is it a directory? */
 	if (S_ISDIR(st.st_mode)) {
+		int ignore_errno;
 		/*
 		 * Even though there is a directory where the loose
 		 * ref is supposed to be, there could still be a
 		 * packed ref:
 		 */
-		if (refs_read_raw_ref(refs->packed_ref_store, refname,
-				      oid, referent, type)) {
+		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
+				      referent, type, &ignore_errno)) {
 			errno = EISDIR;
 			goto out;
 		}
diff --git a/refs/packed-backend.c b/refs/packed-backend.c
index 47247a14917..52cdc94a26e 100644
--- a/refs/packed-backend.c
+++ b/refs/packed-backend.c
@@ -1347,6 +1347,7 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 	ret = 0;
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		int failure_errno;
 		unsigned int type;
 		struct object_id oid;
 
@@ -1357,9 +1358,9 @@ int is_packed_transaction_needed(struct ref_store *ref_store,
 			 */
 			continue;
 
-		if (!refs_read_raw_ref(ref_store, update->refname,
-				       &oid, &referent, &type) ||
-		    errno != ENOENT) {
+		if (!refs_read_raw_ref(ref_store, update->refname, &oid,
+				       &referent, &type, &failure_errno) ||
+		    failure_errno != ENOENT) {
 			/*
 			 * We have to actually delete that reference
 			 * -> this transaction is needed.
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index ddd6d7f8ebd..7ade286260c 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -149,9 +149,9 @@ struct ref_update {
 	const char refname[FLEX_ARRAY];
 };
 
-int refs_read_raw_ref(struct ref_store *ref_store,
-		      const char *refname, struct object_id *oid,
-		      struct strbuf *referent, unsigned int *type);
+int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
+		      struct object_id *oid, struct strbuf *referent,
+		      unsigned int *type, int *failure_errno);
 
 /*
  * Write an error to `err` and return a nonzero value iff the same
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 6/8] branch tests: test for errno propagating on failing read
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
                                     ` (4 preceding siblings ...)
  2021-08-23 11:52                   ` [PATCH v10 5/8] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 7/8] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 8/8] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

Add a test for "git branch" to cover the case where .git/refs is
symlinked. To check availability, refs_verify_refname_available() will
run refs_read_raw_ref() on each prefix, leading to a read() from
.git/refs (which is a directory).

It would probably be more robust to re-issue the lstat() as a normal
stat(), in which case, we would fall back to the directory case, but
for now let's just test for the existing behavior as-is. This test
covers a regression in a commit that only ever made it to "next", see
[1].

1. http://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3200-branch.sh | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index cc4b10236e2..9fae13c2dea 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -731,6 +731,26 @@ test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for
 	test_must_fail git branch -m u v
 '
 
+test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
+	test_when_finished "rm -rf subdir" &&
+	git init subdir &&
+
+	(
+		cd subdir &&
+		for d in refs objects packed-refs
+		do
+			rm -rf .git/$d &&
+			ln -s ../../.git/$d .git/$d
+		done
+	) &&
+	git --git-dir subdir/.git/ branch rename-src &&
+	git rev-parse rename-src >expect &&
+	git --git-dir subdir/.git/ branch -m rename-src rename-dest &&
+	git rev-parse rename-dest >actual &&
+	test_cmp expect actual &&
+	git branch -D rename-dest
+'
+
 test_expect_success 'test tracking setup via --track' '
 	git config remote.local.url . &&
 	git config remote.local.fetch refs/heads/*:refs/remotes/local/* &&
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 7/8] refs: explicitly return failure_errno from parse_loose_ref_contents
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
                                     ` (5 preceding siblings ...)
  2021-08-23 11:52                   ` [PATCH v10 6/8] branch tests: test for errno propagating on failing read Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  2021-08-23 11:52                   ` [PATCH v10 8/8] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

The EINVAL error from parse_loose_ref_contents is used in files-backend
to create a custom error message.

In untangling this we discovered a tricky edge case. The
refs_read_special_head() function was relying on
parse_loose_ref_contents() setting EINVAL.

By converting it to use "saved_errno" we can migrate away from "errno"
in this part of the code entirely, and do away with an existing
"save_errno" pattern, its only purpose was to not clobber the "errno"
we previously needed at the end of files_read_raw_ref().

Let's assert that we can do that by not having files_read_raw_ref()
itself operate on *failure_errno in addition to passing it on. Instead
we'll assert that if we return non-zero we actually do set errno, thus
assuring ourselves and callers that they can trust the resulting
"failure_errno".

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  8 +++++---
 refs/files-backend.c | 31 ++++++++++++++++++++-----------
 refs/refs-internal.h |  6 ++++--
 t/t3200-branch.sh    |  1 +
 4 files changed, 30 insertions(+), 16 deletions(-)

diff --git a/refs.c b/refs.c
index 4c782fa2978..b83fd8c36b3 100644
--- a/refs.c
+++ b/refs.c
@@ -1654,7 +1654,8 @@ int for_each_fullref_in_prefixes(const char *namespace,
 
 static int refs_read_special_head(struct ref_store *ref_store,
 				  const char *refname, struct object_id *oid,
-				  struct strbuf *referent, unsigned int *type)
+				  struct strbuf *referent, unsigned int *type,
+				  int *failure_errno)
 {
 	struct strbuf full_path = STRBUF_INIT;
 	struct strbuf content = STRBUF_INIT;
@@ -1664,7 +1665,8 @@ static int refs_read_special_head(struct ref_store *ref_store,
 	if (strbuf_read_file(&content, full_path.buf, 0) < 0)
 		goto done;
 
-	result = parse_loose_ref_contents(content.buf, oid, referent, type);
+	result = parse_loose_ref_contents(content.buf, oid, referent, type,
+					  failure_errno);
 
 done:
 	strbuf_release(&full_path);
@@ -1679,7 +1681,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	assert(failure_errno);
 	if (!strcmp(refname, "FETCH_HEAD") || !strcmp(refname, "MERGE_HEAD")) {
 		return refs_read_special_head(ref_store, refname, oid, referent,
-					      type);
+					      type, failure_errno);
 	}
 
 	return ref_store->be->read_raw_ref(ref_store, refname, oid, referent,
diff --git a/refs/files-backend.c b/refs/files-backend.c
index f962495e456..41efe5352b5 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -355,6 +355,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	int fd;
 	int ret = -1;
 	int remaining_retries = 3;
+	int myerr = 0;
 
 	*type = 0;
 	strbuf_reset(&sb_path);
@@ -382,11 +383,13 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 
 	if (lstat(path, &st) < 0) {
 		int ignore_errno;
-		if (errno != ENOENT)
+		myerr = errno;
+		errno = 0;
+		if (myerr != ENOENT)
 			goto out;
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = ENOENT;
+			myerr = ENOENT;
 			goto out;
 		}
 		ret = 0;
@@ -397,7 +400,9 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	if (S_ISLNK(st.st_mode)) {
 		strbuf_reset(&sb_contents);
 		if (strbuf_readlink(&sb_contents, path, st.st_size) < 0) {
-			if (errno == ENOENT || errno == EINVAL)
+			myerr = errno;
+			errno = 0;
+			if (myerr == ENOENT || myerr == EINVAL)
 				/* inconsistent with lstat; retry */
 				goto stat_ref;
 			else
@@ -427,7 +432,7 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 		 */
 		if (refs_read_raw_ref(refs->packed_ref_store, refname, oid,
 				      referent, type, &ignore_errno)) {
-			errno = EISDIR;
+			myerr = EISDIR;
 			goto out;
 		}
 		ret = 0;
@@ -440,7 +445,8 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	 */
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (errno == ENOENT && !S_ISLNK(st.st_mode))
+		myerr = errno;
+		if (myerr == ENOENT && !S_ISLNK(st.st_mode))
 			/* inconsistent with lstat; retry */
 			goto stat_ref;
 		else
@@ -448,26 +454,29 @@ static int files_read_raw_ref(struct ref_store *ref_store, const char *refname,
 	}
 	strbuf_reset(&sb_contents);
 	if (strbuf_read(&sb_contents, fd, 256) < 0) {
-		int save_errno = errno;
+		myerr = errno;
 		close(fd);
-		errno = save_errno;
 		goto out;
 	}
 	close(fd);
 	strbuf_rtrim(&sb_contents);
 	buf = sb_contents.buf;
 
-	ret = parse_loose_ref_contents(buf, oid, referent, type);
+	ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr);
 
 out:
-	*failure_errno = errno;
+	if (ret && !myerr)
+		BUG("returning non-zero %d, should have set myerr!", ret);
+	*failure_errno = myerr;
+
 	strbuf_release(&sb_path);
 	strbuf_release(&sb_contents);
 	return ret;
 }
 
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type)
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno)
 {
 	const char *p;
 	if (skip_prefix(buf, "ref:", &buf)) {
@@ -486,7 +495,7 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid,
 	if (parse_oid_hex(buf, oid, &p) ||
 	    (*p != '\0' && !isspace(*p))) {
 		*type |= REF_ISBROKEN;
-		errno = EINVAL;
+		*failure_errno = EINVAL;
 		return -1;
 	}
 	return 0;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 7ade286260c..7a3a61ac22f 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -692,10 +692,12 @@ struct ref_store {
 };
 
 /*
- * Parse contents of a loose ref file.
+ * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for
+ * invalid contents.
  */
 int parse_loose_ref_contents(const char *buf, struct object_id *oid,
-			     struct strbuf *referent, unsigned int *type);
+			     struct strbuf *referent, unsigned int *type,
+			     int *failure_errno);
 
 /*
  * Fill in the generic part of refs and add it to our collection of
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 9fae13c2dea..63fbd71dc5c 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -745,6 +745,7 @@ test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
 	) &&
 	git --git-dir subdir/.git/ branch rename-src &&
 	git rev-parse rename-src >expect &&
+	# Tests a BUG() assertion in files_read_raw_ref()
 	git --git-dir subdir/.git/ branch -m rename-src rename-dest &&
 	git rev-parse rename-dest >actual &&
 	test_cmp expect actual &&
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

* [PATCH v10 8/8] refs: make errno output explicit for refs_resolve_ref_unsafe
  2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
                                     ` (6 preceding siblings ...)
  2021-08-23 11:52                   ` [PATCH v10 7/8] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
@ 2021-08-23 11:52                   ` Ævar Arnfjörð Bjarmason
  7 siblings, 0 replies; 148+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-23 11:52 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Han-Wen Nienhuys, Michael Haggerty,
	Jonathan Tan, Ævar Arnfjörð Bjarmason

From: Han-Wen Nienhuys <hanwen@google.com>

This introduces refs_resolve_ref_unsafe_with_errno(), which makes the API
contract for the errno output explicit. The implementation still relies on
the global errno variable to ensure no side effects of this refactoring.

lock_ref_oid_basic() in files-backend.c is the only caller of refs_resolve_ref()
that needs error information to make logic decisions, so update that caller

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c               |  2 +-
 refs.h               | 11 +++++++++++
 refs/files-backend.c | 10 ++++++----
 3 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index b83fd8c36b3..e3b6d8f8dc0 100644
--- a/refs.c
+++ b/refs.c
@@ -1688,7 +1688,7 @@ int refs_read_raw_ref(struct ref_store *ref_store, const char *refname,
 					   type, failure_errno);
 }
 
-static const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
 					       const char *refname,
 					       int resolve_flags,
 					       struct object_id *oid,
diff --git a/refs.h b/refs.h
index fda8aef1547..a5685c891a9 100644
--- a/refs.h
+++ b/refs.h
@@ -68,6 +68,17 @@ const char *refs_resolve_ref_unsafe(struct ref_store *refs,
 				    int resolve_flags,
 				    struct object_id *oid,
 				    int *flags);
+/**
+ * refs_resolve_ref_unsafe_with_errno() is like
+ * refs_resolve_ref_unsafe(), but provide access to errno code that
+ * lead to a failure. We guarantee that errno is set to a meaningful
+ * value on non-zero return.
+ */
+const char *refs_resolve_ref_unsafe_with_errno(struct ref_store *refs,
+					       const char *refname,
+					       int resolve_flags,
+					       struct object_id *oid,
+					       int *flags, int *failure_errno);
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 			       struct object_id *oid, int *flags);
 
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 41efe5352b5..a14bb6eb96e 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1001,6 +1001,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 {
 	struct strbuf ref_file = STRBUF_INIT;
 	struct ref_lock *lock;
+	int resolve_errno = 0;
 
 	files_assert_main_repository(refs, "lock_ref_oid_basic");
 	assert(err);
@@ -1008,13 +1009,14 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs,
 	CALLOC_ARRAY(lock, 1);
 
 	files_ref_path(refs, &ref_file, refname);
-	if (!refs_resolve_ref_unsafe(&refs->base, refname,
-				     RESOLVE_REF_NO_RECURSE,
-				     &lock->old_oid, type)) {
+	if (!refs_resolve_ref_unsafe_with_errno(&refs->base, refname,
+						RESOLVE_REF_NO_RECURSE,
+						&lock->old_oid, type,
+						&resolve_errno)) {
 		if (!refs_verify_refname_available(&refs->base, refname,
 						   NULL, NULL, err))
 			strbuf_addf(err, "unable to resolve reference '%s': %s",
-				    refname, strerror(errno));
+				    refname, strerror(resolve_errno));
 
 		goto error_return;
 	}
-- 
2.33.0.662.g438caf9576d


^ permalink raw reply related	[flat|nested] 148+ messages in thread

end of thread, other threads:[~2021-08-23 11:53 UTC | newest]

Thread overview: 148+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-29 15:32 [PATCH 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
2021-04-29 15:32 ` [PATCH 1/8] refs: remove EINVAL specification from the errno sideband in read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-04-30  2:38   ` Junio C Hamano
2021-05-19 12:25     ` Han-Wen Nienhuys
2021-06-03  2:19   ` Jonathan Tan
2021-06-09 11:28     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
2021-04-30  3:10   ` Junio C Hamano
2021-05-19 12:29     ` Han-Wen Nienhuys
2021-06-03  2:33   ` Jonathan Tan
2021-06-10 10:02     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-04-30  3:34   ` Junio C Hamano
2021-04-30  6:02     ` Junio C Hamano
2021-05-19 12:33       ` Han-Wen Nienhuys
2021-06-03  2:37   ` Jonathan Tan
2021-06-10 10:05     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
2021-06-03  2:51   ` Jonathan Tan
2021-06-10 11:27     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 5/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
2021-04-29 15:32 ` [PATCH 6/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
2021-06-03  2:53   ` Jonathan Tan
2021-06-10 11:45     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 7/8] refs: stop setting EINVAL and ELOOP in symref resolution Han-Wen Nienhuys via GitGitGadget
2021-06-03  2:55   ` Jonathan Tan
2021-06-10 11:58     ` Han-Wen Nienhuys
2021-04-29 15:32 ` [PATCH 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
2021-06-03  2:13 ` [PATCH 0/8] refs: cleanup errno sideband ref related functions Jonathan Tan
2021-06-09 11:29   ` Han-Wen Nienhuys
2021-06-10 12:57 ` [PATCH v2 " Han-Wen Nienhuys via GitGitGadget
2021-06-10 12:57   ` [PATCH v2 1/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-06-10 12:57   ` [PATCH v2 2/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
2021-07-01 11:13     ` Ævar Arnfjörð Bjarmason
2021-07-05 14:16       ` Han-Wen Nienhuys
2021-06-10 12:57   ` [PATCH v2 3/8] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-01 11:34     ` Ævar Arnfjörð Bjarmason
2021-07-05 14:34       ` Han-Wen Nienhuys
2021-06-10 12:57   ` [PATCH v2 4/8] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
2021-07-01 11:56     ` Ævar Arnfjörð Bjarmason
2021-06-10 12:57   ` [PATCH v2 5/8] refs: use refs_resolve_ref_unsafe_with_errno() where needed Han-Wen Nienhuys via GitGitGadget
2021-07-01 11:58     ` Ævar Arnfjörð Bjarmason
2021-06-10 12:57   ` [PATCH v2 6/8] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
2021-07-01 12:06     ` Ævar Arnfjörð Bjarmason
2021-06-10 12:57   ` [PATCH v2 7/8] refs: clear errno return in refs_resolve_ref_unsafe() Han-Wen Nienhuys via GitGitGadget
2021-07-01 12:19     ` Ævar Arnfjörð Bjarmason
2021-06-10 12:57   ` [PATCH v2 8/8] refs: explicitly propagate errno from refs_read_raw_ref Han-Wen Nienhuys via GitGitGadget
2021-07-01 12:26     ` Ævar Arnfjörð Bjarmason
2021-07-05 16:09       ` Han-Wen Nienhuys
2021-07-05 19:08         ` Ævar Arnfjörð Bjarmason
2021-07-05 19:39           ` Han-Wen Nienhuys
2021-06-14 10:10   ` [PATCH v2 0/8] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys
2021-07-05 20:56   ` [PATCH v3 0/5] " Han-Wen Nienhuys via GitGitGadget
2021-07-05 20:56     ` [PATCH v3 1/5] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-05 20:56     ` [PATCH v3 2/5] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
2021-07-05 20:56     ` [PATCH v3 3/5] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-05 20:56     ` [PATCH v3 4/5] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
2021-07-06 19:28       ` Junio C Hamano
2021-07-05 20:56     ` [PATCH v3 5/5] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
2021-07-06  0:38     ` [PATCH v3 0/5] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
2021-07-06  9:53       ` Han-Wen Nienhuys
2021-07-06 14:27         ` Ævar Arnfjörð Bjarmason
2021-07-06 18:36           ` Han-Wen Nienhuys
2021-07-06 18:55     ` [PATCH v4 0/6] " Han-Wen Nienhuys via GitGitGadget
2021-07-06 18:55       ` [PATCH v4 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-06 18:55       ` [PATCH v4 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
2021-07-06 18:55       ` [PATCH v4 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-06 18:55       ` [PATCH v4 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
2021-07-06 19:39         ` Junio C Hamano
2021-07-06 18:55       ` [PATCH v4 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
2021-07-06 19:37         ` Junio C Hamano
2021-07-07  8:20           ` Han-Wen Nienhuys
2021-07-06 18:55       ` [PATCH v4 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
2021-07-07 19:07       ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Han-Wen Nienhuys via GitGitGadget
2021-07-07 19:07         ` [PATCH v5 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-07 19:07         ` [PATCH v5 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Han-Wen Nienhuys via GitGitGadget
2021-07-11 11:38           ` Ævar Arnfjörð Bjarmason
2021-07-13  8:00             ` Han-Wen Nienhuys
2021-07-07 19:07         ` [PATCH v5 3/6] refs: make errno output explicit for read_raw_ref_fn Han-Wen Nienhuys via GitGitGadget
2021-07-11 11:52           ` Ævar Arnfjörð Bjarmason
2021-07-07 19:07         ` [PATCH v5 4/6] refs: add failure_errno to refs_read_raw_ref() signature Han-Wen Nienhuys via GitGitGadget
2021-07-11 11:59           ` Ævar Arnfjörð Bjarmason
2021-07-13  8:02             ` Han-Wen Nienhuys
2021-07-07 19:07         ` [PATCH v5 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Han-Wen Nienhuys via GitGitGadget
2021-07-11 12:41           ` Ævar Arnfjörð Bjarmason
2021-07-07 19:07         ` [PATCH v5 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Han-Wen Nienhuys via GitGitGadget
2021-07-07 20:44         ` [PATCH v5 0/6] refs: cleanup errno sideband ref related functions Junio C Hamano
2021-07-11 16:30         ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 01/17] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 02/17] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 03/17] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 04/17] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 05/17] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 06/17] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 07/17] refs: make errno ignoring explicit in lock_ref_oid_basic() Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 08/17] refs file-backend.c: stop setting "EBUSY" in verify_lock() Ævar Arnfjörð Bjarmason
2021-07-13  8:08             ` Han-Wen Nienhuys
2021-07-11 16:30           ` [PATCH v6? 09/17] refs file-backend.c: deal with errno directly " Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 10/17] refs API: remove refs_read_ref_full() wrapper Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 11/17] refs API: make resolve_gitlink_ref() not set errno Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 12/17] refs API: make refs_resolve_ref_unsafe() static Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 13/17] refs API: make refs_resolve_refdup() not set errno Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 14/17] refs API: make refs_ref_exists() " Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 15/17] refs API: make resolve_ref_unsafe() " Ævar Arnfjörð Bjarmason
2021-07-13  8:13             ` Han-Wen Nienhuys
2021-07-14  8:32               ` Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 16/17] refs API: make expand_ref() and repo_dwim_log() " Ævar Arnfjörð Bjarmason
2021-07-11 16:30           ` [PATCH v6? 17/17] refs API: don't leak "errno" in run_transaction_hook() Ævar Arnfjörð Bjarmason
2021-07-13  8:28           ` [PATCH v6? 00/17] refs API: get rid of errno setting entirely Han-Wen Nienhuys
2021-07-13 18:26             ` Ævar Arnfjörð Bjarmason
2021-07-14  8:38               ` Ævar Arnfjörð Bjarmason
2021-07-14  8:43                 ` Han-Wen Nienhuys
2021-07-14 11:43           ` [PATCH v7 0/6] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 1/6] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 2/6] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 3/6] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 4/6] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 5/6] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
2021-07-14 11:43             ` [PATCH v7 6/6] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
2021-07-16 14:22             ` [PATCH v8 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
2021-07-16 14:22               ` [PATCH v8 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
2021-07-20 10:33               ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
2021-07-20 10:33                 ` [PATCH v9 1/7] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
2021-07-20 10:33                 ` [PATCH v9 2/7] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-07-20 10:33                 ` [PATCH v9 3/7] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
2021-07-20 10:33                 ` [PATCH v9 4/7] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-08-16 13:00                   ` Han-Wen Nienhuys
2021-07-20 10:33                 ` [PATCH v9 5/7] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
2021-07-20 10:33                 ` [PATCH v9 6/7] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
2021-08-13 20:54                   ` Jonathan Tan
2021-07-20 10:33                 ` [PATCH v9 7/7] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason
2021-07-26 23:49                 ` [PATCH v9 0/7] refs: cleanup errno sideband ref related functions Ævar Arnfjörð Bjarmason
2021-07-27  0:18                   ` Junio C Hamano
2021-08-23 11:52                 ` [PATCH v10 0/8] " Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 1/8] refs file backend: move raceproof_create_file() here Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 2/8] refs: remove EINVAL errno output from specification of read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 3/8] refs/files-backend: stop setting errno from lock_ref_oid_basic Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 4/8] refs: make errno output explicit for read_raw_ref_fn Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 5/8] refs: add failure_errno to refs_read_raw_ref() signature Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 6/8] branch tests: test for errno propagating on failing read Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 7/8] refs: explicitly return failure_errno from parse_loose_ref_contents Ævar Arnfjörð Bjarmason
2021-08-23 11:52                   ` [PATCH v10 8/8] refs: make errno output explicit for refs_resolve_ref_unsafe Ævar Arnfjörð Bjarmason

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).