* [PATCH 0/1] Fix for mh/packed-ref-store-prep
@ 2017-06-12 8:06 Michael Haggerty
2017-06-12 8:06 ` [PATCH 1/1] lock_packed_refs(): fix cache validity check Michael Haggerty
0 siblings, 1 reply; 3+ messages in thread
From: Michael Haggerty @ 2017-06-12 8:06 UTC (permalink / raw)
To: Junio C Hamano
Cc: Nguyễn Thái Ngọc Duy, Stefan Beller, Jeff King,
Ævar Arnfjörð Bjarmason, David Turner,
Brandon Williams, Johannes Sixt, git, Michael Haggerty
When preparing a new patch series, I noticed a bug that crept into
mh/packed-ref-store-prep while I was shuffling patches around. It
introduces a race condition if another process rewrites the
`packed-refs` file just before we lock it. See the commit message for
more information.
This patch applies on top of mh/packed-ref-store-prep and merges to
master without problems.
Michael
Michael Haggerty (1):
lock_packed_refs(): fix cache validity check
refs/files-backend.c | 32 +++++++++++++++++++++++---------
1 file changed, 23 insertions(+), 9 deletions(-)
--
2.11.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 1/1] lock_packed_refs(): fix cache validity check
2017-06-12 8:06 [PATCH 0/1] Fix for mh/packed-ref-store-prep Michael Haggerty
@ 2017-06-12 8:06 ` Michael Haggerty
2017-06-12 18:21 ` Jeff King
0 siblings, 1 reply; 3+ messages in thread
From: Michael Haggerty @ 2017-06-12 8:06 UTC (permalink / raw)
To: Junio C Hamano
Cc: Nguyễn Thái Ngọc Duy, Stefan Beller, Jeff King,
Ævar Arnfjörð Bjarmason, David Turner,
Brandon Williams, Johannes Sixt, git, Michael Haggerty
Commit 28ed9830b1 (get_packed_ref_cache(): assume "packed-refs" won't
change while locked, 2017-05-22) assumes that the "packed-refs" file
cannot change while we hold the lock. That assumption is
justified *if* the lock has been held the whole time since the
"packed-refs" file was last read.
But in `lock_packed_refs()`, we ourselves lock the "packed-refs" file
and then call `get_packed_ref_cache()` to ensure that the cache agrees
with the file. The intent is to guard against the possibility that
another process changed the "packed-refs" file the moment before we
locked it.
This check was defeated because `get_packed_ref_cache()` saw that the
file was locked, and therefore didn't do the `stat_validity_check()`
that we want.
The mistake was compounded with a misleading comment in
`lock_packed_refs()` claiming that it was doing the right thing. That
comment came from an earlier draft of the mh/packed-ref-store-prep
patch series when the commits were in a different order.
So instead:
* Extract a function `validate_packed_ref_cache()` that does the
validity check independent of whether the lock is held.
* Change `get_packed_ref_cache()` to call the new function, but only
if the lock *isn't* held.
* Change `lock_packed_refs()` to call the new function in any case
before calling `get_packed_ref_cache()`.
* Fix the comment in `lock_packed_refs()`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
If anybody is curious, patch 28ed9830b1 originally appeared before
00d174489e (files_ref_store: put the packed files lock directly in
this struct, 2017-05-22), when `refs->packed_refs_lock` could be left
NULL until after `get_packed_ref_cache()` was called. In that version
of the patch series, the breakage occurred when the the lock was moved
and the check in `get_packed_ref_cache()` was changed from
`refs->packed_refs_lock` to
`is_lock_file_locked(&refs->packed_refs_lock)`. So either way, it was
the combination of the two patches that caused the regression.
refs/files-backend.c | 32 +++++++++++++++++++++++---------
1 file changed, 23 insertions(+), 9 deletions(-)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index d8b3f73147..b040bb3b0a 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -369,6 +369,18 @@ static void files_ref_path(struct files_ref_store *refs,
}
}
+/*
+ * Check that the packed refs cache (if any) still reflects the
+ * contents of the file. If not, clear the cache.
+ */
+static void validate_packed_ref_cache(struct files_ref_store *refs)
+{
+ if (refs->packed &&
+ !stat_validity_check(&refs->packed->validity,
+ files_packed_refs_path(refs)))
+ clear_packed_ref_cache(refs);
+}
+
/*
* Get the packed_ref_cache for the specified files_ref_store,
* creating and populating it if it hasn't been read before or if the
@@ -381,10 +393,8 @@ static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *ref
{
const char *packed_refs_file = files_packed_refs_path(refs);
- if (refs->packed &&
- !is_lock_file_locked(&refs->packed_refs_lock) &&
- !stat_validity_check(&refs->packed->validity, packed_refs_file))
- clear_packed_ref_cache(refs);
+ if (!is_lock_file_locked(&refs->packed_refs_lock))
+ validate_packed_ref_cache(refs);
if (!refs->packed)
refs->packed = read_packed_refs(packed_refs_file);
@@ -1311,13 +1321,17 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)
&refs->packed_refs_lock, files_packed_refs_path(refs),
flags, timeout_value) < 0)
return -1;
+
/*
- * Get the current packed-refs while holding the lock. It is
- * important that we call `get_packed_ref_cache()` before
- * setting `packed_ref_cache->lock`, because otherwise the
- * former will see that the file is locked and assume that the
- * cache can't be stale.
+ * Now that we hold the `packed-refs` lock, make sure that our
+ * cache matches the current version of the file. Normally
+ * `get_packed_ref_cache()` does that for us, but that
+ * function assumes that when the file is locked, any existing
+ * cache is still valid. We've just locked the file, but it
+ * might have changed the moment *before* we locked it.
*/
+ validate_packed_ref_cache(refs);
+
packed_ref_cache = get_packed_ref_cache(refs);
/* Increment the reference count to prevent it from being freed: */
acquire_packed_ref_cache(packed_ref_cache);
--
2.11.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH 1/1] lock_packed_refs(): fix cache validity check
2017-06-12 8:06 ` [PATCH 1/1] lock_packed_refs(): fix cache validity check Michael Haggerty
@ 2017-06-12 18:21 ` Jeff King
0 siblings, 0 replies; 3+ messages in thread
From: Jeff King @ 2017-06-12 18:21 UTC (permalink / raw)
To: Michael Haggerty
Cc: Junio C Hamano, Nguyễn Thái Ngọc Duy,
Stefan Beller, Ævar Arnfjörð Bjarmason,
David Turner, Brandon Williams, Johannes Sixt, git
On Mon, Jun 12, 2017 at 10:06:13AM +0200, Michael Haggerty wrote:
> So instead:
>
> * Extract a function `validate_packed_ref_cache()` that does the
> validity check independent of whether the lock is held.
>
> * Change `get_packed_ref_cache()` to call the new function, but only
> if the lock *isn't* held.
>
> * Change `lock_packed_refs()` to call the new function in any case
> before calling `get_packed_ref_cache()`.
>
> * Fix the comment in `lock_packed_refs()`.
Your explanation makes perfect sense, and I agree this should fix the
problem. Sorry to have missed it in review.
-Peff
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-06-12 18:21 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-12 8:06 [PATCH 0/1] Fix for mh/packed-ref-store-prep Michael Haggerty
2017-06-12 8:06 ` [PATCH 1/1] lock_packed_refs(): fix cache validity check Michael Haggerty
2017-06-12 18:21 ` Jeff King
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.