All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stefan Beller <sbeller@google.com>
To: peff@peff.net, git@vger.kernel.org, gitster@pobox.com,
	mhagger@alum.mit.edu, loic@dachary.org
Cc: Stefan Beller <sbeller@google.com>
Subject: [PATCHv2 5/5] refs.c: enable large transactions
Date: Wed, 21 Jan 2015 18:32:09 -0800	[thread overview]
Message-ID: <1421893929-7447-6-git-send-email-sbeller@google.com> (raw)
In-Reply-To: <1421893929-7447-1-git-send-email-sbeller@google.com>

By closing the file descriptors after creating the lock file we are not
limiting the size of the transaction by the number of available file
descriptors.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 refs.c                | 17 +++++++++++++----
 t/t1400-update-ref.sh |  4 ++--
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 2013d37..9d01102 100644
--- a/refs.c
+++ b/refs.c
@@ -3055,11 +3055,18 @@ int is_branch(const char *refname)
 static int write_sha1_to_lock_file(struct ref_lock *lock,
 				   const unsigned char *sha1)
 {
-	if (fdopen_lock_file(lock->lk, "w") < 0
-	    || fprintf(lock->lk->fp, "%s\n", sha1_to_hex(sha1)) != 41)
+	if (lock->lk->fd == -1) {
+		if (reopen_lock_file(lock->lk) < 0
+		    || fdopen_lock_file(lock->lk, "w") < 0
+		    || fprintf(lock->lk->fp, "%s\n", sha1_to_hex(sha1)) != 41
+		    || close_lock_file(lock->lk) < 0)
+		    return -1;
+	} else {
+		if (fdopen_lock_file(lock->lk, "w") < 0
+		    || fprintf(lock->lk->fp, "%s\n", sha1_to_hex(sha1)) != 41)
 		return -1;
-	else
-		return 0;
+	}
+	return 0;
 }
 
 /*
@@ -3761,6 +3768,8 @@ int ref_transaction_commit(struct ref_transaction *transaction,
 				    update->refname);
 			goto cleanup;
 		}
+		/* Do not keep all lock files open at the same time. */
+		close_lock_file(update->lock->lk);
 	}
 
 	/* Perform updates first so live commits remain referenced */
diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 47d2fe9..c593a1d 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -979,7 +979,7 @@ run_with_limited_open_files () {
 
 test_lazy_prereq ULIMIT_FILE_DESCRIPTORS 'run_with_limited_open_files true'
 
-test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches does not burst open file limit' '
+test_expect_success ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches does not burst open file limit' '
 (
 	for i in $(test_seq 33)
 	do
@@ -990,7 +990,7 @@ test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction creating branches
 )
 '
 
-test_expect_failure ULIMIT_FILE_DESCRIPTORS 'large transaction deleting branches does not burst open file limit' '
+test_expect_success ULIMIT_FILE_DESCRIPTORS 'large transaction deleting branches does not burst open file limit' '
 (
 	for i in $(test_seq 33)
 	do
-- 
2.2.1.62.g3f15098

  parent reply	other threads:[~2015-01-22  2:32 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-22  2:32 [PATCHv2 0/5] Fix bug in large transactions Stefan Beller
2015-01-22  2:32 ` [PATCHv2 1/5] update-ref: test handling large transactions properly Stefan Beller
2015-01-22 10:54   ` Michael Haggerty
2015-01-22 13:07     ` Jeff King
2015-01-22  2:32 ` [PATCHv2 2/5] t7004: rename ULIMIT test prerequisite to ULIMIT_STACK_SIZE Stefan Beller
2015-01-22  2:32 ` [PATCHv2 3/5] refs.c: remove lock_fd from struct ref_lock Stefan Beller
2015-01-22  2:32 ` [PATCHv2 4/5] refs.c: have a write_sha1_to_lock_file wrapper Stefan Beller
2015-01-22  2:32 ` Stefan Beller [this message]
2015-01-22 11:24   ` [PATCHv2 5/5] refs.c: enable large transactions Michael Haggerty
2015-01-22 13:10     ` Jeff King
2015-01-22 16:33       ` Michael Haggerty
2015-01-22 19:24         ` Stefan Beller
2015-01-22 23:11     ` [RFC PATCH 0/5] So you dislike the sequence of system calls? Stefan Beller
2015-01-22 23:11       ` [PATCH 1/5] fixup for "refs.c: enable large transactions" Stefan Beller
2015-01-22 23:11       ` [PATCH 2/5] refs.c: remove unlock_ref from write_ref_sha1 Stefan Beller
2015-01-22 23:11       ` [PATCH 3/5] refs.c: move static functions to close and commit refs Stefan Beller
2015-01-22 23:11       ` [PATCH 4/5] refs.c: remove committing the ref from write_ref_sha1 Stefan Beller
2015-01-22 23:11       ` [PATCH 5/5] refs.c: write values to lock files early for committing Stefan Beller
2015-01-22 12:59   ` [PATCHv2 5/5] refs.c: enable large transactions Ramsay Jones
2015-01-22 19:16     ` Stefan Beller
2015-01-22 19:51       ` Ramsay Jones
2015-01-22 20:13         ` Ramsay Jones
2015-01-22 20:20           ` Stefan Beller
2015-01-22 20:59             ` Ramsay Jones
2015-01-22 12:05 ` [PATCHv2 0/5] Fix bug in " Michael Haggerty
2015-01-23 20:03   ` [PATCHv3 0/6] " Stefan Beller
2015-01-23 20:03     ` [PATCHv3 1/6] update-ref: test handling large transactions properly Stefan Beller
2015-01-23 20:03     ` [PATCHv3 2/6] t7004: rename ULIMIT test prerequisite to ULIMIT_STACK_SIZE Stefan Beller
2015-01-23 20:03     ` [PATCHv3 3/6] refs.c: remove lock_fd from struct ref_lock Stefan Beller
2015-01-23 20:04     ` [PATCHv3 4/6] refs.c: move static functions to close and commit refs Stefan Beller
2015-01-23 20:04     ` [PATCHv3 5/6] refs.c: remove unlock_ref and commit_ref from write_ref_sha1 Stefan Beller
2015-01-23 23:57       ` Junio C Hamano
2015-01-24  0:22         ` Stefan Beller
2015-01-24  0:39           ` Junio C Hamano
2015-01-24  1:04             ` Stefan Beller
2015-01-24  1:29               ` Junio C Hamano
2015-01-23 20:04     ` [PATCHv3 6/6] refs.c: enable large transactions Stefan Beller
2015-01-24  0:14       ` Junio C Hamano
2015-01-24  0:24         ` Stefan Beller
2015-01-24  0:38           ` Junio C Hamano
2015-01-26 19:30             ` Stefan Beller
2015-01-26 21:10               ` [PATCH] refs.c: clean up write_ref_sha1 returns Stefan Beller
2015-01-27  3:22                 ` Junio C Hamano
2015-01-28 21:35                   ` Stefan Beller
2015-01-27  3:17               ` [PATCHv3 6/6] refs.c: enable large transactions Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1421893929-7447-6-git-send-email-sbeller@google.com \
    --to=sbeller@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=loic@dachary.org \
    --cc=mhagger@alum.mit.edu \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.