From: Christoph Hellwig <hch@lst.de> To: Junio C Hamano <gitster@pobox.com> Cc: Christoph Hellwig <hch@lst.de>, git@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH] enable core.fsyncObjectFiles by default Date: Wed, 17 Jan 2018 20:35:10 +0100 [thread overview] Message-ID: <20180117193510.GA30657@lst.de> (raw) In-Reply-To: <xmqqd128s3wf.fsf@gitster.mtv.corp.google.com> On Wed, Jan 17, 2018 at 11:04:32AM -0800, Junio C Hamano wrote: > I am somewhat sympathetic to the desire to flip the default to > "safe" and allow those who know they are already safe to tweak the > knob for performance, and it also makes sense to document that the > default is "true" here. But I do not see the point of removing the > four lines from this paragraph; the sole effect of the removal is to > rob information from readers that they can use to decide if they > want to disable the configuration, no? Does this sound any better? It is a little to technical for my taste, but I couldn't come up with anything better: --- From ab8f2d38dfe40e74de5399af0d069427c7473b76 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig <hch@lst.de> Date: Wed, 17 Jan 2018 19:42:46 +0100 Subject: enable core.fsyncObjectFiles by default fsync is required for data integrity as there is no gurantee that data makes it to disk at any specified time without it. Even for ext3 with data=ordered mode the file system will only commit all data at some point in time that is not guaranteed. I've lost data on development machines with various times countless times due to the lack of this option, and now lost trees on a git server with ext4 as well yesterday. It's time to make git safe by default. Signed-off-by: Christoph Hellwig <hch@lst.de> --- Documentation/config.txt | 11 +++++++---- environment.c | 2 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 0e25b2c92..8b99f1389 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -866,10 +866,13 @@ core.whitespace:: core.fsyncObjectFiles:: This boolean will enable 'fsync()' when writing object files. + -This is a total waste of time and effort on a filesystem that orders -data writes properly, but can be useful for filesystems that do not use -journalling (traditional UNIX filesystems) or that only journal metadata -and not file contents (OS X's HFS+, or Linux ext3 with "data=writeback"). +This option is enabled by default and ensures actual data integrity +by calling fsync after writing object files. + +Note that this might be really slow on ext3 in the traditional +data=ordered mode, in which case you might want to disable this option. +ext3 in data=ordered mode will order the actual data writeout with +metadata operation, but not actually guarantee data integrity either. core.preloadIndex:: Enable parallel index preload for operations like 'git diff' diff --git a/environment.c b/environment.c index 63ac38a46..c74375b5e 100644 --- a/environment.c +++ b/environment.c @@ -36,7 +36,7 @@ const char *git_hooks_path; int zlib_compression_level = Z_BEST_SPEED; int core_compression_level; int pack_compression_level = Z_DEFAULT_COMPRESSION; -int fsync_object_files; +int fsync_object_files = 1; size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE; size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT; size_t delta_base_cache_limit = 96 * 1024 * 1024; -- 2.14.2
WARNING: multiple messages have this Message-ID (diff)
From: Christoph Hellwig <hch@lst.de> To: Junio C Hamano <gitster@pobox.com> Cc: Christoph Hellwig <hch@lst.de>, git@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH] enable core.fsyncObjectFiles by default Date: Wed, 17 Jan 2018 20:35:10 +0100 [thread overview] Message-ID: <20180117193510.GA30657@lst.de> (raw) In-Reply-To: <xmqqd128s3wf.fsf@gitster.mtv.corp.google.com> On Wed, Jan 17, 2018 at 11:04:32AM -0800, Junio C Hamano wrote: > I am somewhat sympathetic to the desire to flip the default to > "safe" and allow those who know they are already safe to tweak the > knob for performance, and it also makes sense to document that the > default is "true" here. But I do not see the point of removing the > four lines from this paragraph; the sole effect of the removal is to > rob information from readers that they can use to decide if they > want to disable the configuration, no? Does this sound any better? It is a little to technical for my taste, but I couldn't come up with anything better: --- >From ab8f2d38dfe40e74de5399af0d069427c7473b76 Mon Sep 17 00:00:00 2001 From: Christoph Hellwig <hch@lst.de> Date: Wed, 17 Jan 2018 19:42:46 +0100 Subject: enable core.fsyncObjectFiles by default fsync is required for data integrity as there is no gurantee that data makes it to disk at any specified time without it. Even for ext3 with data=ordered mode the file system will only commit all data at some point in time that is not guaranteed. I've lost data on development machines with various times countless times due to the lack of this option, and now lost trees on a git server with ext4 as well yesterday. It's time to make git safe by default. Signed-off-by: Christoph Hellwig <hch@lst.de> --- Documentation/config.txt | 11 +++++++---- environment.c | 2 +- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 0e25b2c92..8b99f1389 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -866,10 +866,13 @@ core.whitespace:: core.fsyncObjectFiles:: This boolean will enable 'fsync()' when writing object files. + -This is a total waste of time and effort on a filesystem that orders -data writes properly, but can be useful for filesystems that do not use -journalling (traditional UNIX filesystems) or that only journal metadata -and not file contents (OS X's HFS+, or Linux ext3 with "data=writeback"). +This option is enabled by default and ensures actual data integrity +by calling fsync after writing object files. + +Note that this might be really slow on ext3 in the traditional +data=ordered mode, in which case you might want to disable this option. +ext3 in data=ordered mode will order the actual data writeout with +metadata operation, but not actually guarantee data integrity either. core.preloadIndex:: Enable parallel index preload for operations like 'git diff' diff --git a/environment.c b/environment.c index 63ac38a46..c74375b5e 100644 --- a/environment.c +++ b/environment.c @@ -36,7 +36,7 @@ const char *git_hooks_path; int zlib_compression_level = Z_BEST_SPEED; int core_compression_level; int pack_compression_level = Z_DEFAULT_COMPRESSION; -int fsync_object_files; +int fsync_object_files = 1; size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE; size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT; size_t delta_base_cache_limit = 96 * 1024 * 1024; -- 2.14.2
next prev parent reply other threads:[~2018-01-17 19:35 UTC|newest] Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-01-17 18:48 [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig 2018-01-17 19:04 ` Junio C Hamano 2018-01-17 19:35 ` Christoph Hellwig [this message] 2018-01-17 19:35 ` Christoph Hellwig 2018-01-17 20:05 ` Andreas Schwab 2018-01-17 19:37 ` Matthew Wilcox 2018-01-17 19:42 ` Christoph Hellwig 2018-01-17 21:44 ` Ævar Arnfjörð Bjarmason 2018-01-17 22:07 ` Linus Torvalds 2018-01-17 22:25 ` Linus Torvalds 2018-01-17 23:16 ` Ævar Arnfjörð Bjarmason 2018-01-17 23:42 ` Linus Torvalds 2018-01-17 23:52 ` Theodore Ts'o 2018-01-17 23:57 ` Linus Torvalds 2018-01-18 16:27 ` Christoph Hellwig 2018-01-19 19:08 ` Junio C Hamano 2018-01-20 22:14 ` Theodore Ts'o 2018-01-20 22:27 ` Junio C Hamano 2018-01-22 15:09 ` Ævar Arnfjörð Bjarmason 2018-01-22 18:09 ` Theodore Ts'o 2018-01-22 18:09 ` Theodore Ts'o 2018-01-23 0:47 ` Jeff King 2018-01-23 5:45 ` Theodore Ts'o 2018-01-23 5:45 ` Theodore Ts'o 2018-01-23 16:17 ` Jeff King 2018-01-23 0:25 ` Jeff King 2018-01-21 21:32 ` Chris Mason 2020-09-17 11:06 ` Ævar Arnfjörð Bjarmason 2020-09-17 11:28 ` [RFC PATCH 0/2] should core.fsyncObjectFiles fsync the dir entry + docs Ævar Arnfjörð Bjarmason 2020-09-17 11:28 ` [RFC PATCH 1/2] sha1-file: fsync() loose dir entry when core.fsyncObjectFiles Ævar Arnfjörð Bjarmason 2020-09-17 13:16 ` Jeff King 2020-09-17 15:09 ` Christoph Hellwig 2020-09-17 14:09 ` Christoph Hellwig 2020-09-17 14:55 ` Jeff King 2020-09-17 14:56 ` Christoph Hellwig 2020-09-17 15:37 ` Junio C Hamano 2020-09-17 17:12 ` Jeff King 2020-09-17 20:37 ` Taylor Blau 2020-09-22 10:42 ` Ævar Arnfjörð Bjarmason 2020-09-17 20:21 ` Johannes Sixt 2020-09-22 8:24 ` Ævar Arnfjörð Bjarmason 2020-11-19 11:38 ` Johannes Schindelin 2020-09-17 11:28 ` [RFC PATCH 2/2] core.fsyncObjectFiles: make the docs less flippant Ævar Arnfjörð Bjarmason 2020-09-17 14:12 ` Christoph Hellwig 2020-09-17 15:43 ` Junio C Hamano 2020-09-17 20:15 ` Johannes Sixt 2020-10-08 8:13 ` Johannes Schindelin 2020-10-08 15:57 ` Ævar Arnfjörð Bjarmason 2020-10-08 18:53 ` Junio C Hamano 2020-10-09 10:44 ` Johannes Schindelin 2020-09-17 19:21 ` Marc Branchaud 2020-09-17 14:14 ` [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig 2020-09-17 15:30 ` Junio C Hamano 2018-01-17 20:55 ` Jeff King 2018-01-17 21:10 ` Christoph Hellwig -- strict thread matches above, loose matches on Subject: below -- 2015-06-23 21:57 [PATCH] Enable " Stefan Beller 2015-06-23 22:21 ` Junio C Hamano 2015-06-23 23:29 ` Theodore Ts'o 2015-06-24 5:32 ` Junio C Hamano 2015-06-24 14:30 ` Theodore Ts'o 2015-06-24 1:07 ` Duy Nguyen 2015-06-24 3:37 ` Jeff King 2015-06-24 5:20 ` Junio C Hamano
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180117193510.GA30657@lst.de \ --to=hch@lst.de \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=linux-fsdevel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.