From: Christoph Hellwig <hch@lst.de> To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com> Cc: git@vger.kernel.org, tytso@mit.edu, Junio C Hamano <gitster@pobox.com>, Christoph Hellwig <hch@lst.de>, Linus Torvalds <torvalds@linux-foundation.org>, linux-fsdevel <linux-fsdevel@vger.kernel.org> Subject: Re: [RFC PATCH 2/2] core.fsyncObjectFiles: make the docs less flippant Date: Thu, 17 Sep 2020 16:12:57 +0200 Message-ID: <20200917141257.GB27653@lst.de> (raw) In-Reply-To: <20200917112830.26606-3-avarab@gmail.com> > core.fsyncObjectFiles:: > + This boolean will enable 'fsync()' when writing loose object > + files. Both the file itself and its containng directory will > + be fsynced. > ++ > +When git writes data any required object writes will precede the > +corresponding reference update(s). For example, a > +linkgit:git-receive-pack[1] accepting a push might write a pack or > +loose objects (depending on settings such as `transfer.unpackLimit`). > ++ > +Therefore on a journaled file system which ensures that data is > +flushed to disk in chronological order an fsync shouldn't be > +needed. The loose objects might be lost with a crash, but so will the > +ref update that would have referenced them. Git's own state in such a > +crash will remain consistent. While this is much better than what we had before I'm not sure it is all that useful. The only file system I know of that actually had the above behavior was ext3, and the fact that it always wrote back that way made it a complete performance desaster. So even mentioning this here will probably create a lot more confusion than actually clearing things up. > ++ > +This option exists because that assumption doesn't hold on filesystems > +where the data ordering is not preserved, such as on ext3 and ext4 > +with "data=writeback". On such a filesystem the `rename()` that drops > +the new reference in place might be preserved, but the contents or > +directory entry for the loose object(s) might not have been synced to > +disk. As well as just about any other file system. Which is another argument on why it needs to be on by default. Every time I install a new development system (aka one that often crashes) and forget to enable the option I keep corrupting my git repos. And that is with at least btrfs, ext4 and xfs as it is pretty much by design. > +However, that's highly filesystem-dependent, on some filesystems > +simply calling fsync() might force an unrelated bulk background write > +to be serialized to disk. Such edge cases are the reason this option > +is off by default. That default setting might change in future > +versions. Again the only "some file system" that was widely used that did this was ext3. And ext3 has long been removed from the Linux kernel..
next prev parent reply index Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-01-17 18:48 [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig 2018-01-17 19:04 ` Junio C Hamano 2018-01-17 19:35 ` Christoph Hellwig 2018-01-17 20:05 ` Andreas Schwab 2018-01-17 19:37 ` Matthew Wilcox 2018-01-17 19:42 ` Christoph Hellwig 2018-01-17 21:44 ` Ævar Arnfjörð Bjarmason 2018-01-17 22:07 ` Linus Torvalds 2018-01-17 22:25 ` Linus Torvalds 2018-01-17 23:16 ` Ævar Arnfjörð Bjarmason 2018-01-17 23:42 ` Linus Torvalds 2018-01-17 23:52 ` Theodore Ts'o 2018-01-17 23:57 ` Linus Torvalds 2018-01-18 16:27 ` Christoph Hellwig 2018-01-19 19:08 ` Junio C Hamano 2018-01-20 22:14 ` Theodore Ts'o 2018-01-20 22:27 ` Junio C Hamano 2018-01-22 15:09 ` Ævar Arnfjörð Bjarmason 2018-01-22 18:09 ` Theodore Ts'o 2018-01-23 0:47 ` Jeff King 2018-01-23 5:45 ` Theodore Ts'o 2018-01-23 16:17 ` Jeff King 2018-01-23 0:25 ` Jeff King 2018-01-21 21:32 ` Chris Mason 2020-09-17 11:06 ` Ævar Arnfjörð Bjarmason 2020-09-17 11:28 ` [RFC PATCH 0/2] should core.fsyncObjectFiles fsync the dir entry + docs Ævar Arnfjörð Bjarmason 2020-09-17 11:28 ` [RFC PATCH 1/2] sha1-file: fsync() loose dir entry when core.fsyncObjectFiles Ævar Arnfjörð Bjarmason 2020-09-17 13:16 ` Jeff King 2020-09-17 15:09 ` Christoph Hellwig 2020-09-17 14:09 ` Christoph Hellwig 2020-09-17 14:55 ` Jeff King 2020-09-17 14:56 ` Christoph Hellwig 2020-09-17 15:37 ` Junio C Hamano 2020-09-17 17:12 ` Jeff King 2020-09-17 20:37 ` Taylor Blau 2020-09-22 10:42 ` Ævar Arnfjörð Bjarmason 2020-09-17 20:21 ` Johannes Sixt 2020-09-22 8:24 ` Ævar Arnfjörð Bjarmason 2020-11-19 11:38 ` Johannes Schindelin 2020-09-17 11:28 ` [RFC PATCH 2/2] core.fsyncObjectFiles: make the docs less flippant Ævar Arnfjörð Bjarmason 2020-09-17 14:12 ` Christoph Hellwig [this message] 2020-09-17 15:43 ` Junio C Hamano 2020-09-17 20:15 ` Johannes Sixt 2020-10-08 8:13 ` Johannes Schindelin 2020-10-08 15:57 ` Ævar Arnfjörð Bjarmason 2020-10-08 18:53 ` Junio C Hamano 2020-10-09 10:44 ` Johannes Schindelin 2020-09-17 19:21 ` Marc Branchaud 2020-09-17 14:14 ` [PATCH] enable core.fsyncObjectFiles by default Christoph Hellwig 2020-09-17 15:30 ` Junio C Hamano 2018-01-17 20:55 ` Jeff King 2018-01-17 21:10 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200917141257.GB27653@lst.de \ --to=hch@lst.de \ --cc=avarab@gmail.com \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=linux-fsdevel@vger.kernel.org \ --cc=torvalds@linux-foundation.org \ --cc=tytso@mit.edu \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Git Mailing List Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/git/0 git/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 git git/ https://lore.kernel.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git