Re: [PATCH 2/7] core.fsyncmethod: batched disk flushes for loose-objects

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 2515 bytes --]

On Wed, Mar 16, 2022 at 11:21:56AM -0700, Neeraj Singh wrote:
> On Wed, Mar 16, 2022 at 12:31 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > On Tue, Mar 15, 2022 at 09:30:54PM +0000, Neeraj Singh via GitGitGadget wrote:
> > > From: Neeraj Singh <neerajsi@microsoft.com>
> > > diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt
> > > index 062e5259905..c041ed33801 100644
> > > --- a/Documentation/config/core.txt
> > > +++ b/Documentation/config/core.txt
> > > @@ -628,6 +628,11 @@ core.fsyncMethod::
> > >  * `writeout-only` issues pagecache writeback requests, but depending on the
> > >    filesystem and storage hardware, data added to the repository may not be
> > >    durable in the event of a system crash. This is the default mode on macOS.
> > > +* `batch` enables a mode that uses writeout-only flushes to stage multiple
> > > +  updates in the disk writeback cache and then a single full fsync to trigger
> > > +  the disk cache flush at the end of the operation. This mode is expected to
> > > +  be as safe as `fsync` on macOS for repos stored on HFS+ or APFS filesystems
> > > +  and on Windows for repos stored on NTFS or ReFS filesystems.
> >
> > This mode will not be supported by all parts of our stack that use our
> > new fsync infra. So I think we should both document that some parts of
> > the stack don't support batching, and say what the fallback behaviour is
> > for those that don't.
> >
> 
> Can do. I'm hoping that you'll revive your batch-mode refs change too so that
> we get batching across the ODB and Refs, which are the two data stores that
> may receive many updates in a single Git command.

Huh, I completely forgot that my previous implementation already had
such a mechanism. I may have a go at it again, but it would take me a
while given that I'll be OOO most of April.

> This documentation
> comment will read:
> ```
> * `batch` enables a mode that uses writeout-only flushes to stage multiple
>   updates in the disk writeback cache and then does a single full fsync of
>   a dummy file to trigger the disk cache flush at the end of the operation.
>   Currently `batch` mode only applies to loose-object files. Other repository
>   data is made durable as if `fsync` was specified. This mode is expected to
>   be as safe as `fsync` on macOS for repos stored on HFS+ or APFS filesystems
>   and on Windows for repos stored on NTFS or ReFS filesystems.
> ```

Reads good to me, thanks!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]