Re: fsync() after close() and re-open -- insights appreciated

Re: fsync() after close() and re-open -- insights appreciated

[Amir Goldstein]

On Thu, May 3, 2018 at 3:30 PM, Niklas Hambüchen <mail@nh2.me> wrote:
> Hello,
>
> here's a simple question
> I'm having trouble finding answers to that is
> probably trivial to answer for file system developers.
>

Not trivial :)

Are you asking the question without being aware of all the discussions
that led to this patch https://lkml.org/lkml/2018/4/23/994

> Does a sequence of close()/re-open()/fsync() provide the same durability
> guarantees as fsync()/close()?

The short answer is no, the latter provides a better guaranty.
The longer answer is that durability guarantees depends on kernel version,
because situation has been changing in v4.13, v4.14 and now again in
v4.17-rc and stable kernels.

>
> http://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html
> specifies that
>
>   "The fsync() function shall request that all data for the open file
> descriptor named by fildes is to be transferred to the storage device"
>
> which seems to restrict the statement to "for the open file descriptor",
> possibly suggesting that close()/open()/fsync() may not have the desired
> effect.
>
> Being able to link to an authorative answer would be very appreciated.
>

Sadly, there is no documentation at the level that you desire.

Thanks,
Amir.

Re: fsync() after close() and re-open -- insights appreciated

[Niklas Hambüchen]

Hello Amir,

thanks a lot for your reply!

> Are you asking the question without being aware of all the discussions
> that led to this patch https://lkml.org/lkml/2018/4/23/994

Yes, I was not aware of those.

Am I right assuming that what you mention is what Posgres is now
referring to as "fsyncgate" (https://wiki.postgresql.org/wiki/Fsync_Errors)?

> The short answer is no, the latter provides a better guaranty.
> The longer answer is that durability guarantees depends on kernel version,
> because situation has been changing in v4.13, v4.14 and now again in
> v4.17-rc and stable kernels.

Thanks for this summary.

>> Being able to link to an authorative answer would be very appreciated.
>
> Sadly, there is no documentation at the level that you desire.

Ah, I was calling the the answer I might get on this list the
authoritative answer that I would link to :)

Niklas

Re: fsync() after close() and re-open -- insights appreciated

[Amir Goldstein]

On Thu, May 3, 2018 at 5:30 PM, Niklas Hambüchen <mail@nh2.me> wrote:
> Hello Amir,
>
> thanks a lot for your reply!
>
>> Are you asking the question without being aware of all the discussions
>> that led to this patch https://lkml.org/lkml/2018/4/23/994
>
> Yes, I was not aware of those.
>
> Am I right assuming that what you mention is what Posgres is now
> referring to as "fsyncgate" (https://wiki.postgresql.org/wiki/Fsync_Errors)?
>

Correct.

Thanks,
Amir.