lazytime mount option—no support in Btrfs

lazytime mount option—no support in Btrfs

[Adam Hunt]

Back in 2014 Ted Tso introduced the lazytime mount option for ext4 and 
shortly thereafter a more generic VFS implementation which was then 
merged into mainline. His early patches included support for Btrfs but 
those changes were removed prior to the feature being merged. His 
changelog includes the following note about the removal:

  - Per Christoph's suggestion, drop support for btrfs and xfs for now, 
    issues with how btrfs and xfs handle dirty inode tracking.  We can add
    btrfs and xfs support back later or at the end of this series if we
    want to revisit this decision. 

My reading of the current mainline shows that Btrfs still lacks any 
support for lazytime. Has any thought been given to adding support for 
lazytime to Btrfs?

Thanks,

Adam

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2017-08-13 07:50, Adam Hunt wrote:
> Back in 2014 Ted Tso introduced the lazytime mount option for ext4 and
> shortly thereafter a more generic VFS implementation which was then
> merged into mainline. His early patches included support for Btrfs but
> those changes were removed prior to the feature being merged. His
> changelog includes the following note about the removal:
> 
>    - Per Christoph's suggestion, drop support for btrfs and xfs for now,
>      issues with how btrfs and xfs handle dirty inode tracking.  We can add
>      btrfs and xfs support back later or at the end of this series if we
>      want to revisit this decision.
> 
> My reading of the current mainline shows that Btrfs still lacks any
> support for lazytime. Has any thought been given to adding support for
> lazytime to Btrfs?
It has bee at least lightly discussed (I forget the thread, but I did a 
reasonably specific explanation of the interaction of the *atime and 
lazytime options in a thread a while back when trying to explain to 
someone why I wanted to be able to run with noatime and lazytime), but I 
don't think the discussion got anywhere.

I would personally love to see support for it myself (or you know, at 
least have some warning that it isn't supported instead of just silently 
accepting and ignoring it like we do currently), but I unfortunately 
don't have the time or expertise to work on implementing it.  If someone 
does post a patch though, I'll be more than happy to throw a few dozen 
VM's at testing it.

Re: lazytime mount option—no support in Btrfs

[waxhead]

Adam Hunt wrote:
> Back in 2014 Ted Tso introduced the lazytime mount option for ext4 and
> shortly thereafter a more generic VFS implementation which was then
> merged into mainline. His early patches included support for Btrfs but
> those changes were removed prior to the feature being merged. His
> changelog includes the following note about the removal:
> 
>    - Per Christoph's suggestion, drop support for btrfs and xfs for now,
>      issues with how btrfs and xfs handle dirty inode tracking.  We can add
>      btrfs and xfs support back later or at the end of this series if we
>      want to revisit this decision.
> 
> My reading of the current mainline shows that Btrfs still lacks any
> support for lazytime. Has any thought been given to adding support for
> lazytime to Btrfs?
> 
> Thanks,
> 
> Adam
> 

Is there any new regarding this?

Re: lazytime mount option—no support in Btrfs

[Martin Steigerwald]

waxhead - 18.08.18, 22:45:
> Adam Hunt wrote:
> > Back in 2014 Ted Tso introduced the lazytime mount option for ext4
> > and shortly thereafter a more generic VFS implementation which was
> > then merged into mainline. His early patches included support for
> > Btrfs but those changes were removed prior to the feature being
> > merged. His> 
> > changelog includes the following note about the removal:
> >    - Per Christoph's suggestion, drop support for btrfs and xfs for
> >    now,
> >    
> >      issues with how btrfs and xfs handle dirty inode tracking.  We
> >      can add btrfs and xfs support back later or at the end of this
> >      series if we want to revisit this decision.
> > 
> > My reading of the current mainline shows that Btrfs still lacks any
> > support for lazytime. Has any thought been given to adding support
> > for lazytime to Btrfs?
[…]
> Is there any new regarding this?

I´d like to know whether there is any news about this as well.

If I understand it correctly this could even help BTRFS performance a 
lot cause it is COW´ing metadata.

Thanks,
-- 
Martin

Re: lazytime mount option—no support in Btrfs

[Andrei Borzenkov]

Отправлено с iPhone

> 19 авг. 2018 г., в 11:37, Martin Steigerwald <martin@lichtvoll.de> написал(а):
> 
> waxhead - 18.08.18, 22:45:
>> Adam Hunt wrote:
>>> Back in 2014 Ted Tso introduced the lazytime mount option for ext4
>>> and shortly thereafter a more generic VFS implementation which was
>>> then merged into mainline. His early patches included support for
>>> Btrfs but those changes were removed prior to the feature being
>>> merged. His> 
>>> changelog includes the following note about the removal:
>>>   - Per Christoph's suggestion, drop support for btrfs and xfs for
>>>   now,
>>> 
>>>     issues with how btrfs and xfs handle dirty inode tracking.  We
>>>     can add btrfs and xfs support back later or at the end of this
>>>     series if we want to revisit this decision.
>>> 
>>> My reading of the current mainline shows that Btrfs still lacks any
>>> support for lazytime. Has any thought been given to adding support
>>> for lazytime to Btrfs?
> […]
>> Is there any new regarding this?
> 
> I´d like to know whether there is any news about this as well.
> 
> If I understand it correctly this could even help BTRFS performance a 
> lot cause it is COW´ing metadata.
> 

I do not see how btrfs can support it exactly due to cow. Modified atime means checksum no more matches so you must update all related metadata. At which point you have kind of shadow in-memory metadata trees. And if this metadata is not written out, then some other metadata that refers to them becomes invalid.

I suspect any file system that keeps checksums of metadata will run into the same issue.

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-19 06:25, Andrei Borzenkov wrote:
> 
> 
> Отправлено с iPhone
> 
>> 19 авг. 2018 г., в 11:37, Martin Steigerwald <martin@lichtvoll.de> написал(а):
>>
>> waxhead - 18.08.18, 22:45:
>>> Adam Hunt wrote:
>>>> Back in 2014 Ted Tso introduced the lazytime mount option for ext4
>>>> and shortly thereafter a more generic VFS implementation which was
>>>> then merged into mainline. His early patches included support for
>>>> Btrfs but those changes were removed prior to the feature being
>>>> merged. His>
>>>> changelog includes the following note about the removal:
>>>>    - Per Christoph's suggestion, drop support for btrfs and xfs for
>>>>    now,
>>>>
>>>>      issues with how btrfs and xfs handle dirty inode tracking.  We
>>>>      can add btrfs and xfs support back later or at the end of this
>>>>      series if we want to revisit this decision.
>>>>
>>>> My reading of the current mainline shows that Btrfs still lacks any
>>>> support for lazytime. Has any thought been given to adding support
>>>> for lazytime to Btrfs?
>> […]
>>> Is there any new regarding this?
>>
>> I´d like to know whether there is any news about this as well.
>>
>> If I understand it correctly this could even help BTRFS performance a
>> lot cause it is COW´ing metadata.
>>
> 
> I do not see how btrfs can support it exactly due to cow. Modified atime means checksum no more matches so you must update all related metadata. At which point you have kind of shadow in-memory metadata trees. And if this metadata is not written out, then some other metadata that refers to them becomes invalid.
I think you might be misunderstanding something here, either how 
lazytime actually works, or how BTRFS checksumming works.

Lazytime prevents timestamp updates from triggering writeback of a 
cached inode.  Other changes will trigger writeback, as will anything 
that evicts the inode from the cache, and an automatic writeback will be 
triggered if the timestamp changed more than 24 hours ago, but until any 
of those situations happens, no writeback will be triggered.

BTRFS checksumming only verifies checksums of blocks which are being 
read.  If the inode is in the cache (which it has to be for lazytime to 
have _any_ effect on it), the block containing it on disk does not need 
to be read, so no checksum verification happens.  Even if there was 
verification, we would not be verifying blocks that are in memory using 
the on-disk checksums (because that would break writeback caching, which 
we already do and already works correctly).

So, given all this, the only inconsistency on-disk for BTRFS with this 
would be identical to the inconsistency it causes for other filesystems, 
namely that mtimes and atimes may not be accurate.

Also, slightly OT, but atimes are not where the real benefit is here for 
most people.  No sane software other than mutt uses atimes (and mutt's 
use of them is not sane, but that's a different argument), so pretty 
much everyone who wants to avoid the overhead from them can just use the 
`noatime` mount option.  The real benefit for most people is with 
mtimes, for which there is no other way to limit the impact they have on 
performance.
> 
> I suspect any file system that keeps checksums of metadata will run into the same issue.
> 
Nope, only if they verify checksums on stuff that's already cached _and_ 
they pull the checksums for verification from the block device and not 
the cache.

Re: lazytime mount option—no support in Btrfs

[Adam Borowski]

On Mon, Aug 20, 2018 at 08:16:16AM -0400, Austin S. Hemmelgarn wrote:
> Also, slightly OT, but atimes are not where the real benefit is here for
> most people.  No sane software other than mutt uses atimes (and mutt's use
> of them is not sane, but that's a different argument)

Right.  There are two competing forks of mutt: neomutt and vanilla:
https://github.com/neomutt/neomutt/commit/816095bfdb72caafd8845e8fb28cbc8c6afc114f
https://gitlab.com/dops/mutt/commit/489a1c394c29e4b12b705b62da413f322406326f

So this has already been taken care of.

> so pretty much everyone who wants to avoid the overhead from them can just
> use the `noatime` mount option.

atime updates (including relatime) are bad not only for performance, they
also explode disk size used by snapshots (btrfs, LVM, ...) -- to the tune of
~5% per snapshot for some non-crafted loads.  And, are bad for media with
low write endurance (SD cards, as used by most SoCs).

Thus, atime needs to die.

> The real benefit for most people is with mtimes, for which there is no
> other way to limit the impact they have on performance.

With btrfs, any write already triggers metadata update (except nocow), thus
there's little benefit of lazytime for mtimes.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄⠀⠀⠀⠀ • use glitches to walk on water [Mt14:25-26]

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-21 08:06, Adam Borowski wrote:
> On Mon, Aug 20, 2018 at 08:16:16AM -0400, Austin S. Hemmelgarn wrote:
>> Also, slightly OT, but atimes are not where the real benefit is here for
>> most people.  No sane software other than mutt uses atimes (and mutt's use
>> of them is not sane, but that's a different argument)
> 
> Right.  There are two competing forks of mutt: neomutt and vanilla:
> https://github.com/neomutt/neomutt/commit/816095bfdb72caafd8845e8fb28cbc8c6afc114f
> https://gitlab.com/dops/mutt/commit/489a1c394c29e4b12b705b62da413f322406326f
> 
> So this has already been taken care of.
> 
>> so pretty much everyone who wants to avoid the overhead from them can just
>> use the `noatime` mount option.
> 
> atime updates (including relatime) are bad not only for performance, they
> also explode disk size used by snapshots (btrfs, LVM, ...) -- to the tune of
> ~5% per snapshot for some non-crafted loads.  And, are bad for media with
> low write endurance (SD cards, as used by most SoCs).
> 
> Thus, atime needs to die.
> 
>> The real benefit for most people is with mtimes, for which there is no
>> other way to limit the impact they have on performance.
> 
> With btrfs, any write already triggers metadata update (except nocow), thus
> there's little benefit of lazytime for mtimes.
But does that actually propagate all the way up to the point of updating 
the inode itself?  If so, then yes, there is not really any point.  if 
not though, then there is still a benefit.

Re: lazytime mount option—no support in Btrfs

[Janos Toth F.]

> >> so pretty much everyone who wants to avoid the overhead from them can just
> >> use the `noatime` mount option.

It would be great if someone finally fixed this old bug then:
https://bugzilla.kernel.org/show_bug.cgi?id=61601
Until then, it seems practically impossible to use both noatime (this
can't be added as rootflag in the command line and won't apply if the
kernel already mounted the root as RW) and space-cache-v2 (has to be
added as a rootflag along with RW to take effect) for the root
filesystem (at least without an init*fs, which I never use, so can't
tell).

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-21 09:32, Janos Toth F. wrote:
>>>> so pretty much everyone who wants to avoid the overhead from them can just
>>>> use the `noatime` mount option.
> 
> It would be great if someone finally fixed this old bug then:
> https://bugzilla.kernel.org/show_bug.cgi?id=61601
> Until then, it seems practically impossible to use both noatime (this
> can't be added as rootflag in the command line and won't apply if the
> kernel already mounted the root as RW) and space-cache-v2 (has to be
> added as a rootflag along with RW to take effect) for the root
> filesystem (at least without an init*fs, which I never use, so can't
> tell).
> 
Last I knew, it was fixed.  Of course, it's been quite a while since I 
last tried this, as I run locally patched kernels that have `noatime` as 
the default instead of `relatime`.

Also, once you've got the space cache set up by mounting once writable 
with the appropriate flag and then waiting for it to initialize, you 
should not ever need to specify the `space_cache` option again.

Re: lazytime mount option—no support in Btrfs

[David Sterba]

On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
> On 2018-08-21 09:32, Janos Toth F. wrote:
> >>>> so pretty much everyone who wants to avoid the overhead from them can just
> >>>> use the `noatime` mount option.
> > 
> > It would be great if someone finally fixed this old bug then:
> > https://bugzilla.kernel.org/show_bug.cgi?id=61601
> > Until then, it seems practically impossible to use both noatime (this
> > can't be added as rootflag in the command line and won't apply if the
> > kernel already mounted the root as RW) and space-cache-v2 (has to be
> > added as a rootflag along with RW to take effect) for the root
> > filesystem (at least without an init*fs, which I never use, so can't
> > tell).
> > 
> Last I knew, it was fixed.  Of course, it's been quite a while since I 
> last tried this, as I run locally patched kernels that have `noatime` as 
> the default instead of `relatime`.

I'm using VMs without initrd, tested the rootflags=noatime and it still
fails, the same way as in the bugreport.

As the 'noatime' mount option is part of the mount(2) API (passed as a
bit via mountflags), the remaining option in the filesystem is to
whitelist the generic options and ignore them. But this brings some
layering violation question.

On the other hand, this would be come confusing as the user expectation
is to see the effects of 'noatime'.

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-21 12:05, David Sterba wrote:
> On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
>> On 2018-08-21 09:32, Janos Toth F. wrote:
>>>>>> so pretty much everyone who wants to avoid the overhead from them can just
>>>>>> use the `noatime` mount option.
>>>
>>> It would be great if someone finally fixed this old bug then:
>>> https://bugzilla.kernel.org/show_bug.cgi?id=61601
>>> Until then, it seems practically impossible to use both noatime (this
>>> can't be added as rootflag in the command line and won't apply if the
>>> kernel already mounted the root as RW) and space-cache-v2 (has to be
>>> added as a rootflag along with RW to take effect) for the root
>>> filesystem (at least without an init*fs, which I never use, so can't
>>> tell).
>>>
>> Last I knew, it was fixed.  Of course, it's been quite a while since I
>> last tried this, as I run locally patched kernels that have `noatime` as
>> the default instead of `relatime`.
> 
> I'm using VMs without initrd, tested the rootflags=noatime and it still
> fails, the same way as in the bugreport.
> 
> As the 'noatime' mount option is part of the mount(2) API (passed as a
> bit via mountflags), the remaining option in the filesystem is to
> whitelist the generic options and ignore them. But this brings some
> layering violation question.
> 
> On the other hand, this would be come confusing as the user expectation
> is to see the effects of 'noatime'.
> 
Ideally there would be a way to get this to actually work properly.  I 
think ext4 at least doesn't panic, though I'm not sure if it actually 
works correctly.

Otherwise, the only option for people who want it set is to patch the 
kernel to get noatime as the default (instead of relatime).  I would 
look at pushing such a patch upstream myself actually, if it weren't for 
the fact that I'm fairly certain that it would be immediately NACK'ed by 
at least Linus, and probably a couple of other people too.

Re: lazytime mount option—no support in Btrfs

[Duncan]

Austin S. Hemmelgarn posted on Tue, 21 Aug 2018 13:01:00 -0400 as
excerpted:

> Otherwise, the only option for people who want it set is to patch the
> kernel to get noatime as the default (instead of relatime).  I would
> look at pushing such a patch upstream myself actually, if it weren't for
> the fact that I'm fairly certain that it would be immediately NACK'ed by
> at least Linus, and probably a couple of other people too.

What about making default-noatime a kconfig option, presumably set to 
default-relatime by default?  That seems to be the way many legacy-
incompatible changes work.  Then for most it's up to the distro, which in 
fact it is already, only if the distro set noatime-default they'd at 
least be using an upstream option instead of patching it themselves, 
making it upstream code that could be accounted for instead of downstream 
code that... who knows?

Meanwhile, I'd be interested in seeing your local patch.  I'm local-
patching noatime-default here too, but not being a dev, I'm not entirely 
sure I'm doing it "correctly", tho AFAICT it does seem to work.  FWIW, 
here's what I'm doing (posting inline so may be white-space damaged, and 
IIRC I just recently manually updated the line numbers so they don't 
reflect the code at the 2014 date any more, but as I'm not sure of the 
"correctness" it's not intended to be applied in any case):

--- fs/namespace.c.orig	2014-04-18 23:54:42.167666098 -0700
+++ fs/namespace.c	2014-04-19 00:19:08.622741946 -0700
@@ -2823,8 +2823,9 @@ long do_mount(const char *dev_name, cons
 		goto dput_out;
 
 	/* Default to relatime unless overriden */
-	if (!(flags & MS_NOATIME))
-		mnt_flags |= MNT_RELATIME;
+	/* JED: Make that noatime */
+	if (!(flags & MS_RELATIME))
+		mnt_flags |= MNT_NOATIME;
 
 	/* Separate the per-mountpoint flags */
 	if (flags & MS_NOSUID)
@@ -2837,6 +2837,8 @@ long do_mount(const char *dev_name, cons
 		mnt_flags |= MNT_NOATIME;
 	if (flags & MS_NODIRATIME)
 		mnt_flags |= MNT_NODIRATIME;
+	if (flags & MS_RELATIME)
+		mnt_flags |= MNT_RELATIME;
 	if (flags & MS_STRICTATIME)
 	mnt_flags &= ~(MNT_RELATIME | MNT_NOATIME);
 	if (flags & MS_RDONLY)

Sane, or am I "doing it wrong!"(TM), or perhaps doing it correctly, but 
missing a chunk that should be applied elsewhere?


Meanwhile, since broken rootflags requiring an initr* came up let me take 
the opportunity to ask once again, does btrfs-raid1 root still require an 
initr*?  It'd be /so/ nice to be able to supply the appropriate 
rootflags=device=...,device=... and actually have it work so I didn't 
need the initr* any longer!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-21 23:57, Duncan wrote:
> Austin S. Hemmelgarn posted on Tue, 21 Aug 2018 13:01:00 -0400 as
> excerpted:
> 
>> Otherwise, the only option for people who want it set is to patch the
>> kernel to get noatime as the default (instead of relatime).  I would
>> look at pushing such a patch upstream myself actually, if it weren't for
>> the fact that I'm fairly certain that it would be immediately NACK'ed by
>> at least Linus, and probably a couple of other people too.
> 
> What about making default-noatime a kconfig option, presumably set to
> default-relatime by default?  That seems to be the way many legacy-
> incompatible changes work.  Then for most it's up to the distro, which in
> fact it is already, only if the distro set noatime-default they'd at
> least be using an upstream option instead of patching it themselves,
> making it upstream code that could be accounted for instead of downstream
> code that... who knows?
That's probably a lot more likely to make it upstream, but it's a bit 
beyond my skills when it comes to stuff like this.
> 
> Meanwhile, I'd be interested in seeing your local patch.  I'm local-
> patching noatime-default here too, but not being a dev, I'm not entirely
> sure I'm doing it "correctly", tho AFAICT it does seem to work.  FWIW,
> here's what I'm doing (posting inline so may be white-space damaged, and
> IIRC I just recently manually updated the line numbers so they don't
> reflect the code at the 2014 date any more, but as I'm not sure of the
> "correctness" it's not intended to be applied in any case):
> 
> --- fs/namespace.c.orig	2014-04-18 23:54:42.167666098 -0700
> +++ fs/namespace.c	2014-04-19 00:19:08.622741946 -0700
> @@ -2823,8 +2823,9 @@ long do_mount(const char *dev_name, cons
>   		goto dput_out;
>   
>   	/* Default to relatime unless overriden */
> -	if (!(flags & MS_NOATIME))
> -		mnt_flags |= MNT_RELATIME;
> +	/* JED: Make that noatime */
> +	if (!(flags & MS_RELATIME))
> +		mnt_flags |= MNT_NOATIME;
>   
>   	/* Separate the per-mountpoint flags */
>   	if (flags & MS_NOSUID)
> @@ -2837,6 +2837,8 @@ long do_mount(const char *dev_name, cons
>   		mnt_flags |= MNT_NOATIME;
>   	if (flags & MS_NODIRATIME)
>   		mnt_flags |= MNT_NODIRATIME;
> +	if (flags & MS_RELATIME)
> +		mnt_flags |= MNT_RELATIME;
>   	if (flags & MS_STRICTATIME)
>   	mnt_flags &= ~(MNT_RELATIME | MNT_NOATIME);
>   	if (flags & MS_RDONLY)
> 
> Sane, or am I "doing it wrong!"(TM), or perhaps doing it correctly, but
> missing a chunk that should be applied elsewhere?
Mine only has the first part, not the second, which seems to cover 
making sure it's noatime by default.  I never use relatime though, so 
that may be broken with my patch because of me not having the second part.
> 
> 
> Meanwhile, since broken rootflags requiring an initr* came up let me take
> the opportunity to ask once again, does btrfs-raid1 root still require an
> initr*?  It'd be /so/ nice to be able to supply the appropriate
> rootflags=device=...,device=... and actually have it work so I didn't
> need the initr* any longer!
Last I knew, specifying appropriate `device=` options in rootflags works 
correctly without an initrd.

Re: lazytime mount option—no support in Btrfs

[David Sterba]

On Tue, Aug 21, 2018 at 01:01:00PM -0400, Austin S. Hemmelgarn wrote:
> On 2018-08-21 12:05, David Sterba wrote:
> > On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
> >> On 2018-08-21 09:32, Janos Toth F. wrote:
> >>>>>> so pretty much everyone who wants to avoid the overhead from them can just
> >>>>>> use the `noatime` mount option.
> >>>
> >>> It would be great if someone finally fixed this old bug then:
> >>> https://bugzilla.kernel.org/show_bug.cgi?id=61601
> >>> Until then, it seems practically impossible to use both noatime (this
> >>> can't be added as rootflag in the command line and won't apply if the
> >>> kernel already mounted the root as RW) and space-cache-v2 (has to be
> >>> added as a rootflag along with RW to take effect) for the root
> >>> filesystem (at least without an init*fs, which I never use, so can't
> >>> tell).
> >>>
> >> Last I knew, it was fixed.  Of course, it's been quite a while since I
> >> last tried this, as I run locally patched kernels that have `noatime` as
> >> the default instead of `relatime`.
> > 
> > I'm using VMs without initrd, tested the rootflags=noatime and it still
> > fails, the same way as in the bugreport.
> > 
> > As the 'noatime' mount option is part of the mount(2) API (passed as a
> > bit via mountflags), the remaining option in the filesystem is to
> > whitelist the generic options and ignore them. But this brings some
> > layering violation question.
> > 
> > On the other hand, this would be come confusing as the user expectation
> > is to see the effects of 'noatime'.
> > 
> Ideally there would be a way to get this to actually work properly.  I 
> think ext4 at least doesn't panic, though I'm not sure if it actually 
> works correctly.

No, ext4 also refuses to mount, the panic happens in VFS that tries
either the rootfstype= or all available filesystems.

[    3.763602] EXT4-fs (sda): Unrecognized mount option "noatime" or missing value 

[    3.761315] BTRFS info (device sda): unrecognized mount option 'noatime'

> Otherwise, the only option for people who want it set is to patch the 
> kernel to get noatime as the default (instead of relatime).  I would 
> look at pushing such a patch upstream myself actually, if it weren't for 
> the fact that I'm fairly certain that it would be immediately NACK'ed by 
> at least Linus, and probably a couple of other people too.

An acceptable solution could be to parse the rootflags and translate
them to the MNT_* values, ie. what the commandline tool mount does
before it calls the mount syscall.

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-22 09:48, David Sterba wrote:
> On Tue, Aug 21, 2018 at 01:01:00PM -0400, Austin S. Hemmelgarn wrote:
>> On 2018-08-21 12:05, David Sterba wrote:
>>> On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
>>>> On 2018-08-21 09:32, Janos Toth F. wrote:
>>>>>>>> so pretty much everyone who wants to avoid the overhead from them can just
>>>>>>>> use the `noatime` mount option.
>>>>>
>>>>> It would be great if someone finally fixed this old bug then:
>>>>> https://bugzilla.kernel.org/show_bug.cgi?id=61601
>>>>> Until then, it seems practically impossible to use both noatime (this
>>>>> can't be added as rootflag in the command line and won't apply if the
>>>>> kernel already mounted the root as RW) and space-cache-v2 (has to be
>>>>> added as a rootflag along with RW to take effect) for the root
>>>>> filesystem (at least without an init*fs, which I never use, so can't
>>>>> tell).
>>>>>
>>>> Last I knew, it was fixed.  Of course, it's been quite a while since I
>>>> last tried this, as I run locally patched kernels that have `noatime` as
>>>> the default instead of `relatime`.
>>>
>>> I'm using VMs without initrd, tested the rootflags=noatime and it still
>>> fails, the same way as in the bugreport.
>>>
>>> As the 'noatime' mount option is part of the mount(2) API (passed as a
>>> bit via mountflags), the remaining option in the filesystem is to
>>> whitelist the generic options and ignore them. But this brings some
>>> layering violation question.
>>>
>>> On the other hand, this would be come confusing as the user expectation
>>> is to see the effects of 'noatime'.
>>>
>> Ideally there would be a way to get this to actually work properly.  I
>> think ext4 at least doesn't panic, though I'm not sure if it actually
>> works correctly.
> 
> No, ext4 also refuses to mount, the panic happens in VFS that tries
> either the rootfstype= or all available filesystems.
> 
> [    3.763602] EXT4-fs (sda): Unrecognized mount option "noatime" or missing value
> 
> [    3.761315] BTRFS info (device sda): unrecognized mount option 'noatime'
> 
>> Otherwise, the only option for people who want it set is to patch the
>> kernel to get noatime as the default (instead of relatime).  I would
>> look at pushing such a patch upstream myself actually, if it weren't for
>> the fact that I'm fairly certain that it would be immediately NACK'ed by
>> at least Linus, and probably a couple of other people too.
> 
> An acceptable solution could be to parse the rootflags and translate
> them to the MNT_* values, ie. what the commandline tool mount does
> before it calls the mount syscall.
> 
That would be helpful, but at that point you might as well update the 
CLI mount tool to just pass all the named options to the kernel and have 
it do the parsing (I mean, keep the old interface too obviously, but 
provide a new one and use that preferentially).

I also like Duncan's suggestion to expose the default value for the 
atime options as a kconfig option (Chris Murphy emailed me directly 
about essentially the same thing).

Re: lazytime mount option—no support in Btrfs

[David Sterba]

On Wed, Aug 22, 2018 at 09:56:59AM -0400, Austin S. Hemmelgarn wrote:
> On 2018-08-22 09:48, David Sterba wrote:
> > On Tue, Aug 21, 2018 at 01:01:00PM -0400, Austin S. Hemmelgarn wrote:
> >> On 2018-08-21 12:05, David Sterba wrote:
> >>> On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
> >>>> On 2018-08-21 09:32, Janos Toth F. wrote:
> >>>>>>>> so pretty much everyone who wants to avoid the overhead from them can just
> >>>>>>>> use the `noatime` mount option.
> >>>>>
> >>>>> It would be great if someone finally fixed this old bug then:
> >>>>> https://bugzilla.kernel.org/show_bug.cgi?id=61601
> >>>>> Until then, it seems practically impossible to use both noatime (this
> >>>>> can't be added as rootflag in the command line and won't apply if the
> >>>>> kernel already mounted the root as RW) and space-cache-v2 (has to be
> >>>>> added as a rootflag along with RW to take effect) for the root
> >>>>> filesystem (at least without an init*fs, which I never use, so can't
> >>>>> tell).
> >>>>>
> >>>> Last I knew, it was fixed.  Of course, it's been quite a while since I
> >>>> last tried this, as I run locally patched kernels that have `noatime` as
> >>>> the default instead of `relatime`.
> >>>
> >>> I'm using VMs without initrd, tested the rootflags=noatime and it still
> >>> fails, the same way as in the bugreport.
> >>>
> >>> As the 'noatime' mount option is part of the mount(2) API (passed as a
> >>> bit via mountflags), the remaining option in the filesystem is to
> >>> whitelist the generic options and ignore them. But this brings some
> >>> layering violation question.
> >>>
> >>> On the other hand, this would be come confusing as the user expectation
> >>> is to see the effects of 'noatime'.
> >>>
> >> Ideally there would be a way to get this to actually work properly.  I
> >> think ext4 at least doesn't panic, though I'm not sure if it actually
> >> works correctly.
> > 
> > No, ext4 also refuses to mount, the panic happens in VFS that tries
> > either the rootfstype= or all available filesystems.
> > 
> > [    3.763602] EXT4-fs (sda): Unrecognized mount option "noatime" or missing value
> > 
> > [    3.761315] BTRFS info (device sda): unrecognized mount option 'noatime'
> > 
> >> Otherwise, the only option for people who want it set is to patch the
> >> kernel to get noatime as the default (instead of relatime).  I would
> >> look at pushing such a patch upstream myself actually, if it weren't for
> >> the fact that I'm fairly certain that it would be immediately NACK'ed by
> >> at least Linus, and probably a couple of other people too.
> > 
> > An acceptable solution could be to parse the rootflags and translate
> > them to the MNT_* values, ie. what the commandline tool mount does
> > before it calls the mount syscall.
> > 
> That would be helpful, but at that point you might as well update the 
> CLI mount tool to just pass all the named options to the kernel and have 
> it do the parsing (I mean, keep the old interface too obviously, but 
> provide a new one and use that preferentially).

The initial mount is not done by the mount tool but internally by
kernel init sequence (files in init/):

mount_block_root
  do_mount_root
    ksys_mount

The mount options (as a string) is passed unchanged via variable
root_mount_data (== rootflags). So before this step, the options would
have to be filtered and all known generic options turned into bit flags.

Re: lazytime mount option—no support in Btrfs

[Austin S. Hemmelgarn]

On 2018-08-22 11:01, David Sterba wrote:
> On Wed, Aug 22, 2018 at 09:56:59AM -0400, Austin S. Hemmelgarn wrote:
>> On 2018-08-22 09:48, David Sterba wrote:
>>> On Tue, Aug 21, 2018 at 01:01:00PM -0400, Austin S. Hemmelgarn wrote:
>>>> On 2018-08-21 12:05, David Sterba wrote:
>>>>> On Tue, Aug 21, 2018 at 10:10:04AM -0400, Austin S. Hemmelgarn wrote:
>>>>>> On 2018-08-21 09:32, Janos Toth F. wrote:
>>>>>>>>>> so pretty much everyone who wants to avoid the overhead from them can just
>>>>>>>>>> use the `noatime` mount option.
>>>>>>>
>>>>>>> It would be great if someone finally fixed this old bug then:
>>>>>>> https://bugzilla.kernel.org/show_bug.cgi?id=61601
>>>>>>> Until then, it seems practically impossible to use both noatime (this
>>>>>>> can't be added as rootflag in the command line and won't apply if the
>>>>>>> kernel already mounted the root as RW) and space-cache-v2 (has to be
>>>>>>> added as a rootflag along with RW to take effect) for the root
>>>>>>> filesystem (at least without an init*fs, which I never use, so can't
>>>>>>> tell).
>>>>>>>
>>>>>> Last I knew, it was fixed.  Of course, it's been quite a while since I
>>>>>> last tried this, as I run locally patched kernels that have `noatime` as
>>>>>> the default instead of `relatime`.
>>>>>
>>>>> I'm using VMs without initrd, tested the rootflags=noatime and it still
>>>>> fails, the same way as in the bugreport.
>>>>>
>>>>> As the 'noatime' mount option is part of the mount(2) API (passed as a
>>>>> bit via mountflags), the remaining option in the filesystem is to
>>>>> whitelist the generic options and ignore them. But this brings some
>>>>> layering violation question.
>>>>>
>>>>> On the other hand, this would be come confusing as the user expectation
>>>>> is to see the effects of 'noatime'.
>>>>>
>>>> Ideally there would be a way to get this to actually work properly.  I
>>>> think ext4 at least doesn't panic, though I'm not sure if it actually
>>>> works correctly.
>>>
>>> No, ext4 also refuses to mount, the panic happens in VFS that tries
>>> either the rootfstype= or all available filesystems.
>>>
>>> [    3.763602] EXT4-fs (sda): Unrecognized mount option "noatime" or missing value
>>>
>>> [    3.761315] BTRFS info (device sda): unrecognized mount option 'noatime'
>>>
>>>> Otherwise, the only option for people who want it set is to patch the
>>>> kernel to get noatime as the default (instead of relatime).  I would
>>>> look at pushing such a patch upstream myself actually, if it weren't for
>>>> the fact that I'm fairly certain that it would be immediately NACK'ed by
>>>> at least Linus, and probably a couple of other people too.
>>>
>>> An acceptable solution could be to parse the rootflags and translate
>>> them to the MNT_* values, ie. what the commandline tool mount does
>>> before it calls the mount syscall.
>>>
>> That would be helpful, but at that point you might as well update the
>> CLI mount tool to just pass all the named options to the kernel and have
>> it do the parsing (I mean, keep the old interface too obviously, but
>> provide a new one and use that preferentially).
> 
> The initial mount is not done by the mount tool but internally by
> kernel init sequence (files in init/):
> 
> mount_block_root
>    do_mount_root
>      ksys_mount
> 
> The mount options (as a string) is passed unchanged via variable
> root_mount_data (== rootflags). So before this step, the options would
> have to be filtered and all known generic options turned into bit flags.
> 
What I'm saying is that if there's going to be parsing for it in the 
kernel anyway, why not expose that interface to userspace too so that 
the regular `mount` tool can take advantage of it as well.

Re: lazytime mount option—no support in Btrfs

[Duncan]

Austin S. Hemmelgarn posted on Wed, 22 Aug 2018 07:30:09 -0400 as
excerpted:

>> Meanwhile, since broken rootflags requiring an initr* came up let me
>> take the opportunity to ask once again, does btrfs-raid1 root still
>> require an initr*?  It'd be /so/ nice to be able to supply the
>> appropriate rootflags=device=...,device=... and actually have it work
>> so I didn't need the initr* any longer!

> Last I knew, specifying appropriate `device=` options in rootflags works
> correctly without an initrd.

Just to confirm, that's with multi-device btrfs rootfs?  Because it used 
to work when the btrfs was single-device, but not multi-device.

(For multi-device, or at least raid1, one had to add degraded, also, or 
it would refuse to mount despite all the appropriate device= entries in 
rootflags, thus of course risking all the problems running degraded raid1 
operationally can bring, tho I never figured out for sure whether btrfs 
was smart enough to eventually pick up the other devices, after the scan 
before bringing other btrfs online or not, but either way it was a risk I 
wasn't willing to take.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: lazytime mount option—no support in Btrfs

[Janos Toth F.]

On Tue, Aug 21, 2018 at 4:10 PM Austin S. Hemmelgarn
<ahferroin7@gmail.com> wrote:
> Also, once you've got the space cache set up by mounting once writable
> with the appropriate flag and then waiting for it to initialize, you
> should not ever need to specify the `space_cache` option again.

True.
I am not sure why I thought I have to keep cache-v2 in the rootflag
list. I guess I forgot it was meant to be removed after a reboot. Or
may be some old kernels misbehaved (always cleared the space-tree for
some reason and re-initialized a space-cache instead unless the flag
was there). But I removed it now and everything works as intended.
Thanks.