[f2fs-dev] TRIM issue

[f2fs-dev] TRIM issue

[Piergiorgio Sartor]

Hi all,

I've a small box, with Fedora 31 up-to-date.

The system has LUKS + LVM + f2fs.

Since kernel 5.3.16-300.fc31.x86_64 the fstrim.service,
triggered by the systemd service fstrim.timer or from
command line, does return an error:

Starting Discard unused blocks on filesystems from /etc/fstab...
fstrim: /home: FITRIM ioctl failed: Read-only file system
fstrim: /: FITRIM ioctl failed: Read-only file system
fstrim.service: Main process exited, code=exited, status=32/n/a
fstrim.service: Failed with result 'exit-code'.
Failed to start Discard unused blocks on filesystems from /etc/fstab.

Note the following:
1) Both /home and / are rw
2) LUKS and LVM allow discard to the underlying device
3) An other PC, with same layout, but ext4, does work as expected
4) kernel 5.4.7-200.fc31.x86_64 (latest for f31) has same result
5) fstrim -vA or -va does not return an error, exit code is 0,
but it always says 0 (zero) bytes are trimmed:

/home: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-home
/: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-root

6) mount shows that the f2fs mounts have "discard" flag

So, I suspect there is something with f2fs which somehow
confuses the trimming process.

I had a quick search, but I could not find any other
similar report, except one about systemd, but that
does not seem related.

Any idea? Suggestions?
Debugging possibilities?

Thanks a lot in advance,

bye,

-- 

piergiorgio


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] TRIM issue

[Jaegeuk Kim]

Hi,

On 01/08, Piergiorgio Sartor wrote:
> Hi all,
> 
> I've a small box, with Fedora 31 up-to-date.
> 
> The system has LUKS + LVM + f2fs.
> 
> Since kernel 5.3.16-300.fc31.x86_64 the fstrim.service,
> triggered by the systemd service fstrim.timer or from
> command line, does return an error:
> 
> Starting Discard unused blocks on filesystems from /etc/fstab...
> fstrim: /home: FITRIM ioctl failed: Read-only file system
> fstrim: /: FITRIM ioctl failed: Read-only file system
> fstrim.service: Main process exited, code=exited, status=32/n/a
> fstrim.service: Failed with result 'exit-code'.
> Failed to start Discard unused blocks on filesystems from /etc/fstab.

It smells like trying fstrim along with /etc/fstab before mounting the
partitions?

> 
> Note the following:
> 1) Both /home and / are rw
> 2) LUKS and LVM allow discard to the underlying device
> 3) An other PC, with same layout, but ext4, does work as expected
> 4) kernel 5.4.7-200.fc31.x86_64 (latest for f31) has same result
> 5) fstrim -vA or -va does not return an error, exit code is 0,
> but it always says 0 (zero) bytes are trimmed:
> 
> /home: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-home
> /: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-root

This is normal. Since f2fs issues discard commands asynchronously, fstrim
just collects discard candidates and returns right away. Later, it will
actually issue the candidates in background. Whereas, if it was mounted as
"-o nodiscard", fstrim will wait for whole discard commands being issued
and completed as you expect.

Thanks,

> 
> 6) mount shows that the f2fs mounts have "discard" flag
> 
> So, I suspect there is something with f2fs which somehow
> confuses the trimming process.
> 
> I had a quick search, but I could not find any other
> similar report, except one about systemd, but that
> does not seem related.
> 
> Any idea? Suggestions?
> Debugging possibilities?
> 
> Thanks a lot in advance,
> 
> bye,
> 
> -- 
> 
> piergiorgio
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] TRIM issue

[Piergiorgio Sartor]

Hi,

thanks for the reply, please see my comments
embedded below.

On Wed, Jan 08, 2020 at 06:29:30PM -0800, Jaegeuk Kim wrote:
> Hi,
> 
> On 01/08, Piergiorgio Sartor wrote:
> > Hi all,
> > 
> > I've a small box, with Fedora 31 up-to-date.
> > 
> > The system has LUKS + LVM + f2fs.
> > 
> > Since kernel 5.3.16-300.fc31.x86_64 the fstrim.service,
> > triggered by the systemd service fstrim.timer or from
> > command line, does return an error:
> > 
> > Starting Discard unused blocks on filesystems from /etc/fstab...
> > fstrim: /home: FITRIM ioctl failed: Read-only file system
> > fstrim: /: FITRIM ioctl failed: Read-only file system
> > fstrim.service: Main process exited, code=exited, status=32/n/a
> > fstrim.service: Failed with result 'exit-code'.
> > Failed to start Discard unused blocks on filesystems from /etc/fstab.
> 
> It smells like trying fstrim along with /etc/fstab before mounting the
> partitions?

The error is reported also when running, as root,
the command "systemctl start fstrim.service" from
command line, that is with partitions available.
So, I do not think it is before mounting. Also,
this is usually started weekly from fstrim.timer,
and this requires a running system.
It could be a strange conflict with systemd, but
with ext4 there is no problem, it happens with f2fs.

> > Note the following:
> > 1) Both /home and / are rw
> > 2) LUKS and LVM allow discard to the underlying device
> > 3) An other PC, with same layout, but ext4, does work as expected
> > 4) kernel 5.4.7-200.fc31.x86_64 (latest for f31) has same result
> > 5) fstrim -vA or -va does not return an error, exit code is 0,
> > but it always says 0 (zero) bytes are trimmed:
> > 
> > /home: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-home
> > /: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-root
> 
> This is normal. Since f2fs issues discard commands asynchronously, fstrim
> just collects discard candidates and returns right away. Later, it will
> actually issue the candidates in background. Whereas, if it was mounted as
> "-o nodiscard", fstrim will wait for whole discard commands being issued
> and completed as you expect.

OK, I remounted / with "nodiscard", created a 1GB
file, synched the fs, deleted the file and issued
"fstrim -vA".
I can confirm 1GB was trimmed, more precisely 1.1GB
according to "fstrim".

I repeated the test, but instead using "systemctl
start fstrim.service".
Same error as before.
But, a succesive "fstrim -vA" returned only 2.7MB
trimmed.
This could be that the "fstrim" call was executed
properly, but some error report was triggered.
Maybe the read-only file system is not correct,
but something happened.
I would like to stress again that ext4 does not
trigger this error.

So, there is something related with f2fs, at
least with the interaction with fstrim and
systemd.

I have one question.

Since f2fs is mounted with "discard" and, as I
understand it, f2fs will issue discard as soon
as a segment is freed, is it relevant to have
the weekly fstrim from systemd?

Thanks a lot in advance,

bye,

pg

> 
> Thanks,
> 
> > 
> > 6) mount shows that the f2fs mounts have "discard" flag
> > 
> > So, I suspect there is something with f2fs which somehow
> > confuses the trimming process.
> > 
> > I had a quick search, but I could not find any other
> > similar report, except one about systemd, but that
> > does not seem related.
> > 
> > Any idea? Suggestions?
> > Debugging possibilities?
> > 
> > Thanks a lot in advance,
> > 
> > bye,
> > 
> > -- 
> > 
> > piergiorgio
> > 
> > 
> > _______________________________________________
> > Linux-f2fs-devel mailing list
> > Linux-f2fs-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

-- 

piergiorgio


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] TRIM issue

[Jaegeuk Kim]

On 01/09, Piergiorgio Sartor wrote:
> Hi,
> 
> thanks for the reply, please see my comments
> embedded below.
> 
> On Wed, Jan 08, 2020 at 06:29:30PM -0800, Jaegeuk Kim wrote:
> > Hi,
> > 
> > On 01/08, Piergiorgio Sartor wrote:
> > > Hi all,
> > > 
> > > I've a small box, with Fedora 31 up-to-date.
> > > 
> > > The system has LUKS + LVM + f2fs.
> > > 
> > > Since kernel 5.3.16-300.fc31.x86_64 the fstrim.service,
> > > triggered by the systemd service fstrim.timer or from
> > > command line, does return an error:
> > > 
> > > Starting Discard unused blocks on filesystems from /etc/fstab...
> > > fstrim: /home: FITRIM ioctl failed: Read-only file system
> > > fstrim: /: FITRIM ioctl failed: Read-only file system
> > > fstrim.service: Main process exited, code=exited, status=32/n/a
> > > fstrim.service: Failed with result 'exit-code'.
> > > Failed to start Discard unused blocks on filesystems from /etc/fstab.
> > 
> > It smells like trying fstrim along with /etc/fstab before mounting the
> > partitions?
> 
> The error is reported also when running, as root,
> the command "systemctl start fstrim.service" from
> command line, that is with partitions available.
> So, I do not think it is before mounting. Also,
> this is usually started weekly from fstrim.timer,
> and this requires a running system.
> It could be a strange conflict with systemd, but
> with ext4 there is no problem, it happens with f2fs.

Okay.

> 
> > > Note the following:
> > > 1) Both /home and / are rw
> > > 2) LUKS and LVM allow discard to the underlying device
> > > 3) An other PC, with same layout, but ext4, does work as expected
> > > 4) kernel 5.4.7-200.fc31.x86_64 (latest for f31) has same result
> > > 5) fstrim -vA or -va does not return an error, exit code is 0,
> > > but it always says 0 (zero) bytes are trimmed:
> > > 
> > > /home: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-home
> > > /: 0 B (0 bytes) trimmed on /dev/mapper/fedora--server-root
> > 
> > This is normal. Since f2fs issues discard commands asynchronously, fstrim
> > just collects discard candidates and returns right away. Later, it will
> > actually issue the candidates in background. Whereas, if it was mounted as
> > "-o nodiscard", fstrim will wait for whole discard commands being issued
> > and completed as you expect.
> 
> OK, I remounted / with "nodiscard", created a 1GB
> file, synched the fs, deleted the file and issued
> "fstrim -vA".
> I can confirm 1GB was trimmed, more precisely 1.1GB
> according to "fstrim".
> 
> I repeated the test, but instead using "systemctl
> start fstrim.service".
> Same error as before.
> But, a succesive "fstrim -vA" returned only 2.7MB
> trimmed.
> This could be that the "fstrim" call was executed
> properly, but some error report was triggered.
> Maybe the read-only file system is not correct,
> but something happened.
> I would like to stress again that ext4 does not
> trigger this error.
> 
> So, there is something related with f2fs, at
> least with the interaction with fstrim and
> systemd.

I'm not sure, maybe systemd interpreted zero return as RO partition?

> 
> I have one question.
> 
> Since f2fs is mounted with "discard" and, as I
> understand it, f2fs will issue discard as soon
> as a segment is freed, is it relevant to have
> the weekly fstrim from systemd?

Not exactly segment. f2fs supports small sized discards as well in order to
get better performance of small crappy mmc cards.

The return value of fstrim in f2fs represents the actual size of discard
commands being issued during the syscall. So, fstrim w/ "-o discard" gives zero
all the time. OTOH, it reports non-zero value given "-o nodiscard", but again
won't give the full size all the time, since f2fs doesn't issue discard commands
onto previously discarded space.

So, in general, I prefer managing this like in Android [1] with "-o discard" and
1) run f2fs GC/discard w/ "-o discard" aggressively at night, 2) run fstrim
instantly, 3) sync.

1) can be simply done by:
- "echo 1 > /sys/fs/f2fs/dev/gc_urgent" 
- wait for 10 minutres
- "echo 0 > /sys/fs/f2fs/dev/gc_urgent"
- sync

[1] https://android.googlesource.com/platform/system/vold/+/91a68dfc98defe4de8630290b0dcff0311de6678/IdleMaint.cpp#71

> 
> Thanks a lot in advance,
> 
> bye,
> 
> pg
> 
> > 
> > Thanks,
> > 
> > > 
> > > 6) mount shows that the f2fs mounts have "discard" flag
> > > 
> > > So, I suspect there is something with f2fs which somehow
> > > confuses the trimming process.
> > > 
> > > I had a quick search, but I could not find any other
> > > similar report, except one about systemd, but that
> > > does not seem related.
> > > 
> > > Any idea? Suggestions?
> > > Debugging possibilities?
> > > 
> > > Thanks a lot in advance,
> > > 
> > > bye,
> > > 
> > > -- 
> > > 
> > > piergiorgio
> > > 
> > > 
> > > _______________________________________________
> > > Linux-f2fs-devel mailing list
> > > Linux-f2fs-devel@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 
> -- 
> 
> piergiorgio
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel