Hibernation into swap file

Hibernation into swap file

[Maksim Fomin]

Good day.

Since 5.0 btrfs supports swap files. Does it support hibernation into a swap file?

With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2 (unlikely to be relevant, but still) when I try to hibernate with systemctl or by directly manipulating '/sys/power/resume' and '/sys/power/resume_offset', the kernel logs:

PM: Cannot find swap device, try swapon -a
PM: Cannot get swap writer

After digging into this issue I suspect that currently btrfs does not support hibernation into swap file (or does it?). Furthermore, I suspect that kernel mechanism of accessing swap file via 'resume_offset' is unreliable in btrfs case because it is more likely (comparing to other fs) to move files across filesystem which invalidates swap file offset, so the whole idea is dubious. So,

1) does btrfs supports hibernation into swap file?
2) is there mechanism to lock swap file?

Best regards,
Maxim Fomin

Re: Hibernation into swap file

[Andrei Borzenkov]

05.05.2019 10:50, Maksim Fomin пишет:
> Good day.
> 
> Since 5.0 btrfs supports swap files. Does it support hibernation into
> a swap file?
> 
> With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2
> (unlikely to be relevant, but still) when I try to hibernate with
> systemctl or by directly manipulating '/sys/power/resume' and
> '/sys/power/resume_offset', the kernel logs:
> 
> PM: Cannot find swap device, try swapon -a PM: Cannot get swap
> writer
> 

How exactly do you compute resume_offset? What are exact commands you
ise to initiate hibernation? systemctl will likely not work anyway as
systemd is using FIEMAP which returns logical offset of file extents in
btrfs address space, not physical offset on containing device. You will
need to jump from extent vaddr to device offset manually.

> After digging into this issue I suspect that currently btrfs does not
> support hibernation into swap file (or does it?). Furthermore, I
> suspect that kernel mechanism of accessing swap file via
> 'resume_offset' is unreliable in btrfs case because it is more likely
> (comparing to other fs) to move files across filesystem which
> invalidates swap file offset, so the whole idea is dubious. So,
> 
> 1) does btrfs supports hibernation into swap file? 2) is there
> mechanism to lock swap file?
> 

btrfs performs some checks before allowing file as swap and it also
implicitly locks it during swap usage. Note that btrfs will disable
snapshots of subvolume with swapfile.

File also must be using single allocation profile. There is no way to
force allocation profile for individual file I am aware of in case
filesytem defaults to something else. Also for multi-device btrfs I am
not aware of any way to force allocation of file on specific physical
device, and this is required to use file as swap on btrfs. So you are
likely restricted to single device single data profile.

Finally you are not allowed to mount any filesystem read-write (or
replay any pending log/journal) before doing resume. Does btrfs support
"true" read-only mount, which is guaranteed to not change filesystem
state in any way?

Re: Hibernation into swap file

[Chris Murphy]

On Sun, May 5, 2019 at 3:09 AM Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>
> 05.05.2019 10:50, Maksim Fomin пишет:
> > Good day.
> >
> > Since 5.0 btrfs supports swap files. Does it support hibernation into
> > a swap file?
> >
> > With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2
> > (unlikely to be relevant, but still) when I try to hibernate with
> > systemctl or by directly manipulating '/sys/power/resume' and
> > '/sys/power/resume_offset', the kernel logs:
> >
> > PM: Cannot find swap device, try swapon -a PM: Cannot get swap
> > writer
> >
>
> How exactly do you compute resume_offset? What are exact commands you
> ise to initiate hibernation? systemctl will likely not work anyway as
> systemd is using FIEMAP which returns logical offset of file extents in
> btrfs address space, not physical offset on containing device. You will
> need to jump from extent vaddr to device offset manually.


https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/power/swsusp-and-swap-files.txt?h=v5.0.13

This says the  resume_offset= is an "offset, in <PAGE_SIZE> units,
from the beginning of the partition which holds the swap file"

Use filefrag (uses FIEMAP) to get the virtual address, multiply by
4KIB to get bytes, and plug that into

btrfs-map-logical -l vaddrbyte <dev>

The physical number returned is also in bytes. Normally to get LBA
you'd divide by 512 (for anything other than a 4Kn drive), but if I
understand the kernel document correctly, it needs to be in x86_64
page size, so divide by 4096 instead.



> > After digging into this issue I suspect that currently btrfs does not
> > support hibernation into swap file (or does it?). Furthermore, I
> > suspect that kernel mechanism of accessing swap file via
> > 'resume_offset' is unreliable in btrfs case because it is more likely
> > (comparing to other fs) to move files across filesystem which
> > invalidates swap file offset, so the whole idea is dubious. So,
> >
> > 1) does btrfs supports hibernation into swap file? 2) is there
> > mechanism to lock swap file?
> >
>
> btrfs performs some checks before allowing file as swap and it also
> implicitly locks it during swap usage. Note that btrfs will disable
> snapshots of subvolume with swapfile.

Yes I think it's probably a best practice to have a "swaps" subvolume
created at the top level of the file system, to make sure contained
swapfiles are separate and immune from snapshotting the top level
subvolume itself. It's also reasonable to set chattr +C on that
subvolume, and files created there will inherit it.


>
> File also must be using single allocation profile. There is no way to
> force allocation profile for individual file I am aware of in case
> filesytem defaults to something else. Also for multi-device btrfs I am
> not aware of any way to force allocation of file on specific physical
> device, and this is required to use file as swap on btrfs. So you are
> likely restricted to single device single data profile.

Yes the man page specifically says it's supported only for single
device volumes.

> Finally you are not allowed to mount any filesystem read-write (or
> replay any pending log/journal) before doing resume. Does btrfs support
> "true" read-only mount, which is guaranteed to not change filesystem
> state in any way?

Resume is done by the kernel just after initramfs is unpacked, well
before the file system is ro mounted. I think it's probably a good
idea to make sure bootloader kernel command line contains the 'ro'
parameter. I don't know if Btrfs ro is guaranteed to change on-disk
metadata, I know logreplay affects the in-memory state of Btrfs but I
don't think it changes on-disk metadata.

But yeah, worth testing and making sure the backups are kept current.


-- 
Chris Murphy

Re: Hibernation into swap file

[David Sterba]

Hi,

On Sun, May 05, 2019 at 07:50:09AM +0000, Maksim Fomin wrote:
> Good day.
> 
> Since 5.0 btrfs supports swap files. Does it support hibernation into a swap file?
> 
> With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2 (unlikely to be relevant, but still) when I try to hibernate with systemctl or by directly manipulating '/sys/power/resume' and '/sys/power/resume_offset', the kernel logs:
> 
> PM: Cannot find swap device, try swapon -a
> PM: Cannot get swap writer
> 
> After digging into this issue I suspect that currently btrfs does not support hibernation into swap file (or does it?). Furthermore, I suspect that kernel mechanism of accessing swap file via 'resume_offset' is unreliable in btrfs case because it is more likely (comparing to other fs) to move files across filesystem which invalidates swap file offset, so the whole idea is dubious. So,
> 
> 1) does btrfs supports hibernation into swap file?
> 2) is there mechanism to lock swap file?

for the reference https://bugzilla.kernel.org/show_bug.cgi?id=202803
and https://github.com/systemd/systemd/issues/11939

Re: Hibernation into swap file

[Maksim Fomin]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, 6 May 2019 г., 8:40, Chris Murphy <lists@colorremedies.com> wrote:

> On Sun, May 5, 2019 at 3:09 AM Andrei Borzenkov arvidjaar@gmail.com wrote:
>
> > 05.05.2019 10:50, Maksim Fomin пишет:
> >
> > > Good day.
> > > Since 5.0 btrfs supports swap files. Does it support hibernation into
> > > a swap file?
> > > With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2
> > > (unlikely to be relevant, but still) when I try to hibernate with
> > > systemctl or by directly manipulating '/sys/power/resume' and
> > > '/sys/power/resume_offset', the kernel logs:
> > > PM: Cannot find swap device, try swapon -a PM: Cannot get swap
> > > writer
> >
> > How exactly do you compute resume_offset? What are exact commands you
> > ise to initiate hibernation? systemctl will likely not work anyway as
> > systemd is using FIEMAP which returns logical offset of file extents in
> > btrfs address space, not physical offset on containing device. You will
> > need to jump from extent vaddr to device offset manually.
>
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/power/swsusp-and-swap-files.txt?h=v5.0.13
>
> This says the resume_offset= is an "offset, in <PAGE_SIZE> units,
> from the beginning of the partition which holds the swap file"
>
> Use filefrag (uses FIEMAP) to get the virtual address, multiply by
> 4KIB to get bytes, and plug that into
>
> btrfs-map-logical -l vaddrbyte <dev>
>
> The physical number returned is also in bytes. Normally to get LBA
> you'd divide by 512 (for anything other than a 4Kn drive), but if I
> understand the kernel document correctly, it needs to be in x86_64
> page size, so divide by 4096 instead.
>
>
>
> Chris Murphy

Thanks everybody for clarification! After several attempts to create 1-extent file it appears that btrfs can do this only for files around 500-600 MiB which is low for my practical needs. If swap file is increased to 1-1.5 GiB, then there is non-contigous extents problem.

In any case, since swap file can be (with high probability) moved across filesystem, then 1) offset configuration cannot be stored (should be reconfigured for each hibernation) and 2) there is risk that kernel writes directly to disk at wrong place and will corrupt the filesystem. I don't like this feature.

Best regards,
Maksim Fomin

Re: Hibernation into swap file

[Andrei Borzenkov]

06.05.2019 21:25, Maksim Fomin пишет:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, 6 May 2019 г., 8:40, Chris Murphy <lists@colorremedies.com> wrote:
> 
>> On Sun, May 5, 2019 at 3:09 AM Andrei Borzenkov arvidjaar@gmail.com wrote:
>>
>>> 05.05.2019 10:50, Maksim Fomin пишет:
>>>
>>>> Good day.
>>>> Since 5.0 btrfs supports swap files. Does it support hibernation into
>>>> a swap file?
>>>> With kernel version 5.0.10 (archlinux) and btrfs-progs 4.20.2
>>>> (unlikely to be relevant, but still) when I try to hibernate with
>>>> systemctl or by directly manipulating '/sys/power/resume' and
>>>> '/sys/power/resume_offset', the kernel logs:
>>>> PM: Cannot find swap device, try swapon -a PM: Cannot get swap
>>>> writer
>>>
>>> How exactly do you compute resume_offset? What are exact commands you
>>> ise to initiate hibernation? systemctl will likely not work anyway as
>>> systemd is using FIEMAP which returns logical offset of file extents in
>>> btrfs address space, not physical offset on containing device. You will
>>> need to jump from extent vaddr to device offset manually.
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/power/swsusp-and-swap-files.txt?h=v5.0.13
>>
>> This says the resume_offset= is an "offset, in <PAGE_SIZE> units,
>> from the beginning of the partition which holds the swap file"
>>
>> Use filefrag (uses FIEMAP) to get the virtual address, multiply by
>> 4KIB to get bytes, and plug that into
>>
>> btrfs-map-logical -l vaddrbyte <dev>
>>
>> The physical number returned is also in bytes. Normally to get LBA
>> you'd divide by 512 (for anything other than a 4Kn drive), but if I
>> understand the kernel document correctly, it needs to be in x86_64
>> page size, so divide by 4096 instead.
>>
>>
>>
>> Chris Murphy
> 
> Thanks everybody for clarification! After several attempts to create 1-extent file it appears that btrfs can do this only for files around 500-600 MiB which is low for my practical needs. If swap file is increased to 1-1.5 GiB, then there is non-contigous extents problem.

Swap files (also used for hibernation) are not required to be contiguous.

> 
> In any case, since swap file can be (with high probability) moved across filesystem, then 1) offset configuration cannot be stored (should be reconfigured for each hibernation) and 2) there is risk that kernel writes directly to disk at wrong place and will corrupt the filesystem. I don't like this feature.

I think we need to allow at least some amount of trust to developers and
expect that this feature would not be released if it had such an obvious
problem.

Of course bug happens and it surely needs wider testing.

> 
> Best regards,
> Maksim Fomin
> 

Hibernation into swap file

[Maksim Fomin]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, May 9, 2019 6:13 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> I think we need to allow at least some amount of trust to developers and
> expect that this feature would not be released if it had such an obvious
> problem.
>
> Of course bug happens and it surely needs wider testing.
>
> > Best regards,
> > Maksim Fomin

Nowhere in this thread I critiqued developers for releasing btrfs with obvious problems.

What was released is 'swap file support' (from changelog). This does not necessarily mean 'swap file hibernation support'. That's why I asked question about this. After receiving answers here and reading linux/systemd bugzilla, I decided that I am not interested in hibernation in swap file because its costs outweigh my potential benefits. This has nothing to do with trust.

Best regards,
Maksim Fomin

Re: Hibernation into swap file

[Chris Murphy]

On Mon, May 6, 2019 at 5:31 AM David Sterba <dsterba@suse.cz> wrote:
> for the reference https://bugzilla.kernel.org/show_bug.cgi?id=202803
> and https://github.com/systemd/systemd/issues/11939

I've read these, but can't tell if it's still necessary to manually
use 'btrfs-map-physical' to find the correct offset, and use it on the
kernel command line manually? It does sound like contiguous extents is
not a requirement for hibernation to a swapfile on btrfs. Correct?

The idea I'm evaluating is a way to dynamically enable a swapfile only
at hibernation time. That way there's no swap thrashing during normal
use, yet it's still possible to support hibernation. It'd be necessary
to insert the swapon quickly after a request (or pre-trigger, maybe by
upowerd) for hibernation, so that the various systemd tests already
find a suitable swap device for the hibernation image.


--
Chris Murphy

Re: Hibernation into swap file

[Andrei Borzenkov]

23.01.2020 20:20, Chris Murphy пишет:
> On Mon, May 6, 2019 at 5:31 AM David Sterba <dsterba@suse.cz> wrote:
>> for the reference https://bugzilla.kernel.org/show_bug.cgi?id=202803
>> and https://github.com/systemd/systemd/issues/11939
> 
> I've read these, but can't tell if it's still necessary to manually
> use 'btrfs-map-physical' to find the correct offset, and use it on the
> kernel command line manually?

Yes, it is. systemd does not compute it automatically.

> It does sound like contiguous extents is
> not a requirement for hibernation to a swapfile on btrfs. Correct?
> 
> The idea I'm evaluating is a way to dynamically enable a swapfile only
> at hibernation time. That way there's no swap thrashing during normal
> use, yet it's still possible to support hibernation. It'd be necessary
> to insert the swapon quickly after a request (or pre-trigger, maybe by
> upowerd) for hibernation, so that the various systemd tests already
> find a suitable swap device for the hibernation image.
> 
> 
> --
> Chris Murphy
> 

Re: Hibernation into swap file

[Omar Sandoval]

On Thu, Jan 23, 2020 at 10:20:40AM -0700, Chris Murphy wrote:
> On Mon, May 6, 2019 at 5:31 AM David Sterba <dsterba@suse.cz> wrote:
> > for the reference https://bugzilla.kernel.org/show_bug.cgi?id=202803
> > and https://github.com/systemd/systemd/issues/11939
> 
> I've read these, but can't tell if it's still necessary to manually
> use 'btrfs-map-physical' to find the correct offset, and use it on the
> kernel command line manually?

Yes, it's still necessary, and it's unlikely that systemd will ever go
through the trouble of doing the btrfs-map-physical itself. What we
really want is the interface I described in [1], but I won't be able to
work on that any time soon. It'd be great if someone interested in the
feature would take a stab at it.

1: https://github.com/systemd/systemd/issues/11939#issuecomment-471684411