* [Help] Can't login to my systemd-homed user account due to fallocate failure
@ 2020-08-23 15:31 Andrii Zymohliad
2020-08-23 15:37 ` Hugo Mills
2020-08-24 2:50 ` Chris Murphy
0 siblings, 2 replies; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-23 15:31 UTC (permalink / raw)
To: linux-btrfs
Hello! I've lost the ability to log in to my systemd-homed user account, and after some investigation on Systemd mailing list I was directed here. I would be very grateful for any help!
My root partition is ~475GiB with BTRFS, my home partition is a ~400GiB LUKS-encrypted partition on a loopback file (also BTRFS) created by systemd-homed (residing at /home/azymohliad.home). Which leaves ~75GiB for the rest of the root FS.
Recently I've lost the ability to log in to that user account, because during authentication systemd does fallocate call for the image. CLI alternative for my case (suggested on systemd mailing list, I don't really know what is it) is:
fallocate -l 403G -n /home/azymohliad.home
Which fails:
fallocate: fallocate failed: No space left on device
My first idea was that I occupied all those ~75GiB on a root partition, but cleaning didn't help (I definitely released more space than I could occupy during the last working session).
Here are some details about my system:
uname -a
Linux az-wolf-pc 5.8.3-arch1-1 #1 SMP PREEMPT Fri, 21 Aug 2020 16:54:16 +0000 x86_64 GNU/Linux
btrfs --version
btrfs-progs v5.7
I can mount my home manually like this:
losetup -fP /home/azymohliad.home
cryptsetup open /dev/loop0p1
mount /dev/mapper/home /mnt
and then,
btrfs fi show
Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
Total devices 1 FS bytes used 299.71GiB
devid 1 size 476.44GiB used 476.44GiB path /dev/nvme0n1p2
Label: 'azymohliad' uuid: 4ffae38b-42c9-4e53-89a1-3d21cd862938
Total devices 1 FS bytes used 221.92GiB
devid 1 size 402.72GiB used 258.02GiB path /dev/mapper/home
btrfs fi df /
Data, single: total=475.43GiB, used=299.28GiB
System, single: total=4.00MiB, used=80.00KiB
Metadata, single: total=1.01GiB, used=437.05MiB
GlobalReserve, single: total=61.03MiB, used=0.00B
btrfs fi df /mnt
Data, single: total=256.01GiB, used=221.18GiB
System, single: total=4.00MiB, used=48.00KiB
Metadata, single: total=2.01GiB, used=749.92MiB
GlobalReserve, single: total=297.11MiB, used=0.00B
dmesg.log: https://gitlab.com/-/snippets/2007155
What's interesting to me from above, the partition size on /home/azymohliad.home is 402.72GiB, but the file system size is 256.01GiB, and the image file size is 256.64GiB (from btrfs fi du /home, although ls -lh reports 403GiB). I'm not really sure, but iirc the fs and image sizes were around 403GiB too earlier. Could it be that it somehow got automatically reduced?
Could I do anything to make that fallocate call (with -l 403G) working? It will allow me to authenticate to homectl and resize the home partition from there.
If not, what is the safe way to shrink that LUKS-partition size? Maybe then systemd-homed would do fallocate for less space and it would work.
If from my assumptions you could tell that I'm looking in the wrong direction, please give me a hint. Thanks for taking time to read it!
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 15:31 [Help] Can't login to my systemd-homed user account due to fallocate failure Andrii Zymohliad
@ 2020-08-23 15:37 ` Hugo Mills
2020-08-23 16:21 ` Andrei Borzenkov
2020-08-24 2:23 ` Chris Murphy
2020-08-24 2:50 ` Chris Murphy
1 sibling, 2 replies; 38+ messages in thread
From: Hugo Mills @ 2020-08-23 15:37 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: linux-btrfs
On Sun, Aug 23, 2020 at 03:31:47PM +0000, Andrii Zymohliad wrote:
> Hello! I've lost the ability to log in to my systemd-homed user account, and after some investigation on Systemd mailing list I was directed here. I would be very grateful for any help!
>
> My root partition is ~475GiB with BTRFS, my home partition is a ~400GiB LUKS-encrypted partition on a loopback file (also BTRFS) created by systemd-homed (residing at /home/azymohliad.home). Which leaves ~75GiB for the rest of the root FS.
>
> Recently I've lost the ability to log in to that user account, because during authentication systemd does fallocate call for the image. CLI alternative for my case (suggested on systemd mailing list, I don't really know what is it) is:
>
> fallocate -l 403G -n /home/azymohliad.home
>
> Which fails:
>
> fallocate: fallocate failed: No space left on device
>
> My first idea was that I occupied all those ~75GiB on a root partition, but cleaning didn't help (I definitely released more space than I could occupy during the last working session).
>
> Here are some details about my system:
>
> uname -a
>
> Linux az-wolf-pc 5.8.3-arch1-1 #1 SMP PREEMPT Fri, 21 Aug 2020 16:54:16 +0000 x86_64 GNU/Linux
>
> btrfs --version
>
> btrfs-progs v5.7
>
>
> I can mount my home manually like this:
>
> losetup -fP /home/azymohliad.home
> cryptsetup open /dev/loop0p1
> mount /dev/mapper/home /mnt
>
> and then,
>
> btrfs fi show
>
> Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
> Total devices 1 FS bytes used 299.71GiB
> devid 1 size 476.44GiB used 476.44GiB path /dev/nvme0n1p2
>
> Label: 'azymohliad' uuid: 4ffae38b-42c9-4e53-89a1-3d21cd862938
> Total devices 1 FS bytes used 221.92GiB
> devid 1 size 402.72GiB used 258.02GiB path /dev/mapper/home
>
>
> btrfs fi df /
>
> Data, single: total=475.43GiB, used=299.28GiB
> System, single: total=4.00MiB, used=80.00KiB
> Metadata, single: total=1.01GiB, used=437.05MiB
> GlobalReserve, single: total=61.03MiB, used=0.00B
>
>
> btrfs fi df /mnt
>
> Data, single: total=256.01GiB, used=221.18GiB
> System, single: total=4.00MiB, used=48.00KiB
> Metadata, single: total=2.01GiB, used=749.92MiB
> GlobalReserve, single: total=297.11MiB, used=0.00B
>
> dmesg.log: https://gitlab.com/-/snippets/2007155
The / filesystem is a clear case of needing a data balance. See
this link for what to do:
https://btrfs.wiki.kernel.org/index.php/FAQ#if_your_device_is_large_.28.3E16GiB.29
and see also this link for how to read the data you've pasted here:
https://btrfs.wiki.kernel.org/index.php/FAQ#or_My_filesystem_is_full.2C_and_I.27ve_put_almost_nothing_into_it.21
> What's interesting to me from above, the partition size on /home/azymohliad.home is 402.72GiB, but the file system size is 256.01GiB, and the image file size is 256.64GiB (from btrfs fi du /home, although ls -lh reports 403GiB). I'm not really sure, but iirc the fs and image sizes were around 403GiB too earlier. Could it be that it somehow got automatically reduced?
>
> Could I do anything to make that fallocate call (with -l 403G) working? It will allow me to authenticate to homectl and resize the home partition from there.
>
> If not, what is the safe way to shrink that LUKS-partition size? Maybe then systemd-homed would do fallocate for less space and it would work.
>
> If from my assumptions you could tell that I'm looking in the wrong direction, please give me a hint. Thanks for taking time to read it!
The /mnt filesystem (LABEL=azymohliad) looks OK. Are you seeing
problems with that one at all?
Hugo.
--
Hugo Mills | Sometimes, when I'm alone, I Google myself.
hugo@... carfax.org.uk |
http://carfax.org.uk/ |
PGP: E2AB1DE4 |
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 15:37 ` Hugo Mills
@ 2020-08-23 16:21 ` Andrei Borzenkov
2020-08-23 16:55 ` Andrii Zymohliad
2020-08-24 2:23 ` Chris Murphy
1 sibling, 1 reply; 38+ messages in thread
From: Andrei Borzenkov @ 2020-08-23 16:21 UTC (permalink / raw)
To: Hugo Mills, Andrii Zymohliad, linux-btrfs
23.08.2020 18:37, Hugo Mills пишет:
> The /mnt filesystem (LABEL=azymohliad) looks OK. Are you seeing
> problems with that one at all?
>
It's not about LUKS container. systemd tool attempts to reserve full
size of image with LUKS inside using fallocate which fails. It never
gets as far as actually unlocking it or mounting filesystem inside.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 16:21 ` Andrei Borzenkov
@ 2020-08-23 16:55 ` Andrii Zymohliad
2020-08-23 18:16 ` Andrei Borzenkov
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-23 16:55 UTC (permalink / raw)
To: Andrei Borzenkov, Hugo Mills; +Cc: linux-btrfs
On Sunday, August 23, 2020 7:21 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> It's not about LUKS container. systemd tool attempts to reserve full
> size of image with LUKS inside using fallocate which fails. It never
> gets as far as actually unlocking it or mounting filesystem inside.
Yes, thanks Andrei for additional clarification. Here I unlocked and mounted home partition manually just to show what's the state of filesystem on it (I guess that's not important here). Usually, it should be unlocked and mounted by systemd, but it fails because of fallocate failure.
On Sunday, August 23, 2020 6:37 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> The / filesystem is a clear case of needing a data balance. See
> this link for what to do:
>
> https://btrfs.wiki.kernel.org/index.php/FAQ#if_your_device_is_large_.28.3E16GiB.29
>
> and see also this link for how to read the data you've pasted here:
>
> https://btrfs.wiki.kernel.org/index.php/FAQ#or_My_filesystem_is_full.2C_and_I.27ve_put_almost_nothing_into_it.21
Thanks for a very quick response and links, Hugo, those are definitely useful for me. But the issue here seems more complicated. I forgot to write about it, but the state of root filesystem differs before and after fallocate attempt (or trying to log in as systemd-homed user).
Before:
# btrfs fi show
Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
Total devices 1 FS bytes used 299.71GiB
devid 1 size 476.44GiB used 301.03GiB path /dev/nvme0n1p2
# btrfs fi df /
Data, single: total=300.00GiB, used=299.29GiB
System, single: total=32.00MiB, used=48.00KiB
Metadata, single: total=1.00GiB, used=438.38MiB
GlobalReserve, single: total=62.16MiB, used=0.00B
After:
# btrfs fi show
Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
Total devices 1 FS bytes used 299.71GiB
devid 1 size 476.44GiB used 476.44GiB path /dev/nvme0n1p2
btrfs fi df /
Data, single: total=475.41GiB, used=299.29GiB
System, single: total=32.00MiB, used=80.00KiB
Metadata, single: total=1.00GiB, used=438.39MiB
GlobalReserve, single: total=62.16MiB, used=0.00B
If I do "btrfs balance start /" it becomes like before fallocate attempt, but it still doesn't fix the issue for me. fallocate and hence "homectl authenticate" would still fail.
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 16:55 ` Andrii Zymohliad
@ 2020-08-23 18:16 ` Andrei Borzenkov
2020-08-23 18:49 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Andrei Borzenkov @ 2020-08-23 18:16 UTC (permalink / raw)
To: Andrii Zymohliad, Hugo Mills; +Cc: linux-btrfs
23.08.2020 19:55, Andrii Zymohliad пишет:
> On Sunday, August 23, 2020 7:21 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>> It's not about LUKS container. systemd tool attempts to reserve full
>> size of image with LUKS inside using fallocate which fails. It never
>> gets as far as actually unlocking it or mounting filesystem inside.
>
> Yes, thanks Andrei for additional clarification. Here I unlocked and mounted home partition manually just to show what's the state of filesystem on it (I guess that's not important here). Usually, it should be unlocked and mounted by systemd, but it fails because of fallocate failure.
>
>
> On Sunday, August 23, 2020 6:37 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
>> The / filesystem is a clear case of needing a data balance. See
>> this link for what to do:
>>
>> https://btrfs.wiki.kernel.org/index.php/FAQ#if_your_device_is_large_.28.3E16GiB.29
>>
>> and see also this link for how to read the data you've pasted here:
>>
>> https://btrfs.wiki.kernel.org/index.php/FAQ#or_My_filesystem_is_full.2C_and_I.27ve_put_almost_nothing_into_it.21
>
>
> Thanks for a very quick response and links, Hugo, those are definitely useful for me. But the issue here seems more complicated. I forgot to write about it, but the state of root filesystem differs before and after fallocate attempt (or trying to log in as systemd-homed user).
>
> Before:
>
> # btrfs fi show
>
> Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
> Total devices 1 FS bytes used 299.71GiB
> devid 1 size 476.44GiB used 301.03GiB path /dev/nvme0n1p2
>
> # btrfs fi df /
>
> Data, single: total=300.00GiB, used=299.29GiB
> System, single: total=32.00MiB, used=48.00KiB
> Metadata, single: total=1.00GiB, used=438.38MiB
> GlobalReserve, single: total=62.16MiB, used=0.00B
>
> After:
>
> # btrfs fi show
>
> Label: none uuid: b68411ce-702a-4259-9121-ac21c9119ddf
> Total devices 1 FS bytes used 299.71GiB
> devid 1 size 476.44GiB used 476.44GiB path /dev/nvme0n1p2
>
> btrfs fi df /
>
> Data, single: total=475.41GiB, used=299.29GiB
> System, single: total=32.00MiB, used=80.00KiB
> Metadata, single: total=1.00GiB, used=438.39MiB
> GlobalReserve, single: total=62.16MiB, used=0.00B
>
> If I do "btrfs balance start /" it becomes like before fallocate attempt, but it still doesn't fix the issue for me. fallocate and hence "homectl authenticate" would still fail.
>
Output of "btrfs fi us /" before and after fallocate would probably give
more information. Also "btrfs sub li /" and "btrfs qgroup show -re /"
would be interesting. And "btrfs fi du /home/azymohliad.home".
Rough math so far - filesystem has 175GiB free space. Filesystem inside
container uses 258GiB so assuming container consumes exactly the same
space, in the worst case it needs 145GiB on containing filesystem for
full size. There should be enough free (even unallocated) space. I
thought there could be some snapshots that force btrfs to attempt to
allocate full 403GiB, bit in my testing even after creating read-only
snapshots fallocate does not attempt to duplicate space.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 18:16 ` Andrei Borzenkov
@ 2020-08-23 18:49 ` Andrii Zymohliad
2020-08-24 3:01 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-23 18:49 UTC (permalink / raw)
To: Andrei Borzenkov; +Cc: Hugo Mills, linux-btrfs
> Output of "btrfs fi us /" before and after fallocate would probably give
> more information. Also "btrfs sub li /" and "btrfs qgroup show -re /"
> would be interesting. And "btrfs fi du /home/azymohliad.home".
Collected the outputs of all those commands here (in order of execution): https://gitlab.com/-/snippets/2007189
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 18:49 ` Andrii Zymohliad
@ 2020-08-24 3:01 ` Chris Murphy
0 siblings, 0 replies; 38+ messages in thread
From: Chris Murphy @ 2020-08-24 3:01 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Andrei Borzenkov, Hugo Mills, linux-btrfs
On Sun, Aug 23, 2020 at 12:49 PM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> > Output of "btrfs fi us /" before and after fallocate would probably give
> > more information. Also "btrfs sub li /" and "btrfs qgroup show -re /"
> > would be interesting. And "btrfs fi du /home/azymohliad.home".
>
> Collected the outputs of all those commands here (in order of execution): https://gitlab.com/-/snippets/2007189
>
[pasted from above snippets url]
>># btrfs fi du /home/azymohliad.home
>>
>> Total Exclusive Set shared Filename
>> 256.64GiB 234.46GiB 22.18GiB /home/azymohliad.home
When you manually mount this luks file, what do you get for 'btrfs fi
us /home/azymohliad' ?
Because if that file system is 403G, and sd-homed is trying to
fallocate a 256G file to be 403G, that's +147. And also in your
snippets from '/' btfs fi us
>>Free (estimated): 176.17GiB (min: 176.17GiB)
The fallocate should work. Has this home file been snapshot or reflink
copied or deduped?
filefrag -v /home/azymohliad.home | grep shared
Here's the thing. /home/azymohliad.home, the file, should be the same
size as the Btrfs on that file. Unless it's been subject to discards,
and is thus sparse already for some reason.
Is it possible the file is being trimmed then fallocated then trimmed
then fallocated? This could be new behavior in sd-246.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 15:37 ` Hugo Mills
2020-08-23 16:21 ` Andrei Borzenkov
@ 2020-08-24 2:23 ` Chris Murphy
1 sibling, 0 replies; 38+ messages in thread
From: Chris Murphy @ 2020-08-24 2:23 UTC (permalink / raw)
To: Hugo Mills, Andrii Zymohliad, linux-btrfs
On Sun, Aug 23, 2020 at 9:37 AM Hugo Mills <hugo@carfax.org.uk> wrote:
>
> On Sun, Aug 23, 2020 at 03:31:47PM +0000, Andrii Zymohliad wrote:
> > btrfs fi df /
> >
> > Data, single: total=475.43GiB, used=299.28GiB
> > System, single: total=4.00MiB, used=80.00KiB
> > Metadata, single: total=1.01GiB, used=437.05MiB
> > GlobalReserve, single: total=61.03MiB, used=0.00B
> >
> >
> The / filesystem is a clear case of needing a data balance. See
> this link for what to do:
If it really needs to be balanced, it's a bug. It's 2020, this is a
5.8.3 kernel, we can't keep telling people they should baby sit Btrfs
in such cases. And also it's not clear it needs a data balance.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-23 15:31 [Help] Can't login to my systemd-homed user account due to fallocate failure Andrii Zymohliad
2020-08-23 15:37 ` Hugo Mills
@ 2020-08-24 2:50 ` Chris Murphy
2020-08-24 6:11 ` Andrii Zymohliad
1 sibling, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-24 2:50 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: linux-btrfs
On Sun, Aug 23, 2020 at 9:31 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
> My root partition is ~475GiB with BTRFS, my home partition is a ~400GiB LUKS-encrypted partition on a loopback file (also BTRFS) created by systemd-homed (residing at /home/azymohliad.home). Which leaves ~75GiB for the rest of the root FS.
>
> Recently I've lost the ability to log in to that user account, because during authentication systemd does fallocate call for the image. CLI alternative for my case (suggested on systemd mailing list, I don't really know what is it) is:
>
> fallocate -l 403G -n /home/azymohliad.home
>
> Which fails:
>
> fallocate: fallocate failed: No space left on device
I don't understand why homectl is fallocating just to activate. I'd
expect homectl only uses fallocate for create and resize. Can you
confirm the exact homectl command you're using?
There is a homectl option that might help, when activating, --luks-discard=true
A consequence of this option, however, is it will make the backing
file sparse. i.e. it'll punch holes in it, freeing up space on the
underlying file system. You should read the warning for the
--allow-discards option in 'man cryptsetup' which is what 'homectl
--luks-discard' is using.
Also, this option will fairly quickly fstrim the user home upon
activation (last time I tested it on systemd-245, some things have
changed in 246). And that will erase all evidence about why you're
having this problem in the first place.
Preferably if you could post the results from:
ls -ls /home/azymohliad.home
Grab this debug file from upstream btrfs-progs
https://github.com/kdave/btrfs-progs/blob/master/btrfs-debugfs
sudo ./btrfs-debugfs -b /
Mount or remount the file system with mount option enospc_debug, and
then reproduce the problem, i.e. exact same homectl command you're
using that's failing with out of space error.
> I can mount my home manually like this:
>
> losetup -fP /home/azymohliad.home
> cryptsetup open /dev/loop0p1
> mount /dev/mapper/home /mnt
Yeah, exactly. Therefore I'll argue this is a systemd-homed bug. If
fallocate fails, systemd-homed can't just lock users out of their user
home. It needs to have a fallback where it doesn't do the fallocate,
and warns the user about whatever dependent operation also can't
happen as a result. But the user home should activate.
> What's interesting to me from above, the partition size on /home/azymohliad.home is 402.72GiB, but the file system size is 256.01GiB, and the image file size is 256.64GiB (from btrfs fi du /home, although ls -lh reports 403GiB). I'm not really sure, but iirc the fs and image sizes were around 403GiB too earlier. Could it be that it somehow got automatically reduced?
I'm confused. But I'm gonna set this aside for now until you post back
with the other information.
> Could I do anything to make that fallocate call (with -l 403G) working? It will allow me to authenticate to homectl and resize the home partition from there.
At the expense of sounding like a smart ass, yes you can make the root
file system bigger (by adding another device) and then presumably the
fallocate will work. But the fallocate seems wrong to me - but I also
don't know what command you're using and why fallocate is even being
used.
>
> If not, what is the safe way to shrink that LUKS-partition size? Maybe then systemd-homed would do fallocate for less space and it would work.
>
> If from my assumptions you could tell that I'm looking in the wrong direction, please give me a hint. Thanks for taking time to read it!
For sure my top advice, is since you can manually mount this sd-homed
home, *freshen your backups now* while you still can. Later versions
of sd-homed create a subvolume on that LUKS Btrfs file system, so you
can snapshot it and use btrfs-send if you want or however else you're
doing backups.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 2:50 ` Chris Murphy
@ 2020-08-24 6:11 ` Andrii Zymohliad
2020-08-24 7:13 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-24 6:11 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs
Hello Chris! Thanks a lot for so many ideas, I updated this snippet with most of the commands you asked the output for: https://gitlab.com/-/snippets/2007189, more details inline.
> I don't understand why homectl is fallocating just to activate. I'd
> expect homectl only uses fallocate for create and resize. Can you
> confirm the exact homectl command you're using?
Any homectl command that requires authentication. For example:
homectl authenticate azymohliad
After typing password, the output is the following:
Operation on home azymohliad failed: Not enough disk space for home azymohliad
Here is the part about fallocate on Systemd mailing list: https://lists.freedesktop.org/archives/systemd-devel/2020-August/045074.html (from the same issue discussion). Andrei Borzenkov might give more insight about it (if you are reading this and when you have time).
> Preferably if you could post the results from:
>
> ls -ls /home/azymohliad.home
269111212 -rw------- 1 root root 432439787520 сер 23 18:20 /home/azymohliad.home
Also, added to the gitlab snippet.
> When you manually mount this luks file, what do you get for 'btrfs fi
> us /home/azymohliad' ?
Added to the gitlab snippet.
> Because if that file system is 403G, and sd-homed is trying to
> fallocate a 256G file to be 403G, that's +147. And also in your
> snippets from '/' btfs fi us
>
> > > Free (estimated): 176.17GiB (min: 176.17GiB)
>
> The fallocate should work. Has this home file been snapshot or reflink
> copied or deduped?
I haven't made any explicitly, but I'm not very familiar with those concepts, maybe sd-homed did it implicitly.
> filefrag -v /home/azymohliad.home | grep shared
This one is 18k lines, uploaded here: https://gitlab.com/-/snippets/2007301
> Here's the thing. /home/azymohliad.home, the file, should be the same
> size as the Btrfs on that file. Unless it's been subject to discards,
> and is thus sparse already for some reason.
>
> Is it possible the file is being trimmed then fallocated then trimmed
> then fallocated? This could be new behavior in sd-246.
Idk, I guess it's better to go back to Systemd mailing list for that, but just in case I've added "homectl inspect azymohliad" output to the gitlab snippet. Here's what it says about discard:
LUKS Discard: online=no offline=yes
> Grab this debug file from upstream btrfs-progs
> https://github.com/kdave/btrfs-progs/blob/master/btrfs-debugfs
>
> sudo ./btrfs-debugfs -b /
>
> Mount or remount the file system with mount option enospc_debug, and
> then reproduce the problem, i.e. exact same homectl command you're
> using that's failing with out of space error.
I didn't quite get what information I should extract from this.
Here is the btrfs-debugfs output: https://gitlab.com/-/snippets/2007302
I remounted root filesystem as with enospc_debug, reproduced the issue with "homectl authenticate azymohliad", but it didn't print any additional info, nor any additional logs in journalctl.
> For sure my top advice, is since you can manually mount this sd-homed
> home, freshen your backups now while you still can. Later versions
> of sd-homed create a subvolume on that LUKS Btrfs file system, so you
> can snapshot it and use btrfs-send if you want or however else you're
> doing backups.
Thanks! I'm unfamiliar with snapshots and btrfs-send yet, but it's the perfect time to learn.
> There is a homectl option that might help, when activating, --luks-discard=true
>
> A consequence of this option, however, is it will make the backing
> file sparse. i.e. it'll punch holes in it, freeing up space on the
> underlying file system. You should read the warning for the
> --allow-discards option in 'man cryptsetup' which is what 'homectl
> --luks-discard' is using.
>
> Also, this option will fairly quickly fstrim the user home upon
> activation (last time I tested it on systemd-245, some things have
> changed in 246). And that will erase all evidence about why you're
> having this problem in the first place.
Gonna read about this too, thanks!
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 6:11 ` Andrii Zymohliad
@ 2020-08-24 7:13 ` Chris Murphy
2020-08-24 7:25 ` Andrii Zymohliad
2020-08-24 15:23 ` Andrei Borzenkov
0 siblings, 2 replies; 38+ messages in thread
From: Chris Murphy @ 2020-08-24 7:13 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: linux-btrfs, Andrei Borzenkov
On Mon, Aug 24, 2020 at 12:12 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> Hello Chris! Thanks a lot for so many ideas, I updated this snippet with most of the commands you asked the output for: https://gitlab.com/-/snippets/2007189, more details inline.
>> # ls -lhs /home/azymohliad.home
>> 257G -rw------- 1 root root 403G сер 23 18:20 /home/azymohliad.home
OK so it's a sparse file..
> This one is 18k lines, uploaded here: https://gitlab.com/-/snippets/2007301
There are many shared extents, so it has been subject to snapshot,
reflink copy, or dedup.
To my knowledge, sd-homed doesn't do any of these things. Something
else did it. Are you using snapper or timeshift? Have you used 'cp' to
duplicate the home backing file?
What do you get for
lsattr /home/azymohliad.home
> Idk, I guess it's better to go back to Systemd mailing list for that, but just in case I've added "homectl inspect azymohliad" output to the gitlab snippet. Here's what it says about discard:
>
> LUKS Discard: online=no offline=yes
I'm not sure what it means, but I'll ask on systemd list.
> Here is the btrfs-debugfs output: https://gitlab.com/-/snippets/2007302
OK too late, it's already been balanced.
I think we've got a pretty good idea what's going on. The current work
around in your case will be to use the --luks-discard=true option when
activating. This will avoid the fallocate step. But the question
remains why the fallocate fails. I suspect it has something to do with
shared extents. Somewhere you have one or more snapshots containing
this home file. And that's causing some confusion.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 7:13 ` Chris Murphy
@ 2020-08-24 7:25 ` Andrii Zymohliad
2020-08-24 8:28 ` Andrii Zymohliad
2020-08-24 15:23 ` Andrei Borzenkov
1 sibling, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-24 7:25 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
> Are you using snapper or timeshift?
No
> Have you used 'cp' to duplicate the home backing file?
The /home/azymohliad.home file itself - no
> What do you get for
>
> lsattr /home/azymohliad.home
---------------C---- /home/azymohliad.home
> I think we've got a pretty good idea what's going on. The current work
> around in your case will be to use the --luks-discard=true option when
> activating.
Thanks! I'm gonna finish reading this http://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html
to have a better idea what discard implies, and then will try it out.
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 7:25 ` Andrii Zymohliad
@ 2020-08-24 8:28 ` Andrii Zymohliad
2020-08-24 8:32 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-24 8:28 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
> I think we've got a pretty good idea what's going on. The current work
> around in your case will be to use the --luks-discard=true option when
> activating.
Hm.. It doesn't help.
# homectl activate --luks-discard=true azymohliad
or
# homectl authenticate --luks-discard=true azymohliad
prints exactly the same error as without --luks-discard option:
Operation on home azymohliad failed: Not enough disk space for home azymohliad
and the allocated space on / goes to 476G again (so it seems it still tries to do fallocate).
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 8:28 ` Andrii Zymohliad
@ 2020-08-24 8:32 ` Andrii Zymohliad
2020-08-24 18:51 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-24 8:32 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
Sorry, my bad.
# homectl update --luks-discard=on azymohliad
Does the job. I can log in again now. Thank you very very much!!!
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 8:32 ` Andrii Zymohliad
@ 2020-08-24 18:51 ` Chris Murphy
2020-08-25 7:22 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-24 18:51 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, linux-btrfs, Andrei Borzenkov
On Mon, Aug 24, 2020 at 2:33 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> Sorry, my bad.
>
> # homectl update --luks-discard=on azymohliad
>
> Does the job. I can log in again now. Thank you very very much!!!
Seems like bool should take either yes, true, on, or 1. But I'm an odd
duck. Anyway! Glad it works.
Next question is to find out why you have shared extents.
filefrag -v /home/azymohliad.home | grep -m 10 shared
Any of those shared lines that doesn't also include unwritten (ok
maybe unwritten is fine but I only tested shared without unwritten),
use the value in the fourth column which is the physical_offset start
(filefrag calls it physical but on btrfs it's actually a logical
extent).
The value for my case is 1001165.. so I can do the math in the command:
# btrfs inspect-internal logical-resolve $[1001165*4096] /mnt
/mnt/libvimages/fedora.raw
/mnt/libvimages/fedora.raw.bak
That found the two files that share that extent. From that we can find
out why you have shared extents for your home backing file.
Note: I think this command wants to be pointed at the top-level of the
file system. It failed otherwise.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 18:51 ` Chris Murphy
@ 2020-08-25 7:22 ` Andrii Zymohliad
2020-08-25 19:05 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-25 7:22 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
> Seems like bool should take either yes, true, on, or 1. But I'm an odd
> duck. Anyway! Glad it works.
Ah, sorry, I didn't notice I changed true to on. It doesn't matter. The thing is, it should be "homectl update", and not during activation/authorization itself.
> Next question is to find out why you have shared extents.
# filefrag -v /home/azymohliad.home | grep -m 10 shared
55082: 52756487..52756516: 291198215.. 291198244: 30: 291722503: shared
55083: 52756518..52756522: 291198246.. 291198250: 5: shared
55084: 52756524..52756528: 291198252.. 291198256: 5: shared
55085: 52756530..52756530: 291198258.. 291198258: 1: shared
55086: 52756532..52756534: 291198260.. 291198262: 3: shared
55087: 52756536..52756546: 291198264.. 291198274: 11: shared
55088: 52756548..52756553: 291198276.. 291198281: 6: shared
55089: 52756555..52756557: 291198283.. 291198285: 3: shared
55090: 52756559..52756578: 291198287.. 291198306: 20: shared
55091: 52756582..52756595: 291198310.. 291198323: 14: shared
And for all of these extents the output of "btrfs inspect-internal" is:
//home/azymohliad.home
I call it on root / filesystem, e.g.:
# btrfs inspect-internal logical-resolve $[291198246 * 4096] /
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-25 7:22 ` Andrii Zymohliad
@ 2020-08-25 19:05 ` Chris Murphy
2020-08-25 20:47 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-25 19:05 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, linux-btrfs, Andrei Borzenkov
On Tue, Aug 25, 2020 at 1:22 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> > Seems like bool should take either yes, true, on, or 1. But I'm an odd
> > duck. Anyway! Glad it works.
>
> Ah, sorry, I didn't notice I changed true to on. It doesn't matter. The thing is, it should be "homectl update", and not during activation/authorization itself.
>
> > Next question is to find out why you have shared extents.
>
> # filefrag -v /home/azymohliad.home | grep -m 10 shared
>
> 55082: 52756487..52756516: 291198215.. 291198244: 30: 291722503: shared
> 55083: 52756518..52756522: 291198246.. 291198250: 5: shared
> 55084: 52756524..52756528: 291198252.. 291198256: 5: shared
> 55085: 52756530..52756530: 291198258.. 291198258: 1: shared
> 55086: 52756532..52756534: 291198260.. 291198262: 3: shared
> 55087: 52756536..52756546: 291198264.. 291198274: 11: shared
> 55088: 52756548..52756553: 291198276.. 291198281: 6: shared
> 55089: 52756555..52756557: 291198283.. 291198285: 3: shared
> 55090: 52756559..52756578: 291198287.. 291198306: 20: shared
> 55091: 52756582..52756595: 291198310.. 291198323: 14: shared
>
> And for all of these extents the output of "btrfs inspect-internal" is:
>
> //home/azymohliad.home
>
> I call it on root / filesystem, e.g.:
>
> # btrfs inspect-internal logical-resolve $[291198246 * 4096] /
Yeah but is / the top-level?
btrfs sub list -t /
mount | grep btrfs
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-25 19:05 ` Chris Murphy
@ 2020-08-25 20:47 ` Andrii Zymohliad
2020-08-25 22:34 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-25 20:47 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
> Yeah but is / the top-level?
>
> btrfs sub list -t /
ID gen top level path
-- --- --------- ----
258 94017 5 var/lib/portables
259 94017 5 var/lib/machines
> mount | grep btrfs
/dev/nvme0n1p2 on / type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)
/dev/mapper/home-azymohliad on /home/azymohliad type btrfs (rw,nosuid,nodev,relatime,ssd,discard,noacl,space_cache,subvolid=5,subvol=/)
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-25 20:47 ` Andrii Zymohliad
@ 2020-08-25 22:34 ` Chris Murphy
2020-08-26 13:37 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-25 22:34 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, linux-btrfs, Andrei Borzenkov
On Tue, Aug 25, 2020 at 2:47 PM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> > Yeah but is / the top-level?
> >
> > btrfs sub list -t /
>
> ID gen top level path
> -- --- --------- ----
> 258 94017 5 var/lib/portables
> 259 94017 5 var/lib/machines
>
> > mount | grep btrfs
>
> /dev/nvme0n1p2 on / type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)
> /dev/mapper/home-azymohliad on /home/azymohliad type btrfs (rw,nosuid,nodev,relatime,ssd,discard,noacl,space_cache,subvolid=5,subvol=/)
>
It might be that for each extent there's a mix of shared and unshared
extents so maybe you just got unlucky and hit a block that isn't
shared in that extent.
Try more of those shared addresses. If you can do python at all,
there's python-btrfs
https://github.com/knorrie/python-btrfs
He gave this example, which I don't follow because I don't do python
at all, but the vaddr is the filefrag -v address from before * 4096.
You could just iterate by incrementing by 4096 and stop as soon as you
get two files (or two inodes). And then us find -inum to track down
the inode numbers to files.
https://github.com/knorrie/python-btrfs
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-25 22:34 ` Chris Murphy
@ 2020-08-26 13:37 ` Andrii Zymohliad
2020-08-26 13:40 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-26 13:37 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
> Try more of those shared addresses. If you can do python at all,
> there's python-btrfs
> https://github.com/knorrie/python-btrfs
>
> He gave this example, which I don't follow because I don't do python
> at all, but the vaddr is the filefrag -v address from before * 4096.
> You could just iterate by incrementing by 4096 and stop as soon as you
> get two files (or two inodes). And then us find -inum to track down
> the inode numbers to files.
>
> https://github.com/knorrie/python-btrfs
Which example exactly? There are many in the repo.
I'm ok with python, but to save time learning the API of the library I've made a simple bash script:
mkdir -p shared_extents
addrs=$(filefrag -v /home/azymohliad.home | grep shared | awk '{print $3}' | tr -d '.')
for addr in $addrs; do
btrfs inspect-internal logical-resolve $[$addr * 4096] / > shared_extents/$addr.txt
if [ $(wc -l shared_extents/$addr.txt | awk '{print $1}') -gt 1 ]; then
cat shared_extents/$addr.txt
fi
done
It should print the outputs of "btrfs inspect-internal logical-resolve $[$addr * 4096]" if it's more than 1 line. And it prints nothing.
Every "btrfs inspect-internal ..." output here is just "//home/azymohliad.home" for each of these shared extents (I can double check it from "cat shared_extents/*").
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 13:37 ` Andrii Zymohliad
@ 2020-08-26 13:40 ` Andrii Zymohliad
2020-08-26 19:06 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-26 13:40 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs, Andrei Borzenkov
Interestingly, I have much less shared extents now:
# filefrag -v /home/azymohliad.home | grep shared | wc -l
1679
It was over 18k the first time you asked me to check it (even before I enabled discard in homectl).
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 13:40 ` Andrii Zymohliad
@ 2020-08-26 19:06 ` Chris Murphy
[not found] ` <dHPyoaNUuxoqxJjpSUjcxZvk21ew2ObKWFFhX0efJKBqjG8m27px3mPupviQuM3HYQDEcYQGFQ9jOprBuAX4x1Em3oVOyDl6EhKwEJSiuXs=@protonmail.com>
2020-08-27 5:39 ` Andrei Borzenkov
0 siblings, 2 replies; 38+ messages in thread
From: Chris Murphy @ 2020-08-26 19:06 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, linux-btrfs, Andrei Borzenkov
On Wed, Aug 26, 2020 at 7:40 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> Interestingly, I have much less shared extents now:
>
> # filefrag -v /home/azymohliad.home | grep shared | wc -l
> 1679
>
> It was over 18k the first time you asked me to check it (even before I enabled discard in homectl).
OK so Josef found a bug. Filefrag is showing shared extents in certain
cases when there are none. So you can stop worrying there's some copy
of the file somewhere. And stop looking for it.
https://github.com/btrfs/btrfs-todo/issues/6
The fallocate problem is still real, and that might take some time for
Andrei or someone else to figure out why.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread[parent not found: <dHPyoaNUuxoqxJjpSUjcxZvk21ew2ObKWFFhX0efJKBqjG8m27px3mPupviQuM3HYQDEcYQGFQ9jOprBuAX4x1Em3oVOyDl6EhKwEJSiuXs=@protonmail.com>]
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
[not found] ` <dHPyoaNUuxoqxJjpSUjcxZvk21ew2ObKWFFhX0efJKBqjG8m27px3mPupviQuM3HYQDEcYQGFQ9jOprBuAX4x1Em3oVOyDl6EhKwEJSiuXs=@protonmail.com>
@ 2020-08-26 19:15 ` Chris Murphy
2020-08-26 19:57 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-26 19:15 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, Btrfs BTRFS, Andrei Borzenkov
On Wed, Aug 26, 2020 at 1:09 PM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> Ok. Thanks a lot for your great help, Chris!
No problem.
> On Aug 26, 2020, 22:06, Chris Murphy < lists@colorremedies.com> wrote:
>
> The fallocate problem is still real, and that might take some time for
> Andrei or someone else to figure out why.
Actually, if you can reproduce the fallocate problem and strace it,
that would be helpful. For sure, homectl deactivate, for starters. And
then just 'strace fallocate' that file the same way you did
previously. I assume it still fails. Put that strace into a text file
or pastebin (month expiration should be enough) and link it.
And also paste into your reply:
grep -r . /sys/fs/btrfs/<uuidofbtrfs>/allocation/
You'll need to use the uuid for the btrfs that has this file on it.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 19:15 ` Chris Murphy
@ 2020-08-26 19:57 ` Andrii Zymohliad
2020-08-26 21:06 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-26 19:57 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS, Andrei Borzenkov
> Actually, if you can reproduce the fallocate problem and strace it,
> that would be helpful. For sure, homectl deactivate, for starters. And
> then just 'strace fallocate' that file the same way you did
> previously. I assume it still fails. Put that strace into a text file
> or pastebin (month expiration should be enough) and link it.
Yep, still fails. Here's the one with exactly the same arguments as before (-l 403G): https://gitlab.com/-/snippets/2008996
But I did "homectl resize azymohliad 300G" so now "ls -lhs" reports:
228G -rw------- 1 root root 300G сер 26 22:50 /home/azymohliad.home
But even "fallocate -l 300G -n /home/azymohliad.home" fails the same way. Here's strace: https://gitlab.com/-/snippets/2008993
(I hope gitlab is ok too, you can add /raw to URLs to get raw txt file)
> And also paste into your reply:
>
> grep -r . /sys/fs/btrfs/<uuidofbtrfs>/allocation/
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/disk_used:474251264
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_pinned:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_used:474251264
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/single/used_bytes:474251264
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/single/total_bytes:1073741824
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/total_bytes_pinned:163840
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/disk_total:1073741824
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/total_bytes:1073741824
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_reserved:163840
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_readonly:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_may_use:74252288
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/flags:4
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/disk_used:81920
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_pinned:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_used:81920
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/single/used_bytes:81920
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/single/total_bytes:33554432
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/total_bytes_pinned:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/disk_total:33554432
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/total_bytes:33554432
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_reserved:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_readonly:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_may_use:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/flags:2
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/global_rsv_reserved:68730880
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/disk_used:314346594304
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_pinned:0
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_used:314346594304
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/single/used_bytes:314346594304
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/single/total_bytes:510463901696
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/total_bytes_pinned:4096
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/disk_total:510463901696
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/total_bytes:510463901696
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_reserved:4096
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_readonly:131072
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_may_use:4096
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/flags:1
/sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/global_rsv_size:68730880
I also added this output to the collection here: https://gitlab.com/-/snippets/2007189
I'll keep all those gitlab snippets that I shared in this thread online for a while (some months at least).
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 19:57 ` Andrii Zymohliad
@ 2020-08-26 21:06 ` Chris Murphy
2020-08-26 21:08 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-26 21:06 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, Btrfs BTRFS, Andrei Borzenkov
On Wed, Aug 26, 2020 at 1:57 PM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> > Actually, if you can reproduce the fallocate problem and strace it,
> > that would be helpful. For sure, homectl deactivate, for starters. And
> > then just 'strace fallocate' that file the same way you did
> > previously. I assume it still fails. Put that strace into a text file
> > or pastebin (month expiration should be enough) and link it.
>
> Yep, still fails. Here's the one with exactly the same arguments as before (-l 403G): https://gitlab.com/-/snippets/2008996
>
> But I did "homectl resize azymohliad 300G" so now "ls -lhs" reports:
>
> 228G -rw------- 1 root root 300G сер 26 22:50 /home/azymohliad.home
>
> But even "fallocate -l 300G -n /home/azymohliad.home" fails the same way. Here's strace: https://gitlab.com/-/snippets/2008993
>
> (I hope gitlab is ok too, you can add /raw to URLs to get raw txt file)
>
>
> > And also paste into your reply:
> >
> > grep -r . /sys/fs/btrfs/<uuidofbtrfs>/allocation/
>
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/disk_used:474251264
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_pinned:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_used:474251264
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/single/used_bytes:474251264
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/single/total_bytes:1073741824
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/total_bytes_pinned:163840
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/disk_total:1073741824
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/total_bytes:1073741824
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_reserved:163840
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_readonly:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/bytes_may_use:74252288
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/metadata/flags:4
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/disk_used:81920
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_pinned:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_used:81920
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/single/used_bytes:81920
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/single/total_bytes:33554432
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/total_bytes_pinned:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/disk_total:33554432
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/total_bytes:33554432
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_reserved:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_readonly:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/bytes_may_use:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/system/flags:2
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/global_rsv_reserved:68730880
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/disk_used:314346594304
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_pinned:0
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_used:314346594304
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/single/used_bytes:314346594304
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/single/total_bytes:510463901696
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/total_bytes_pinned:4096
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/disk_total:510463901696
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/total_bytes:510463901696
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_reserved:4096
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_readonly:131072
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/bytes_may_use:4096
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/data/flags:1
> /sys/fs/btrfs/b68411ce-702a-4259-9121-ac21c9119ddf/allocation/global_rsv_size:68730880
>
> I also added this output to the collection here: https://gitlab.com/-/snippets/2007189
> I'll keep all those gitlab snippets that I shared in this thread online for a while (some months at least).
OK is the sysfs output from before or after the homectl resize? And it
matches with the second strace fallocate -l 300g ?
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 21:06 ` Chris Murphy
@ 2020-08-26 21:08 ` Chris Murphy
2020-08-26 21:14 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-26 21:08 UTC (permalink / raw)
To: Chris Murphy; +Cc: Andrii Zymohliad, Btrfs BTRFS, Andrei Borzenkov
On Wed, Aug 26, 2020 at 3:06 PM Chris Murphy <lists@colorremedies.com> wrote:
>
> OK is the sysfs output from before or after the homectl resize? And it
> matches with the second strace fallocate -l 300g ?
Figured it out. The sysfs is the first strace before the resize.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 21:08 ` Chris Murphy
@ 2020-08-26 21:14 ` Chris Murphy
2020-08-27 6:48 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-08-26 21:14 UTC (permalink / raw)
To: Chris Murphy; +Cc: Andrii Zymohliad, Btrfs BTRFS, Andrei Borzenkov
On Wed, Aug 26, 2020 at 3:08 PM Chris Murphy <lists@colorremedies.com> wrote:
>
> On Wed, Aug 26, 2020 at 3:06 PM Chris Murphy <lists@colorremedies.com> wrote:
> >
> > OK is the sysfs output from before or after the homectl resize? And it
> > matches with the second strace fallocate -l 300g ?
>
> Figured it out. The sysfs is the first strace before the resize.
Shoot. I confused myself and am not sure, so I need you to tell me
which strace that sysfs goes with, the 403g or 300g.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 21:14 ` Chris Murphy
@ 2020-08-27 6:48 ` Andrii Zymohliad
2020-09-01 15:01 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-08-27 6:48 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS, Andrei Borzenkov
> > > OK is the sysfs output from before or after the homectl resize? And it
> > > matches with the second strace fallocate -l 300g ?
> >
> > Figured it out. The sysfs is the first strace before the resize.
>
> Shoot. I confused myself and am not sure, so I need you to tell me
> which strace that sysfs goes with, the 403g or 300g.
Sorry, I haven't put it clear. I've resized it right after I was able to log in a couple of days ago. So both straces and /sys/fs outputs are after resize. And /sys/fs output is after the second strace (-l 300G).
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-27 6:48 ` Andrii Zymohliad
@ 2020-09-01 15:01 ` Chris Murphy
2020-09-01 17:06 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-09-01 15:01 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Btrfs BTRFS
On Thu, Aug 27, 2020 at 12:48 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> > > > OK is the sysfs output from before or after the homectl resize? And it
> > > > matches with the second strace fallocate -l 300g ?
> > >
> > > Figured it out. The sysfs is the first strace before the resize.
> >
> > Shoot. I confused myself and am not sure, so I need you to tell me
> > which strace that sysfs goes with, the 403g or 300g.
>
> Sorry, I haven't put it clear. I've resized it right after I was able to log in a couple of days ago. So both straces and /sys/fs outputs are after resize. And /sys/fs output is after the second strace (-l 300G).
Andrii,
We're trying to reproduce this problem and need more information.
Could you provide the following:
1. ls -li /home/azymohliad.home
2. filefrag -v /home/azymohliad.home | grep -m 10 shared
3. Pick any line, multiply the first extent (4th column) by 4096, this
is the bytenr
4. btrfs inspect-internal dump-tree -t 5 /dev/nvme0n1p2 | grep -C 10 <bytenr>
Thanks!
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 15:01 ` Chris Murphy
@ 2020-09-01 17:06 ` Andrii Zymohliad
2020-09-01 17:22 ` Chris Murphy
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-09-01 17:06 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
Hi, sure!
> 1. ls -li /home/azymohliad.home
43141 -rw------- 1 root root 322122547200 вер 1 19:57 /home/azymohliad.home
> 2. filefrag -v /home/azymohliad.home | grep -m 10 shared
87954: 52756487..52756516: 291198215.. 291198244: 30: 291722503: shared
87955: 52756518..52756522: 291198246.. 291198250: 5: shared
87956: 52756524..52756528: 291198252.. 291198256: 5: shared
87957: 52756530..52756530: 291198258.. 291198258: 1: shared
87958: 52756532..52756534: 291198260.. 291198262: 3: shared
87959: 52756536..52756546: 291198264.. 291198274: 11: shared
87960: 52756548..52756553: 291198276.. 291198281: 6: shared
87961: 52756555..52756557: 291198283.. 291198285: 3: shared
87962: 52756559..52756578: 291198287.. 291198306: 20: shared
87963: 52756582..52756595: 291198310.. 291198323: 14: shared
> 3. Pick any line, multiply the first extent (4th column) by 4096, this
> is the bytenr
>
> 4. btrfs inspect-internal dump-tree -t 5 /dev/nvme0n1p2 | grep -C 10 <bytenr>
No output for any of the extents above. For example:
291198215 * 4096 = 1192747888640
Then the following produces no output:
sudo btrfs inspect-internal dump-tree -t 5 /dev/nvme0n1p2 | grep -C 10 1192747888640
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 17:06 ` Andrii Zymohliad
@ 2020-09-01 17:22 ` Chris Murphy
2020-09-01 17:27 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Chris Murphy @ 2020-09-01 17:22 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, Btrfs BTRFS
On Tue, Sep 1, 2020 at 11:06 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> Hi, sure!
>
> > 1. ls -li /home/azymohliad.home
>
> 43141 -rw------- 1 root root 322122547200 вер 1 19:57 /home/azymohliad.home
>
> > 2. filefrag -v /home/azymohliad.home | grep -m 10 shared
>
> 87954: 52756487..52756516: 291198215.. 291198244: 30: 291722503: shared
> 87955: 52756518..52756522: 291198246.. 291198250: 5: shared
> 87956: 52756524..52756528: 291198252.. 291198256: 5: shared
> 87957: 52756530..52756530: 291198258.. 291198258: 1: shared
> 87958: 52756532..52756534: 291198260.. 291198262: 3: shared
> 87959: 52756536..52756546: 291198264.. 291198274: 11: shared
> 87960: 52756548..52756553: 291198276.. 291198281: 6: shared
> 87961: 52756555..52756557: 291198283.. 291198285: 3: shared
> 87962: 52756559..52756578: 291198287.. 291198306: 20: shared
> 87963: 52756582..52756595: 291198310.. 291198323: 14: shared
>
>
> > 3. Pick any line, multiply the first extent (4th column) by 4096, this
> > is the bytenr
> >
> > 4. btrfs inspect-internal dump-tree -t 5 /dev/nvme0n1p2 | grep -C 10 <bytenr>
>
> No output for any of the extents above. For example:
>
> 291198215 * 4096 = 1192747888640
>
> Then the following produces no output:
>
> sudo btrfs inspect-internal dump-tree -t 5 /dev/nvme0n1p2 | grep -C 10 1192747888640
OK it will take longer to search but omit '-t 5' and we'll just search
the whole tree dump.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 17:22 ` Chris Murphy
@ 2020-09-01 17:27 ` Andrii Zymohliad
2020-09-01 17:31 ` Andrii Zymohliad
0 siblings, 1 reply; 38+ messages in thread
From: Andrii Zymohliad @ 2020-09-01 17:27 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
> OK it will take longer to search but omit '-t 5' and we'll just search
> the whole tree dump.
This one is also empty:
sudo btrfs inspect-internal dump-tree /dev/nvme0n1p2 | grep -C 10 1192747888640
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 17:27 ` Andrii Zymohliad
@ 2020-09-01 17:31 ` Andrii Zymohliad
2020-09-01 20:45 ` Chris Murphy
2020-09-02 16:35 ` Chris Murphy
0 siblings, 2 replies; 38+ messages in thread
From: Andrii Zymohliad @ 2020-09-01 17:31 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
I wonder how can "1192747888640" be a byte number if it's more than 1TiB, and I have only 476GiB partition. Is the 4096 multiplier surely correct for my filesystem?
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 17:31 ` Andrii Zymohliad
@ 2020-09-01 20:45 ` Chris Murphy
2020-09-02 16:35 ` Chris Murphy
1 sibling, 0 replies; 38+ messages in thread
From: Chris Murphy @ 2020-09-01 20:45 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, Btrfs BTRFS
On Tue, Sep 1, 2020 at 11:31 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> I wonder how can "1192747888640" be a byte number if it's more than 1TiB, and I have only 476GiB partition. Is the 4096 multiplier surely correct for my filesystem?
>
Yes, it's a Btrfs logical address so it can be a value larger than the
physical size of the file system.
Try command 2 again and see if that extent and block are still there.
It should work.
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-09-01 17:31 ` Andrii Zymohliad
2020-09-01 20:45 ` Chris Murphy
@ 2020-09-02 16:35 ` Chris Murphy
1 sibling, 0 replies; 38+ messages in thread
From: Chris Murphy @ 2020-09-02 16:35 UTC (permalink / raw)
To: Andrii Zymohliad; +Cc: Chris Murphy, Btrfs BTRFS
On Tue, Sep 1, 2020 at 11:31 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:
>
> I wonder how can "1192747888640" be a byte number if it's more than 1TiB, and I have only 476GiB partition. Is the 4096 multiplier surely correct for my filesystem?
>
On second thought, it's probably easier if you just make an image
(metadata only). It'll need to be unmounted.
btrfs-image -ss -c4 -t4 /dev/nvme /path/to/file
-ss will scrub the file names. From prior output about this fs, it
looks like the resulting file will be around 100M. You can email me
the URL to the file offline.
Thanks,
--
Chris Murphy
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-26 19:06 ` Chris Murphy
[not found] ` <dHPyoaNUuxoqxJjpSUjcxZvk21ew2ObKWFFhX0efJKBqjG8m27px3mPupviQuM3HYQDEcYQGFQ9jOprBuAX4x1Em3oVOyDl6EhKwEJSiuXs=@protonmail.com>
@ 2020-08-27 5:39 ` Andrei Borzenkov
1 sibling, 0 replies; 38+ messages in thread
From: Andrei Borzenkov @ 2020-08-27 5:39 UTC (permalink / raw)
To: Chris Murphy, Andrii Zymohliad; +Cc: linux-btrfs
26.08.2020 22:06, Chris Murphy пишет:
>
> The fallocate problem is still real, and that might take some time for
> Andrei or someone else to figure out why.
>
>
Sorry? It was already answered that btrfs by design fallocates full file
size no matter what.
https://lore.kernel.org/linux-btrfs/b0d0784a-03b5-c212-f4a1-f09ff487e355@libero.it/T/#meb706f2264e80029835b1d31d82c3a468b53dcda
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 7:13 ` Chris Murphy
2020-08-24 7:25 ` Andrii Zymohliad
@ 2020-08-24 15:23 ` Andrei Borzenkov
2020-08-24 17:42 ` Goffredo Baroncelli
1 sibling, 1 reply; 38+ messages in thread
From: Andrei Borzenkov @ 2020-08-24 15:23 UTC (permalink / raw)
To: Chris Murphy, Andrii Zymohliad; +Cc: linux-btrfs
24.08.2020 10:13, Chris Murphy пишет:
>
> I think we've got a pretty good idea what's going on. The current work
> around in your case will be to use the --luks-discard=true option when
> activating. This will avoid the fallocate step. But the question
> remains why the fallocate fails. I suspect it has something to do with
> shared extents.
Actually I see the same behavior without any shared extent at all.
fallocate attempts to allocate full file size, not just additional
unallocated space:
tw:/mnt # truncate -s $[112*5]M src/file
tw:/mnt # dd if=/dev/urandom of=src/file conv=notrunc,block,sync bs=112M
count=20+2 records in
2+0 records out
234881024 bytes (235 MB, 224 MiB) copied, 1.53186 s, 153 MB/s
tw:/mnt # sync
tw:/mnt # btrfs fi us /mnt
Overall:
Device size: 1.00GiB
Device allocated: 356.00MiB
Device unallocated: 668.00MiB
Device missing: 0.00B
Used: 224.59MiB
Free (estimated): 787.81MiB (min: 787.81MiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 3.25MiB (used: 0.00B)
Multiple profiles: no
Data,single: Size:344.00MiB, Used:224.19MiB (65.17%)
/dev/vdc 344.00MiB
Metadata,single: Size:8.00MiB, Used:400.00KiB (4.88%)
/dev/vdc 8.00MiB
System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
/dev/vdc 4.00MiB
Unallocated:
/dev/vdc 668.00MiB
tw:/mnt # fallocate -l $[112*5]M -n src/file
tw:/mnt # sync
tw:/mnt # btrfs fi us /mnt
Overall:
Device size: 1.00GiB
Device allocated: 916.00MiB
Device unallocated: 108.00MiB
Device missing: 0.00B
Used: 560.78MiB
Free (estimated): 451.62MiB (min: 451.62MiB)
Data ratio: 1.00
Metadata ratio: 1.00
Global reserve: 3.25MiB (used: 0.00B)
Multiple profiles: no
Data,single: Size:904.00MiB, Used:560.38MiB (61.99%)
/dev/vdc 904.00MiB
Metadata,single: Size:8.00MiB, Used:400.00KiB (4.88%)
/dev/vdc 8.00MiB
System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
/dev/vdc 4.00MiB
Unallocated:
/dev/vdc 108.00MiB
tw:/mnt # btrfs fi du src/file
Total Exclusive Set shared Filename
560.00MiB 560.00MiB 0.00B src/file
tw:/mnt # uname -a
Linux tw.0.2.15 5.8.0-1-default #1 SMP Tue Aug 4 07:30:59 UTC 2020
(9bc0044) x86_64 x86_64 x86_64 GNU/Linux
tw:/mnt #
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [Help] Can't login to my systemd-homed user account due to fallocate failure
2020-08-24 15:23 ` Andrei Borzenkov
@ 2020-08-24 17:42 ` Goffredo Baroncelli
0 siblings, 0 replies; 38+ messages in thread
From: Goffredo Baroncelli @ 2020-08-24 17:42 UTC (permalink / raw)
To: Andrei Borzenkov, Chris Murphy, Andrii Zymohliad; +Cc: linux-btrfs
On 8/24/20 5:23 PM, Andrei Borzenkov wrote:
> 24.08.2020 10:13, Chris Murphy пишет:
>>
>> I think we've got a pretty good idea what's going on. The current work
>> around in your case will be to use the --luks-discard=true option when
>> activating. This will avoid the fallocate step. But the question
>> remains why the fallocate fails. I suspect it has something to do with
>> shared extents.
>
>
> Actually I see the same behavior without any shared extent at all.
> fallocate attempts to allocate full file size, not just additional
> unallocated space:
I don't remember if at the time it was reached a conclusion, but this topic was already discussed in the past:
https://lore.kernel.org/linux-btrfs/798a9077-bcbd-076c-a458-3403010ce8ac@libero.it/
I think that this information should be forwarded back to the systemd mailing list...
Anyway IMHO a failing fallocate for a small file is a reasonable failure.
However I am not convinced that this is still true for a filesystem image (like the systemd-homed
case is). In this case it is reasonable to thinking the image as sparse image as a way of
over-provisioning.
However it is true that not all the stack (btrfs <-> loopback <-> luks <-> btrfs) is able to manage
a case where there is no space, so fallocat'ing may be the least worst scenario...
>
> tw:/mnt # truncate -s $[112*5]M src/file
> tw:/mnt # dd if=/dev/urandom of=src/file conv=notrunc,block,sync bs=112M
> count=20+2 records in
> 2+0 records out
> 234881024 bytes (235 MB, 224 MiB) copied, 1.53186 s, 153 MB/s
> tw:/mnt # sync
> tw:/mnt # btrfs fi us /mnt
> Overall:
> Device size: 1.00GiB
> Device allocated: 356.00MiB
> Device unallocated: 668.00MiB
> Device missing: 0.00B
> Used: 224.59MiB
> Free (estimated): 787.81MiB (min: 787.81MiB)
> Data ratio: 1.00
> Metadata ratio: 1.00
> Global reserve: 3.25MiB (used: 0.00B)
> Multiple profiles: no
>
> Data,single: Size:344.00MiB, Used:224.19MiB (65.17%)
> /dev/vdc 344.00MiB
>
> Metadata,single: Size:8.00MiB, Used:400.00KiB (4.88%)
> /dev/vdc 8.00MiB
>
> System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
> /dev/vdc 4.00MiB
>
> Unallocated:
> /dev/vdc 668.00MiB
> tw:/mnt # fallocate -l $[112*5]M -n src/file
> tw:/mnt # sync
> tw:/mnt # btrfs fi us /mnt
> Overall:
> Device size: 1.00GiB
> Device allocated: 916.00MiB
> Device unallocated: 108.00MiB
> Device missing: 0.00B
> Used: 560.78MiB
> Free (estimated): 451.62MiB (min: 451.62MiB)
> Data ratio: 1.00
> Metadata ratio: 1.00
> Global reserve: 3.25MiB (used: 0.00B)
> Multiple profiles: no
>
> Data,single: Size:904.00MiB, Used:560.38MiB (61.99%)
> /dev/vdc 904.00MiB
>
> Metadata,single: Size:8.00MiB, Used:400.00KiB (4.88%)
> /dev/vdc 8.00MiB
>
> System,single: Size:4.00MiB, Used:16.00KiB (0.39%)
> /dev/vdc 4.00MiB
>
> Unallocated:
> /dev/vdc 108.00MiB
> tw:/mnt # btrfs fi du src/file
> Total Exclusive Set shared Filename
> 560.00MiB 560.00MiB 0.00B src/file
> tw:/mnt # uname -a
> Linux tw.0.2.15 5.8.0-1-default #1 SMP Tue Aug 4 07:30:59 UTC 2020
> (9bc0044) x86_64 x86_64 x86_64 GNU/Linux
> tw:/mnt #
>
--
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
^ permalink raw reply [flat|nested] 38+ messages in thread
[parent not found: <bdJVxLiFr=5FPyQSXRUbZJfFW=5FjAjsGgoMetqPHJMbg-hdy54Xt=5FZHhRetmnJ6cJ99eBlcX76wy-AvWwV715c3YndkxneSlod11P1hlaADx0s=3D@protonmail.com>]
end of thread, other threads:[~2020-09-02 16:36 UTC | newest]
Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-23 15:31 [Help] Can't login to my systemd-homed user account due to fallocate failure Andrii Zymohliad
2020-08-23 15:37 ` Hugo Mills
2020-08-23 16:21 ` Andrei Borzenkov
2020-08-23 16:55 ` Andrii Zymohliad
2020-08-23 18:16 ` Andrei Borzenkov
2020-08-23 18:49 ` Andrii Zymohliad
2020-08-24 3:01 ` Chris Murphy
2020-08-24 2:23 ` Chris Murphy
2020-08-24 2:50 ` Chris Murphy
2020-08-24 6:11 ` Andrii Zymohliad
2020-08-24 7:13 ` Chris Murphy
2020-08-24 7:25 ` Andrii Zymohliad
2020-08-24 8:28 ` Andrii Zymohliad
2020-08-24 8:32 ` Andrii Zymohliad
2020-08-24 18:51 ` Chris Murphy
2020-08-25 7:22 ` Andrii Zymohliad
2020-08-25 19:05 ` Chris Murphy
2020-08-25 20:47 ` Andrii Zymohliad
2020-08-25 22:34 ` Chris Murphy
2020-08-26 13:37 ` Andrii Zymohliad
2020-08-26 13:40 ` Andrii Zymohliad
2020-08-26 19:06 ` Chris Murphy
[not found] ` <dHPyoaNUuxoqxJjpSUjcxZvk21ew2ObKWFFhX0efJKBqjG8m27px3mPupviQuM3HYQDEcYQGFQ9jOprBuAX4x1Em3oVOyDl6EhKwEJSiuXs=@protonmail.com>
2020-08-26 19:15 ` Chris Murphy
2020-08-26 19:57 ` Andrii Zymohliad
2020-08-26 21:06 ` Chris Murphy
2020-08-26 21:08 ` Chris Murphy
2020-08-26 21:14 ` Chris Murphy
2020-08-27 6:48 ` Andrii Zymohliad
2020-09-01 15:01 ` Chris Murphy
2020-09-01 17:06 ` Andrii Zymohliad
2020-09-01 17:22 ` Chris Murphy
2020-09-01 17:27 ` Andrii Zymohliad
2020-09-01 17:31 ` Andrii Zymohliad
2020-09-01 20:45 ` Chris Murphy
2020-09-02 16:35 ` Chris Murphy
2020-08-27 5:39 ` Andrei Borzenkov
2020-08-24 15:23 ` Andrei Borzenkov
2020-08-24 17:42 ` Goffredo Baroncelli
[not found] <bdJVxLiFr=5FPyQSXRUbZJfFW=5FjAjsGgoMetqPHJMbg-hdy54Xt=5FZHhRetmnJ6cJ99eBlcX76wy-AvWwV715c3YndkxneSlod11P1hlaADx0s=3D@protonmail.com>
[not found] ` <CAJCQCtTC2yi4HRqg6fkMrxw33TVSBh=5FyAKtnX-Z1-nXSVjBW7w@mail.gmail.com>
[not found] ` <Yy8-04dbNru1LWPcNZ9UxsagH1b0KNsGn7tDEnxVOqS812IqGiwl37dj4rxAh05gEP8QoNJ5F=5FEa6CZ8iZgvnupuq5Qzc38gl69MceWMc9s=3D@protonmail.com>
[not found] ` <CAJCQCtSqe=5FoqRZWYP7iLJcGQnzZkC4vmoYVTm=5F9RPb8eb0-E6Q@mail.gmail.com>
[not found] ` <BfU9s11rmWxGNQdKqifkB1JKOJcgqAN49OZdV4LAOgo1W2AguRebwCPVosOiMVjMTzuSmsk=5FEfbkl02s31niRqtCS67WJ9S7=5Fs4jiK9afeA=3D@protonmail.com>
[not found] ` <E212ihR5U8HVCyaalepkxQUX3wOj6IXd1yUFHj-PFFtyU7ma-A49vmB8QwfQG5gUVo2nCMbVpPo7C2ccooRO0ExVrIbdLP9sBpnjMOcefHo=3D@protonmail.com>
[not found] ` <lyGE8gPEf9cUEMJceWoJWD=5Fibk4viZXU0yG5VzbNe9yueGbkcnl1FkJrFZZufhWd5y2vNOgAwfYSpJ4Gia5Tow4wdmQXiGuETdyuNmnemJY=3D@protonmail.com>
[not found] ` <CAJCQCtR3gbJxJn24qyDfHWh9kQG7BSC=3DNnoGHmRKPnaQ+P7yyg@mail.gmail.com>
[not found] ` <8oT9s0Jlzpgp2ctPAXOixSR03oOiPXaitR0AiOkNdBsYHwjPMfjK7CoVAPXuvj71hiUTH-fKoSevAM-To8iSPPBvGRvZeBkU0Nd1=5F NPonyU=3D?= =?us-ascii?Q?@protonmail.com>
[not found] ` <CAJCQCtQH3h=3DNNr6PX3HZp7SbkgqZtNNdihi4aBMFvx+DN79XeA@mail.gmail.com>
[not found] ` <6LDov933WqF3kLH8jtkEh-pfK6pRe0o6-Y9l3NcO2mVhswDL7rhbHyda71OnztoJKfgqqQT9jj1Ba52lz=5FugNFmmRtzN33BlSa5pCvds0F8=3D@protonmail.com>
[not found] ` <CAJCQCtQDt=3Dx7WCX7KhWz=5FpPn4yB1YdZm9jN29jRuQ DFy=3DZTO?= =?us-ascii?Q?jA@mail.gmail.com>
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).