Re: [Help] Can't login to my systemd-homed user account due to fallocate failure

[Chris Murphy <lists@colorremedies.com>]

On Sun, Aug 23, 2020 at 9:31 AM Andrii Zymohliad
<azymohliad@protonmail.com> wrote:

> My root partition is ~475GiB with BTRFS, my home partition is a ~400GiB LUKS-encrypted partition on a loopback file (also BTRFS) created by systemd-homed (residing at /home/azymohliad.home). Which leaves ~75GiB for the rest of the root FS.
>
> Recently I've lost the ability to log in to that user account, because during authentication systemd does fallocate call for the image. CLI alternative for my case (suggested on systemd mailing list, I don't really know what is it) is:
>
>     fallocate -l 403G -n /home/azymohliad.home
>
> Which fails:
>
>     fallocate: fallocate failed: No space left on device

I don't understand why homectl is fallocating just to activate. I'd
expect homectl only uses fallocate for create and resize. Can you
confirm the exact homectl command you're using?

There is a homectl option that might help, when activating, --luks-discard=true

A consequence of this option, however, is it will make the backing
file sparse. i.e. it'll punch holes in it, freeing up space on the
underlying file system. You should read the warning for the
--allow-discards option in 'man cryptsetup' which is what 'homectl
--luks-discard' is using.

Also, this option will fairly quickly fstrim the user home upon
activation (last time I tested it on systemd-245, some things have
changed in 246). And that will erase all evidence about why you're
having this problem in the first place.

Preferably if you could post the results from:

ls -ls /home/azymohliad.home

Grab this debug file from upstream btrfs-progs
https://github.com/kdave/btrfs-progs/blob/master/btrfs-debugfs

sudo ./btrfs-debugfs -b /

Mount or remount the file system with mount option enospc_debug, and
then reproduce the problem, i.e. exact same homectl command you're
using that's failing with out of space error.


> I can mount my home manually like this:
>
>     losetup -fP /home/azymohliad.home
>     cryptsetup open /dev/loop0p1
>     mount /dev/mapper/home /mnt

Yeah, exactly. Therefore I'll argue this is a systemd-homed bug. If
fallocate fails, systemd-homed can't just lock users out of their user
home. It needs to have a fallback where it doesn't do the fallocate,
and warns the user about whatever dependent operation also can't
happen as a result. But the user home should activate.

> What's interesting to me from above, the partition size on /home/azymohliad.home is 402.72GiB, but the file system size is 256.01GiB, and the image file size is 256.64GiB (from btrfs fi du /home, although ls -lh reports 403GiB). I'm not really sure, but iirc the fs and image sizes were around 403GiB too earlier. Could it be that it somehow got automatically reduced?

I'm confused. But I'm gonna set this aside for now until you post back
with the other information.

> Could I do anything to make that fallocate call (with -l 403G) working? It will allow me to authenticate to homectl and resize the home partition from there.

At the expense of sounding like a smart ass, yes you can make the root
file system bigger (by adding another device) and then presumably the
fallocate will work. But the fallocate seems wrong to me - but I also
don't know what command you're using and why fallocate is even being
used.


>
> If not, what is the safe way to shrink that LUKS-partition size? Maybe then systemd-homed would do fallocate for less space and it would work.
>
> If from my assumptions you could tell that I'm looking in the wrong direction, please give me a hint. Thanks for taking time to read it!

For sure my top advice, is since you can manually mount this sd-homed
home, *freshen your backups now* while you still can. Later versions
of sd-homed create a subvolume on that LUKS Btrfs file system, so you
can snapshot it and use btrfs-send if you want or however else you're
doing backups.


-- 
Chris Murphy