Re: [TOPIC LPC] Filesystem Shrink

[Andreas Dilger <adilger@dilger.ca>]

[-- Attachment #1: Type: text/plain, Size: 3255 bytes --]

On Sep 8, 2021, at 1:27 AM, Allison Henderson <allison.henderson@oracle.com> wrote:
> 
> Hi All,
> 
> Earlier this month I had sent out a lpc micro conference proposal for
> file system shrink.  It sounds like the talk is of interest, but folks
> recommended I forward the discussion to fsdevel for more feed back.
> Below is the abstract for the talk:
> 
> 
> File system shrink allows a file system to be reduced in size by some specified size blocks as long as the file system has enough unallocated space to do so.  This operation is currently unsupported in xfs.  Though a file system can be backed up and recreated in smaller sizes, this is not functionally the same as an in place resize.  Implementing this feature is costly in terms of developer time and resources, so it is important to consider the motivations to implement this feature.  This talk would aim to discuss any user stories for this feature.  What are the possible cases for a user needing to shrink the file system after creation, and by how much?  Can these requirements be satisfied with a simpler mkfs option to backup an existing file system into a new but smaller filesystem?  In the cases of creating a rootfs, will a protofile suffice?  If the shrink feature is needed, we should further discuss the APIs that users would need.
> 
> Beyond the user stories, it is also worth discussing implementation challenges.  Reflink and parent pointers can assist in facilitating shrink operations, but is it reasonable to make them requirements for shrink?  Gathering feedback and addressing these challenges will help guide future development efforts for this feature.
> 
> 
> Comments and feedback are appreciated!

This is an issue that has come up occasionally in the past, and more
frequently these days because of virtualization. "Accidental resize"
kind of mistakes, or an installer formatting a huge root filesystem
but wanting to carve off separate filesystems for more robustness
(e.g. so /var/log and /var/tmp don't fill the single root filesystem
and cause the system to fail).

There was some prototype work for a "lazy" online shrink mechanism
for ext4, that essentially just prevented block allocations at the
end of the filesystem.  This required userspace to move any files
and inodes that were beyond the high watermark, and then some time
later either do the shrink offline once the end of the filesystem
was empty, or later enhance the online resize code to remove unused
block groups at the end of the filesystem.  This turns out to be not
as complex as one expects, if the filesystem is already mostly empty,
which is true in the majority of real use cases ("accidental resize",
or "huge root partition" cases).

There is an old a patch available in Patchworks and some discussion
about what would be needed to make it suitable for production use:

https://patchwork.ozlabs.org/project/linux-ext4/patch/9ba7e5de79b8b25e335026d57ec0640fc25e5ce0.1534905460.git.jaco@uls.co.za/

I don't think it would need a huge effort to update that patch and add
the minor changes that are needed to make it really usable (stop inode
allocations beyond the high watermark, add a group remove ioctl, etc.)

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]