Re: Seed device is broken, again.

[Omar Sandoval <osandov@osandov.com>]

On Fri, Feb 25, 2022 at 09:36:32PM +0800, Qu Wenruo wrote:
> 
> 
> On 2022/2/25 19:47, David Sterba wrote:
> > On Fri, Feb 25, 2022 at 06:08:20PM +0800, Qu Wenruo wrote:
> > > Hi,
> > > 
> > > The very basic seed device usage is broken again:
> > > 
> > > 	mkfs.btrfs -f $dev1 > /dev/null
> > > 	btrfstune -S 1 $dev1
> > > 	mount $dev1 $mnt
> > > 	btrfs dev add $dev2 $mnt
> > > 	umount $mnt
> > > 
> > > 
> > > I'm not sure how many guys are really using seed device.
> > > 
> > > But I see a lot of weird operations, like calling a definite write
> > > operation (device add) on a RO mounted fs.
> > 
> > That's how the seeding device works, in what way would you want to do
> > the ro->rw change?
> 
> In progs-only, so that in kernel we can make dev add ioctl as a real RW
> operation, not adding an exception for dev add.
> 
> > 
> > > Can we make at least the seed sprouting part into btrfs-progs instead?
> > 
> > How? What do you mean? This is an in kernel operation that is done on a
> > mounted filesystem, progs can't do that.
> 
> Why not?
> 
> Progs has the same ability to read-write the fs, I see no reason why we
> shouldn't do it in user space.
> 
> We're just adding a device, update related trees (which will only be
> written to the new device). I see no special reason why it can't be done
> in user space.
> 
> Furthermore, the ability to add device in user space can be a good
> safenet for some ENOSPC space.
> 
> > 
> > > And can seed device even support the upcoming extent-tree-v2?
> > 
> > I should, it operates on the device level.
> > 
> > > Personally speaking I prefer to mark seed device deprecated completely.
> > 
> > If we did that with every feature some developer does not like we'd have
> > nothing left you know. Seeding is a documented usecase, as long as it
> > works it's fine to have it available.
> 
> A documented usecase doesn't mean it can't be deprecated.
> 
> Furthermore, a documented use case doesn't validate the use case itself.
> 
> So, please tell me when did you use seed device last time?
> Or anyone in the mail list, please tell me some real world usecase for
> seed devices.
> 
> I did remember some planned use case like a way to use seed device as a
> VM/container template, but no, it doesn't get much attention.
> 
> 
> I'm not asking for deprecate the feature just because I don't like it.
> 
> This feature is asking for too many exceptions, from the extra
> fs_devices hack (we have in fact two fs_devices, one for rw devices, one
> for seed only), to the dev add ioctl.
> 
> But the little benefit is not really worthy for the cost.

We use seed devices in production. The use case is for servers where we
don't want to persist any sensitive data. The seed device contains a
basic boot environment, then we sprout it with a dm-crypt device and
throw away the encryption key. This ensures that nothing sensitive can
ever be recovered. We previously did this with overlayfs, but seed
devices ended up working better for reasons I don't remember.

Davide Cavalca also told me that "Fedora is also interested in
leveraging seed devices down the road. e.g. doing seed/sprout for cloud
provisioning, and using seed devices for OEM factory recovery on
laptops."

There are definitely hacks we need to fix for seed devices, but there
are valid use cases and we can't just deprecate it.