`fsck -A` and fs-specific options

`fsck -A` and fs-specific options

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

ive got some "interesting" bugs here when using the `fsck -A` helper.  
consider file systems that have unique mount options that sometimes the fsck 
program also needs to know about.  for mounting, this isnt a problem as the 
options are stored in /etc/fstab and `mount` will extract the options field 
and pass it along.  but what about fsck ?

for example, some journaling file systems allow the journal to be stored 
separately.  reiserfs has the "jdev=" mount option and the "--journal" fsck 
option.  ext[34] have the "journal_dev=" mount option and the "-j" fsck 
option.

another example is with loop mounts that take an offset.  fsck cannot operate 
on the loop source as the start of the file is not the image.  it needs to 
first setup the loop with the offset, and then do the fsck on the loop point.
	/tmp/foo.img /mnt/tmp ext3 loop,offset=10000

i could code up some crap in the Gentoo init scripts to take care of this, but 
if we handled it in util-linux, everyone would get this for free.  seems like 
we need to add a per-filesystem-type lists that track the mount option (so we 
can extract it from /etc/fstab) and how to translate it into the related fsck 
option.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: `fsck -A` and fs-specific options

[Theodore Tso]

On Jul 11, 2011, at 10:59 PM, Mike Frysinger wrote:
> 
> for example, some journaling file systems allow the journal to be stored 
> separately.  reiserfs has the "jdev=" mount option and the "--journal" fsck 
> option.  ext[34] have the "journal_dev=" mount option and the "-j" fsck 
> option.

At least for ext[34] and external journals, e2fsck can find the external
journal using the blkid library since the UUID of the external journal
is in the superblock.   (In fact that's why the blkid library was originally
written, and why it was originally included as part of e2fsprogs.)

> another example is with loop mounts that take an offset.  fsck cannot operate 
> on the loop source as the start of the file is not the image.  it needs to 
> first setup the loop with the offset, and then do the fsck on the loop point.
> 	/tmp/foo.img /mnt/tmp ext3 loop,offset=10000

How often are people using loopback mounts as a default, standard thing which
need to be mounted and checked as part of the boot sequence?   I'm just
curious what the use case is for this?

I suppose if this was something people really did care about, the /etc/fstab
format could be extended to add a new field at the end for fsck options ---
but it's more complexity, and could break programs try to programmatically
modify /etc/fstab.

-- Ted

Re: `fsck -A` and fs-specific options

[Mike Frysinger]

On Tue, Jul 12, 2011 at 07:02, Theodore Tso wrote:
> On Jul 11, 2011, at 10:59 PM, Mike Frysinger wrote:
>> for example, some journaling file systems allow the journal to be stored
>> separately.  reiserfs has the "jdev=" mount option and the "--journal" fsck
>> option.  ext[34] have the "journal_dev=" mount option and the "-j" fsck
>> option.
>
> At least for ext[34] and external journals, e2fsck can find the external
> journal using the blkid library since the UUID of the external journal
> is in the superblock.   (In fact that's why the blkid library was originally
> written, and why it was originally included as part of e2fsprogs.)

when i read the kernel source, that seemed to be the case.  but i dont
think it's the case for reiserfs, and i think there's a very tiny edge
case with ext[34] related to moving the journal, but probably so small
we can ignore it :p.

>> another example is with loop mounts that take an offset.  fsck cannot operate
>> on the loop source as the start of the file is not the image.  it needs to
>> first setup the loop with the offset, and then do the fsck on the loop point.
>>       /tmp/foo.img /mnt/tmp ext3 loop,offset=10000
>
> How often are people using loopback mounts as a default, standard thing which
> need to be mounted and checked as part of the boot sequence?   I'm just
> curious what the use case is for this?

common enough that i got a report on it ;).  it was a mount point to
be checked/mounted during boot.

> I suppose if this was something people really did care about, the /etc/fstab
> format could be extended to add a new field at the end for fsck options ---
> but it's more complexity, and could break programs try to programmatically
> modify /etc/fstab.

right, which is why i was thinking that parsing the fstab's options
field is the lesser of two evils ...
-mike

Re: `fsck -A` and fs-specific options

[Karel Zak]

On Tue, Jul 12, 2011 at 07:02:20AM -0400, Theodore Tso wrote:
> 
> On Jul 11, 2011, at 10:59 PM, Mike Frysinger wrote:
> > 
> > for example, some journaling file systems allow the journal to be stored 
> > separately.  reiserfs has the "jdev=" mount option and the "--journal" fsck 
> > option.  ext[34] have the "journal_dev=" mount option and the "-j" fsck 
> > option.
> 
> At least for ext[34] and external journals, e2fsck can find the external
> journal using the blkid library since the UUID of the external journal
> is in the superblock. 

 This seems like a very elegant solution. The fsck.<type> helpers have
 to be able to gather all necessary information (from FS superblock,
 /etc/fstab, ...). This is very filesystems specific and I don't think
 that we can resolve this problem in generic /sbin/fsck util. 

> > another example is with loop mounts that take an offset.  fsck cannot operate 
> > on the loop source as the start of the file is not the image.  it needs to 
> > first setup the loop with the offset, and then do the fsck on the loop point.
> > 	/tmp/foo.img /mnt/tmp ext3 loop,offset=10000

 Good point. There is demand for a generic API to assemble block
 devices (dm-crypt, MD, LVM, loopdev, ...). This functionality has
 been requested by desktop guys, dracut, udev and it seems also
 attractive for mount and fsck. I'll probably start to work on this
 task at the end of this year (I hope with DM guys).

 The idea is to have a simple library (libblkasm ?) that provide API
 to assemble a block device according to the configuration in
 /etc/fstab and /etc/blkasm.d/. The library should be modular, so
 subsystem specific modules (lvm.so, crypt.so, ...) will be maintained
 externally by subsystem developers. It seems like a good way how to
 keep the functionality up to date and minimize some communication
 problems between people :-)

 Note that the original idea is from David Zeuthen
 http://people.freedesktop.org/~david/stc-20101011/stc.conf.html, but
 David's goal was daemon.

 My goal is to provide only library and command line interface, so the
 library will be usable everywhere (for example also in some D-BUS
 daemon, dracut, etc.) 
 
    Karel
 
-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: `fsck -A` and fs-specific options

[Theodore Tso]

On Jul 13, 2011, at 4:32 AM, Karel Zak wrote:

> Good point. There is demand for a generic API to assemble block
> devices (dm-crypt, MD, LVM, loopdev, ...). This functionality has
> been requested by desktop guys, dracut, udev and it seems also
> attractive for mount and fsck. I'll probably start to work on this
> task at the end of this year (I hope with DM guys).
> 
> My goal is to provide only library and command line interface, so the
> library will be usable everywhere (for example also in some D-BUS
> daemon, dracut, etc.) 

For many cases it might also be useful to have some way of specfying
that some of these block devices should be automatically specified at
at boot time, and supply an init script that sets up certain block devices
before the fsckall script is run?

What might also be cool is some way of specifying that certain block
devices should only be created after the user logs on, and then there
would be a PAM module that would take care of using the user's
login password and passing it to LUKS.

Some of thise might be done in plugins, and not your core library,
but I think these are two important use cases to keep in mind.

-- Ted

Re: `fsck -A` and fs-specific options

[Theodore Tso]

On Jul 12, 2011, at 3:18 PM, Mike Frysinger wrote:

> when i read the kernel source, that seemed to be the case.  but i dont
> think it's the case for reiserfs, and i think there's a very tiny edge
> case with ext[34] related to moving the journal, but probably so small
> we can ignore it :p.

What do you think the edge case is?   Yes, the external journal's
dev_t is cached, but that is only used by the kernel since it
was simpler than having callouts to a userspace daemon.

E2fsck will use blkid to locate the external journal, and when it
finds it, it will update the cached dev_t if it is out of date, so that
at mount time the kernel will be able to find the external journal.
This way even if the external journal is on a device with an 
unstable dev_t (e.g., a USB device), the right thing should 
happen...

-- Ted

Fake block devices (Was Re: `fsck -A` and fs-specific options)

[David Zeuthen]

Hey,

On Wed, 2011-07-13 at 10:32 +0200, Karel Zak wrote:
>  Good point. There is demand for a generic API to assemble block
>  devices (dm-crypt, MD, LVM, loopdev, ...). This functionality has
>  been requested by desktop guys, dracut, udev and it seems also
>  attractive for mount and fsck. I'll probably start to work on this
>  task at the end of this year (I hope with DM guys).
> 
>  The idea is to have a simple library (libblkasm ?) that provide API
>  to assemble a block device according to the configuration in
>  /etc/fstab and /etc/blkasm.d/. The library should be modular, so
>  subsystem specific modules (lvm.so, crypt.so, ...) will be maintained
>  externally by subsystem developers. It seems like a good way how to
>  keep the functionality up to date and minimize some communication
>  problems between people :-)
> 
>  Note that the original idea is from David Zeuthen
>  http://people.freedesktop.org/~david/stc-20101011/stc.conf.html, but
>  David's goal was daemon.

FWIW, I've changed my mind about this. Basically, I don't think it's
worth supporting fake block devices (except for dm-luks and maybe loop
devices). Specifically, I will not support it in the next major version
of GNOME Disk Utility (aka Palimpsest) except for showing the "friendly"
dm name (e.g. /dev/mapper/blah) instead of /dev/dm-0. Here's a
work-in-progress screenshot

http://people.freedesktop.org/~david/palimpsest-with-fake-block-devices.png

(Compare to: http://people.freedesktop.org/~david/nautilus-lvm2-b.png )

Anyway, I think it would be better if people instead worked on e.g.
btrfs and making we properly support multi-disk in btrfs... because
btrfs multi-disk is subject to exactly the same problems as you have
when activating RAID or LVM devices (except that you can't do arbitrary
trees - which is a good thing!). For example, there's a policy decision
when to start a device in degraded mode. And, IIRC, the kernel don't
even properly convey what underlying devices a btrfs fs is currently
using  (it didn't last time I checked which is ~12 months ago).

    David

Re: `fsck -A` and fs-specific options

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 993 bytes --]

On Wednesday, July 13, 2011 07:17:27 Theodore Tso wrote:
> On Jul 12, 2011, at 3:18 PM, Mike Frysinger wrote:
> > when i read the kernel source, that seemed to be the case.  but i dont
> > think it's the case for reiserfs, and i think there's a very tiny edge
> > case with ext[34] related to moving the journal, but probably so small
> > we can ignore it :p.
> 
> What do you think the edge case is?   Yes, the external journal's
> dev_t is cached, but that is only used by the kernel since it
> was simpler than having callouts to a userspace daemon.
> 
> E2fsck will use blkid to locate the external journal, and when it
> finds it, it will update the cached dev_t if it is out of date, so that
> at mount time the kernel will be able to find the external journal.
> This way even if the external journal is on a device with an
> unstable dev_t (e.g., a USB device), the right thing should
> happen...

that sounds like it covers all the non-pathological cases ;)
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]