[RFC] refactor mnt_fstype_is_pseudofs()?

[RFC] refactor mnt_fstype_is_pseudofs()?

[Stanislav Brabec]

Hi.

I got a but report about a dumpfs FUSE filesystem reporting bad mount output.

The same problem affects other pseudo file systems not listed in mnt_fstype_is_pseudofs():
# mount | grep encfs
encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
# mkdir encfs
# mount | grep encfs
/root/encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
^^^^^^^^^^^

Debugging shows two reasons why it happens:
1) encfs is not listed in mnt_fstype_is_pseudofs()
2) mnt_resolve_path() calls canonicalize_path_and_cache() on the kernel output, which makes only a little sense

Currently I am aware of three file systems affected at least (i. e. not listed): gvfs (subtype was renamed from gvfs-fuse-daemon to gvfsd-fuse), encfs, dumpfs.

Easy fix would be adding all of them to the list. But probably much more exists.

More complex would allow to add fuse.*. But it is not correct, because FUSE can be a regular file system based on block device (e. g. fuse ntfs). Yes, such reports "fuseblk" as a default. But still, it can be changed.

That is why I started thinking about the better fix than adding these three to the list:

If the column 10 (mount source) of a particular line of /proc/<pid>/mountinfo does not start by "/", then it is a pseudofs. Actually, it could be generalized:
- If it does not contain "/" (or maybe another character) at all, it is a pseudofs.
- If it contains "/" (or maybe another character), but it does not start by it, it is a network file system.
- If it starts by "/", it is a regular file system.

Before implementing, I want to make a wider discussion, whether it could work.

I talked with a kernel developers, and he told that there is no solution "correct by principle". Filesystem developer can provide virtually anything as a source. But it could work in most cases.

What do you think about such change?

And additionally: Does canonicalization of the kernel output make any sense? If not, it could be skipped for commands like a bare "mount". (But I am not sure how complicated it would be.)

-- 
Best Regards / S pozdravem,

Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o.                         e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa)                    tel: +420 284 084 060
186 00 Praha 8-Karlín                          fax:  +420 284 084 001
Czech Republic                                    http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Karel Zak]

On Fri, Feb 01, 2019 at 12:07:12AM +0100, Stanislav Brabec wrote:
> Hi.
> 
> I got a but report about a dumpfs FUSE filesystem reporting bad mount output.
> 
> The same problem affects other pseudo file systems not listed in mnt_fstype_is_pseudofs():
> # mount | grep encfs
> encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
> # mkdir encfs
> # mount | grep encfs
> /root/encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
> ^^^^^^^^^^^
> 
> Debugging shows two reasons why it happens:
> 1) encfs is not listed in mnt_fstype_is_pseudofs()
> 2) mnt_resolve_path() calls canonicalize_path_and_cache() on the kernel output, which makes only a little sense

Where? It should be easy to check by mnt_fs_is_kernel().

> Currently I am aware of three file systems affected at least (i. e. not listed): gvfs (subtype was renamed from gvfs-fuse-daemon to gvfsd-fuse), encfs, dumpfs.
> 
> Easy fix would be adding all of them to the list. But probably much more exists.

It would be nice to have such easy bugfix, so we can backport it to
old version etc. :-)

> More complex would allow to add fuse.*. But it is not correct, because FUSE can be a regular file system based on block device (e. g. fuse ntfs). Yes, such reports "fuseblk" as a default. But still, it can be changed.
> 
> That is why I started thinking about the better fix than adding these three to the list:
> 
> If the column 10 (mount source) of a particular line of /proc/<pid>/mountinfo does not start by "/", then it is a pseudofs. Actually, it could be generalized:

I'm not sure if we always use mnt_fs_is_pseudofs() in situation when
mount source path is available. The libmnt_fs is also able to hold
info about LABEL=foo, etc.

It seems we need to keep mnt_fstype_is_pseudofs() and change
mnt_fs_is_pseudofs() to your suggestion with fallback to
mnt_fstype_is_pseudofs().

> - If it does not contain "/" (or maybe another character) at all, it is a pseudofs.
> - If it contains "/" (or maybe another character), but it does not start by it, it is a network file system.
> - If it starts by "/", it is a regular file system.
> 
> Before implementing, I want to make a wider discussion, whether it could work.
> 
> I talked with a kernel developers, and he told that there is no
> solution "correct by principle". Filesystem developer can provide
> virtually anything as a source. But it could work in most cases.
> 
> What do you think about such change?

If we will keep the fstype lists in the code than we can use it as
the first attempt, if the name is not in the list than use your
heuristic.

> And additionally: Does canonicalization of the kernel output make
> any sense? If not, it could be skipped for commands like a bare
> "mount". (But I am not sure how complicated it would be.)

It depends, kernel canonicalizes mountpoint, but if I good remember it
does not touch source, so if the source is for example symlink and
it's not mounted by mount(8) than whatever is in the kernel output:

 # ln -s /dev/sdc /dev/foo
 # simple_mount /dev/foo /mnt/test ext4 1

 # grep foo /proc/self/mountinfo 
 624 94 8:32 / /mnt/test ro,relatime shared:323 - ext4 /dev/foo ro,stripe=512
                                                       ^^^^^^^^
mount(8) always canonicalizes everything, but in case you use use
something else to call mount(2) syscall...

For mount, findmnt, etc. we make sure (for years) that all is canonical as
it is only way how to keep things consistent. Not sure if we want to
change it and introduce regression in scripts where people use "mount |
grep <devname>".

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Stanislav Brabec]

Karel Zak wrote:
> On Fri, Feb 01, 2019 at 12:07:12AM +0100, Stanislav Brabec wrote:
>> Hi.
>>
>> I got a but report about a dumpfs FUSE filesystem reporting bad mount output.
>>
>> The same problem affects other pseudo file systems not listed in mnt_fstype_is_pseudofs():
>> # mount | grep encfs
>> encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
>> # mkdir encfs
>> # mount | grep encfs
>> /root/encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
>> ^^^^^^^^^^^
>>
>> Debugging shows two reasons why it happens:
>> 1) encfs is not listed in mnt_fstype_is_pseudofs()
>> 2) mnt_resolve_path() calls canonicalize_path_and_cache() on the kernel output, which makes only a little sense
> 
> Where? It should be easy to check by mnt_fs_is_kernel().

This canonicalize_path_and_cache() mangles the mount output.

If fs is not listed as a pseudofs, it tries to canonicalize the string, e. g. "encfs". If a directory called "encfs" exists in the current directory, it is expanded to path of $PWD/encfs.

char *mnt_resolve_path(const char *path, struct libmnt_cache *cache)
{
	char *p = NULL;

	/*DBG(CACHE, ul_debugobj(cache, "resolving path %s", path));*/

	if (!path)
		return NULL;
	if (cache)
		p = (char *) cache_find_path(cache, path);
	if (!p)
->		p = canonicalize_path_and_cache(path, cache);

	return p;
}

>> Currently I am aware of three file systems affected at least (i. e. not listed): gvfs (subtype was renamed from gvfs-fuse-daemon to gvfsd-fuse), encfs, dumpfs.
>>
>> Easy fix would be adding all of them to the list. But probably much more exists.
> 
> It would be nice to have such easy bugfix, so we can backport it to
> old version etc. :-)

OK, I will send it.

> If we will keep the fstype lists in the code than we can use it as
> the first attempt, if the name is not in the list than use your
> heuristic.

It makes sense.

-- 
Best Regards / S pozdravem,

Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o.                         e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa)                    tel: +420 284 084 060
186 00 Praha 8-Karlín                          fax:  +420 284 084 001
Czech Republic                                    http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Karel Zak]

On Mon, Feb 04, 2019 at 05:38:56PM +0100, Stanislav Brabec wrote:
> > If we will keep the fstype lists in the code than we can use it as
> > the first attempt, if the name is not in the list than use your
> > heuristic.
> 
> It makes sense.

Note that source path may be a relative path without any '/'. It's
very unusual, but possible.

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Stanislav Brabec]

Dne 05. 02. 19 v 10:00 Karel Zak napsal(a):
> On Mon, Feb 04, 2019 at 05:38:56PM +0100, Stanislav Brabec wrote:
>>> If we will keep the fstype lists in the code than we can use it as
>>> the first attempt, if the name is not in the list than use your
>>> heuristic.
>>
>> It makes sense.
> 
> Note that source path may be a relative path without any '/'. It's
> very unusual, but possible.

I am aware of the fact that the driver can provide anything. Do we have any examples, where kernel or FUSE driver provides relative source path?

I am just working on a list of all pseudo file systems available in the basic openSUSE repositories. Currently I have ~20 FUSE file systems to inspect.

We also may have two lists: One list of pseudo file systems, second list of obscure pseudo file systems (that don't use relative paths). If the source does not start with "/" and file system is not in one of these lists, mount will return error. I can imagine that we can move this list from code to /etc/libmount/pseudofs.d/.


-- 
Best Regards / S pozdravem,

Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o.                         e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa)                    tel: +420 284 084 060
186 00 Praha 8-Karlín                          fax:  +420 284 084 001
Czech Republic                                    http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Stanislav Brabec]

Dne 04. 02. 19 v 15:03 Karel Zak napsal(a):
> On Fri, Feb 01, 2019 at 12:07:12AM +0100, Stanislav Brabec wrote:
>> The same problem affects other pseudo file systems not listed in mnt_fstype_is_pseudofs():
>> # mount | grep encfs
>> encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
>> # mkdir encfs
>> # mount | grep encfs
>> /root/encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
>> ^^^^^^^^^^^
>>
>> Debugging shows two reasons why it happens:
>> 1) encfs is not listed in mnt_fstype_is_pseudofs()
>> 2) mnt_resolve_path() calls canonicalize_path_and_cache() on the kernel output, which makes only a little sense
> 
> Where? It should be easy to check by mnt_fs_is_kernel().

I just found that the check is incomplete and the same problem also affects ALL network file systems, listed or not listed in mnt_fstype_is_netfs():

# mount nfs.example.com:/home /home-nfs
# mount | grep nfs
nfs.example.com:/home on /home-nfs type nfs4 ...
# mkdir -p nfs.example.com:/home
# mount | grep nfs
/root/nfs.example.com:/home on /home-nfs type nfs4 ...

-- 
Best Regards / S pozdravem,

Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o.                         e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa)                    tel: +420 284 084 060
186 00 Praha 8-Karlín                          fax:  +420 284 084 001
Czech Republic                                    http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Karel Zak]

On Tue, Feb 05, 2019 at 06:09:39PM +0100, Stanislav Brabec wrote:
> Dne 05. 02. 19 v 10:00 Karel Zak napsal(a):
> > On Mon, Feb 04, 2019 at 05:38:56PM +0100, Stanislav Brabec wrote:
> >>> If we will keep the fstype lists in the code than we can use it as
> >>> the first attempt, if the name is not in the list than use your
> >>> heuristic.
> >>
> >> It makes sense.
> > 
> > Note that source path may be a relative path without any '/'. It's
> > very unusual, but possible.
> 
> I am aware of the fact that the driver can provide anything. Do we have any examples, where kernel or FUSE driver provides relative source path?


This is mount(2) syscall feature; for example I have "simple_mount",
just main() with mount(2) call: 

 # cd /dev
 # simple_mount sdc /mnt/test ext4 1

 # grep sdc /proc/self/mountinfo 
 530 94 8:32 / /mnt/test ro,relatime shared:275 - ext4 sdc ro,stripe=512
                                                       ^^^

> I am just working on a list of all pseudo file systems available in
> the basic openSUSE repositories. Currently I have ~20 FUSE file
> systems to inspect.
> 
> We also may have two lists: One list of pseudo file systems, second
> list of obscure pseudo file systems (that don't use relative paths).
> If the source does not start with "/" and file system is not in one
> of these lists, mount will return error. I can imagine that we can
> move this list from code to /etc/libmount/pseudofs.d/.

Some filesystems classification is already in /proc/filesystems. The
problem is that the list is incomplete (contains already loaded
filesystems only).

Maybe we need a generic filesystems classification rather than pseudo-filesystems
specific list. We also keep in the code table to convert magic numbers
to FS name (see mnt_statfs_get_fstype()).

What about /etc/libmount/fs.d/ for arbitrary filesystems and in the files
keep all the hints:

  <name> <type> <magic> 

for example:

  autofs nodev 0x0187
  cifs   netfs 0xff534d42

The another possibility is not call mnt_resolve_path() on mount
output. The problem is that we need mnt_fs_is_pseudofs() on another
places like fstrim.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [RFC] refactor mnt_fstype_is_pseudofs()?

[Karel Zak]

On Tue, Feb 05, 2019 at 11:19:18PM +0100, Stanislav Brabec wrote:
> Dne 04. 02. 19 v 15:03 Karel Zak napsal(a):
> > On Fri, Feb 01, 2019 at 12:07:12AM +0100, Stanislav Brabec wrote:
> >> The same problem affects other pseudo file systems not listed in mnt_fstype_is_pseudofs():
> >> # mount | grep encfs
> >> encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
> >> # mkdir encfs
> >> # mount | grep encfs
> >> /root/encfs on /Encrypted type fuse.encfs (rw,nosuid,nodev,relatime,user_id=10027,group_id=100,default_permissions)
> >> ^^^^^^^^^^^
> >>
> >> Debugging shows two reasons why it happens:
> >> 1) encfs is not listed in mnt_fstype_is_pseudofs()
> >> 2) mnt_resolve_path() calls canonicalize_path_and_cache() on the kernel output, which makes only a little sense
> > 
> > Where? It should be easy to check by mnt_fs_is_kernel().
> 
> I just found that the check is incomplete and the same problem also affects ALL network file systems, listed or not listed in mnt_fstype_is_netfs():

So we need mnt_fstype_is_netfs() there :-) or keep it without the canonicalization.

The canonicalization is only necessary if you do not use libmount/mount(8)
to mount the filesystem. This is probably very unusual use case...

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com