Re: FW: Symbolic link with absolute target path in UDF - not working properly?

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Jan Kara]

  Hi,

On Sun 04-12-11 20:43:31, Gábor S. wrote:
> since it did not make through to the linux-fsdevel list for some unknown
> reason, I'm trying your address - according to the Maintainers file, you are
> maintaining the UDF driver for Linux...
  Yes, you chose the right address :)
 
> I recently wanted to archive a bunch of data from my (well, Linux) box
> faithfully, that is keeping not only the hard links but also the symbolic
> links present. I wanted to directly represent the files on a DVD, so I opted
> for creating a disc (image actually) in UDF format instead of creating a tgz
> archive or similar and burn that to a DVD. The next step was to find an
> application to build the UDF image with symbolic links. A short googling led
> me to mkisofs (from cdrtools 3.01) and I created an image containing both
> ISO9660 and UDF file systems with that.
> 
> Then I mounted the image (because I had the premonition that symbolic links
> in UDF are less commonly used) - and got pretty surprised since the symbolic
> links with absolute target path were not working as I expected. For the
> rest, let us assume that the image contains a file called mounttab which was
> linking to /etc/mounttab on the source from which the image was created.
> 
> - On Linux (kernel 3.0.4): the slash (/) at the beginning is omitted. So
> mounttab actually points to etc/mounttab which is not absolute.
> - On FreeBSD (GhostBSD 2.5 BETA 3): the names of the path components are
> simply pasted together without any directory separator (/). So mounttab
> actually points to /etcmounttab which is obviously erroneous.
> 
> What I experienced made me also suspicious that mkisofs might be at fault.
> So I set up another Unix-like OS and mounted the image...
> 
> - On Oracle Solaris (Oracle Solaris 11 Express 2010.11, LiveCD): all the
> symbolic links including the ones with absolute target path work as
> expected.
> 
> To investigate what is going on I had a look into the UDF specification
> (ECMA 167) and the source code of the UDF writer of mkisofs and the UDF file
> system drivers of the above mentioned operating systems. The result is...
> 
> - According to the UDF specification: the target of the symbolic links is
> broken into path components (/, etc, mounttab in the example) and that
> target is basically represented as a sequence of those, where the root
> directory is represented in a special manner, as so called component type 1
> or 2. (While standard path components such as etc and mounttab have a type
> of 5.)
> - mkisofs represents the root directory of the absolute target path as
> component type 2.
  OK. While the meaning of component type 1 is clear to me in the standard
(i.e. if length of the component is 0, component points to the root of UDF
filesystem, if the length is > 0, the component points to a place as agreed
between the creator and consumer of the disk), I have trouble parsing the
meaning of component type 2. In particular the wording is:
"The component specifies the root directory of the directory hierarchy of
which the predecessor of the first component in the pathname is a member."

Some light to this sheds 2/8.7 which says among other things "The
predecessor of the initial component shall be the directory in which the
pathname is described." - i.e. if the symlink exists in foo/bar/ on the
filesystem then "the predecessor of the first component in the pathname"
simply refers to foo/bar/ if I understand it right. In this light, type 2
is really meant to point to the root of the filesystem and 2/8.7.1
seems to confirm this by stating that a resolved pathname should begin with
a component of type 2.

So in this light, type 1 might be intented to point to the root of the
whole filesystem hierarchy while type 2 points just to the root of the
current filesystem. That being said it is kind of hard to implement
symlink pointing to the root of the current filesystem - such thing cannot
be easily expressed in terms of ., .., or /. It is even harder considering
that root of the filesystem need to even be reachable in the current name
space (because of bind mounts and such stuff). Also readlink(2)
implementation is problematic since root of the current filesystem is not
expressible by a simple path.

I'll try to ask for some suggestions in the linux-fsdevel list.

> - Linux expects the root directory of the absolute target path be
> represented as component type 1. Component type 2 is simply ignored.
> - FreeBSD (see
> http://svnweb.freebsd.org/base/release/8.2.0/sys/fs/udf/udf_vnops.c?view=mar
> kup) processes component type 2 (and bails out finding component type 1
> actually). Upon processing the flag root is set to true which is meant (I
> think) to suppress appending a slash after the component name to avoid
> double slashes at the beginning of an absolute target path (which is used to
> separate the component names otherwise). However, the flag root does not get
> cleared because the corresponding code part gets executed only from the
> second path component only.
> - OpenSolaris (see
> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/
> udfs/udf_vnops.c) processes component type 2 as / itself and component type
> 1 as the root directory of the medium itself. (And in the UDF file system
> creator part: the root directory is actually represented as component type 2
> just as mkisofs does.) This is too my interpretation of the standard...
  Hum, as I wrote above I'd understand the standard just the other way
around - i.e. type 2 is root of the medium, type 1 is '/'.

> What do you think? Are Solaris and mkisofs right? Or Linux? FreeBSD's
> implementation of the symbolic links on the UDF file system is difficult to
> explain for sure...

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Jan Kara]

[-- Attachment #1: Type: text/plain, Size: 7224 bytes --]

  Hello,

On Sat 10-12-11 14:03:36, Gábor S. wrote:
> > -----Original Message-----
> > From: Jan Kara [mailto:jack@suse.cz]
> > Sent: Wednesday, December 07, 2011 7:07 PM
> > 
> >   Hi,
> > 
> > On Sun 04-12-11 20:43:31, Gábor S. wrote:
> > > since it did not make through to the linux-fsdevel list for some
> > unknown
> > > reason, I'm trying your address - according to the Maintainers file,
> > you are
> > > maintaining the UDF driver for Linux...
> >   Yes, you chose the right address :)
> > 
> > > I recently wanted to archive a bunch of data from my (well, Linux)
> > box
> > > faithfully, that is keeping not only the hard links but also the
> > symbolic
> > > links present. I wanted to directly represent the files on a DVD, so
> > I opted
> > > for creating a disc (image actually) in UDF format instead of
> > creating a tgz
> > > archive or similar and burn that to a DVD. The next step was to find
> > an
> > > application to build the UDF image with symbolic links. A short
> > googling led
> > > me to mkisofs (from cdrtools 3.01) and I created an image containing
> > both
> > > ISO9660 and UDF file systems with that.
> > >
> > > Then I mounted the image (because I had the premonition that symbolic
> > links
> > > in UDF are less commonly used) - and got pretty surprised since the
> > symbolic
> > > links with absolute target path were not working as I expected. For
> > the
> > > rest, let us assume that the image contains a file called mounttab
> > which was
> > > linking to /etc/mounttab on the source from which the image was
> > created.
> > >
> > > - On Linux (kernel 3.0.4): the slash (/) at the beginning is omitted.
> > So
> > > mounttab actually points to etc/mounttab which is not absolute.
> > > - On FreeBSD (GhostBSD 2.5 BETA 3): the names of the path components
> > are
> > > simply pasted together without any directory separator (/). So
> > mounttab
> > > actually points to /etcmounttab which is obviously erroneous.
> > >
> > > What I experienced made me also suspicious that mkisofs might be at
> > fault.
> > > So I set up another Unix-like OS and mounted the image...
> > >
> > > - On Oracle Solaris (Oracle Solaris 11 Express 2010.11, LiveCD): all
> > the
> > > symbolic links including the ones with absolute target path work as
> > > expected.
> > >
> > > To investigate what is going on I had a look into the UDF
> > specification
> > > (ECMA 167) and the source code of the UDF writer of mkisofs and the
> > UDF file
> > > system drivers of the above mentioned operating systems. The result
> > is...
> > >
> > > - According to the UDF specification: the target of the symbolic
> > links is
> > > broken into path components (/, etc, mounttab in the example) and
> > that
> > > target is basically represented as a sequence of those, where the
> > root
> > > directory is represented in a special manner, as so called component
> > type 1
> > > or 2. (While standard path components such as etc and mounttab have a
> > type
> > > of 5.)
> > > - mkisofs represents the root directory of the absolute target path
> > as
> > > component type 2.
> >   OK. While the meaning of component type 1 is clear to me in the
> > standard
> > (i.e. if length of the component is 0, component points to the root of
> > UDF
> > filesystem, if the length is > 0, the component points to a place as
> > agreed
> > between the creator and consumer of the disk), I have trouble parsing
> > the
> > meaning of component type 2. In particular the wording is:
> > "The component specifies the root directory of the directory hierarchy
> > of
> > which the predecessor of the first component in the pathname is a
> > member."
 > > 
> > Some light to this sheds 2/8.7 which says among other things "The
> > predecessor of the initial component shall be the directory in which
> > the
> > pathname is described." - i.e. if the symlink exists in foo/bar/ on the
> > filesystem then "the predecessor of the first component in the
> > pathname"
> > simply refers to foo/bar/ if I understand it right.
> 
> 
> > In this light, type 2
> > is really meant to point to the root of the filesystem and 2/8.7.1
> > seems to confirm this by stating that a resolved pathname should begin
> > with a component of type 2.
> > 
> > So in this light, type 1 might be intented to point to the root of the
> > whole filesystem hierarchy while type 2 points just to the root of the
> > current filesystem.
> Well, not all resolved pathnames can begin with the root of the current
> filesystem.
> Because there can be an alias (symbolic link) which refers to the root of
> all filesystems.
> So I think type 2 must represent the root of all filesystems.
> Nevertheless this is how mkisofs, Solaris, FreeBSD and as Jörg Schilly
> (author of mkisofs, I have shortly discussed the situation with him as well,
> now he's in the Cc) pointed out also MacOS treat type 2.
  OK, it's certainly easy to make type 2 point to / and it's certainly more
useful than just ignoring it. The patch for this is attached and I'll merge
it with Linus in the next merge window.

> > That being said it is kind of hard to implement
> > symlink pointing to the root of the current filesystem - such thing
> > cannot
> > be easily expressed in terms of ., .., or /. It is even harder
> > considering
> > that root of the filesystem need to even be reachable in the current
> > name
> > space (because of bind mounts and such stuff). Also readlink(2)
> > implementation is problematic since root of the current filesystem is
> > not
> > expressible by a simple path.
> > 
> 
> > I'll try to ask for some suggestions in the linux-fsdevel list.
> I don't think there are many users interested in having symbolic links
> relative to the root of the UDF volume.
  Well, actually for removable media it makes more sense than having links
starting with /. You never know where your filesystem is going to be
mounted or on which system so such links are often going to point to some
non-existent place. For backups it does not matter but if you actually need
to use those symlinks it makes a lot of sense. But relative symlinks are
probably more portable solution in that case...

> I would rather like to have symbolic links relative to the global root. And
> that to be compatible with other OS' appears to me a good idea.
  Yes, I agree with this, especially the 'compatibility' part, because
that's what UDF is mostly about...

> Nonetheless, the link to the source code of OpenSolaris I included in my
> original mail to you might give you some idea how to implement a
> volume-relative absolute symbolic link.
  Yup, I've checked OpenSolaris code and it's nice and easy but doesn't
work on Linux. The problem is that on Linux, a filesystem can have
arbitrarily many mount points (you can mount one filesystem in several
places, you can even mount only some part of a filesystem tree in some
places) - that makes things more complicated but still workable (we
always know through which mountpoint did we come). But what makes the
implementation rather hard is that root of the filesystem need not be even
reachable in the filesystem tree because it is overlayed but some other
mounts...
								Honza

-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

[-- Attachment #2: 0001-udf-Treat-symlink-component-of-type-2-as.patch --]
[-- Type: text/x-patch, Size: 1303 bytes --]

>From c54a637592e92a749a838132fa83ffc145a2dc8b Mon Sep 17 00:00:00 2001
From: Jan Kara <jack@suse.cz>
Date: Mon, 12 Dec 2011 15:13:50 +0100
Subject: [PATCH] udf: Treat symlink component of type 2 as /
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently, we ignore symlink component of type 2. But mkisofs and other OS'
seem to treat it as / so do the same for compatibility.

Reported-by: "Gábor S." <otnaccess@hotmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/udf/symlink.c |   14 ++++++++++----
 1 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/fs/udf/symlink.c b/fs/udf/symlink.c
index b1d4488..d7c6dbe 100644
--- a/fs/udf/symlink.c
+++ b/fs/udf/symlink.c
@@ -41,10 +41,16 @@ static void udf_pc_to_char(struct super_block *sb, unsigned char *from,
 		pc = (struct pathComponent *)(from + elen);
 		switch (pc->componentType) {
 		case 1:
-			if (pc->lengthComponentIdent == 0) {
-				p = to;
-				*p++ = '/';
-			}
+			/*
+			 * Symlink points to some place which should be agreed
+ 			 * upon between originator and receiver of the media. Ignore.
+			 */
+			if (pc->lengthComponentIdent > 0)
+				break;
+			/* Fall through */
+		case 2:
+			p = to;
+			*p++ = '/';
 			break;
 		case 3:
 			memcpy(p, "../", 3);
-- 
1.7.1

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Joerg Schilling]

Gábor S. <otnaccess@hotmail.com> wrote:


> >   Well, actually for removable media it makes more sense than having
> > links starting with /. You never know where your filesystem is going to
> > be mounted or on which system so such links are often going to point to
> > some non-existent place. For backups it does not matter but if you
> > actually need to use those symlinks it makes a lot of sense. But
> > relative symlinks are probably more portable solution in that case...
> I agree with all these points. Although the feature seems to be useful, I
> don't know any other filesystem (to be honest I know the features of a few
> of them only) which supports symbolic links relative to the root of the
> volume. So to create such a UDF volume a special application is needed.
> That's why most people would rather opt for relative symbolic links for such
> a purpose.

This idea has obviously been imported from the older Rock Ridge filesystem that 
uses a very similar method to deal with symbolic links.

Mkisofs currently does not support absolute symlinks relative to the mount 
point but I did recently write a portable replacement for the Solaris 
"resolvepath()" syscall to fit the needs of a project oriented distributed SCCS.
With this function it is possible to check whether a symlink points to 
something that is outside the ISO media. So in future, it would be possible to 
convert absolute symlinks that point to something inside the ISO into a mount 
point based absolute path, as well as to flag symlinks that do not point to an 
object inside the ISO.

And BTW: given the fact that Linux supports programs like "df", I would guess 
that it is possible to do that mapping. Note that unless you do something 
really weird (which would be in conflict with the POSIX standard), you always 
know the vfs structure for a file and a vfs structure of course has an 
associated mount point. Given the fact that in an FS that does not support 
hard links to directories (UDF does not, so you are on the easy side) there is
exactly one way to traverse the directory tree upwards, As a result, there is 
always exactly one mountpoint related to any leaf fs object.

AFAIK, Linux does getcwd() in kernel as Solaris does, so it should be possible 
to get the path for a mount point, in case that path was not kept during the 
mount(2) operation.

Jörg

-- 
 EMail:joerg@schily.net                  (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Jan Kara]

On Tue 13-12-11 12:01:39, Joerg Schilling wrote:
> And BTW: given the fact that Linux supports programs like "df", I would guess 
> that it is possible to do that mapping. Note that unless you do something 
> really weird (which would be in conflict with the POSIX standard), you always 
> know the vfs structure for a file and a vfs structure of course has an 
> associated mount point. Given the fact that in an FS that does not support 
> hard links to directories (UDF does not, so you are on the easy side) there is
> exactly one way to traverse the directory tree upwards, As a result, there is 
> always exactly one mountpoint related to any leaf fs object.
> 
> AFAIK, Linux does getcwd() in kernel as Solaris does, so it should be possible 
> to get the path for a mount point, in case that path was not kept during the 
> mount(2) operation.
  Jorg, maybe I'm missing something but look:
quack:/tmp/foo # l
total 700
drwxr-xr-x  2 root root   4096 2011-12-13 16:54 ./
drwxrwxrwt 74 root root 712704 2011-12-13 16:54 ../
quack:/tmp/foo # mkdir -p bar/baz
quack:/tmp/foo # mkdir bar2
quack:/tmp/foo # mount -t ext3 -o loop ~jack/fs-images/ext3-image bar/baz/
quack:/tmp/foo # mkdir bar/baz/dir
quack:/tmp/foo # touch bar/baz/dir/some_file
quack:/tmp/foo # mount --bind bar/baz/dir/ bar2
quack:/tmp/foo # ls bar2
some_file
quack:/tmp/foo # ls bar/baz/
aquota.group  aquota.user  dir
quack:/tmp/foo # mount -t ext2 -o loop ~jack/fs-images/ext2-image bar/
quack:/tmp/foo # l bar
total 18
drwxr-xr-x 4 10005 users  1024 2011-02-24 12:18 ./
drwxr-xr-x 4 root  root   4096 2011-12-13 16:55 ../
drwx------ 2 root  root  12288 2010-03-10 12:21 lost+found/
drwxrwxrwx 6 root  root   1024 2011-02-24 12:18 t/
quack:/tmp/foo # 

  So to summarize we have directory structure like:
		/tmp
		  |
		 foo
		/   \
	      bar   bar2
	       |     |
here is ext2_image  here is subtree of 'dir' from ext3_image

  And yes, df doesn't work for ext3_image if you do this:
quack:/tmp/foo # df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             24843384   8170984  15410424  35% /
devtmpfs                488488       240    488248   1% /dev
tmpfs                   489924       544    489380   1% /dev/shm
/dev/mapper/cr_sda3   50007768  36910580  10556904  78% /crypted
df: `/tmp/foo/bar/baz': No such file or directory
/dev/loop1               63461     63461         0 100% /tmp/foo/bar

  And yes, if I get into bar/baz and then mount a filesystem over bar,
getcwd returns a path which is not resolvable:
quack:/tmp/foo/bar/baz # pwd
/tmp/foo/bar/baz
quack:/tmp/foo/bar/baz # cd /tmp/foo/bar/baz
bash: cd: /tmp/foo/bar/baz: No such file or directory

  So you can do rather insane stuff with directory tree under linux. Most
admins don't but e.g. container guys do and linux dcache is designed to
accomodate this which makes answering some otherwise simple questions
rather complicated.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Joerg Schilling]

Jan Kara <jack@suse.cz> wrote:

> > AFAIK, Linux does getcwd() in kernel as Solaris does, so it should be possible 
> > to get the path for a mount point, in case that path was not kept during the 
> > mount(2) operation.
>   Jorg, maybe I'm missing something but look:
> quack:/tmp/foo # l
> total 700
> drwxr-xr-x  2 root root   4096 2011-12-13 16:54 ./
> drwxrwxrwt 74 root root 712704 2011-12-13 16:54 ../
> quack:/tmp/foo # mkdir -p bar/baz
> quack:/tmp/foo # mkdir bar2
> quack:/tmp/foo # mount -t ext3 -o loop ~jack/fs-images/ext3-image bar/baz/
> quack:/tmp/foo # mkdir bar/baz/dir
> quack:/tmp/foo # touch bar/baz/dir/some_file
> quack:/tmp/foo # mount --bind bar/baz/dir/ bar2

This looks loks a loopback mount.

Note that I am using the semantically correct word for that beast.
Loopback mounts have been invented in 1986 on SunOS by Sun. 
What Linux calls "loopback mounts" is what I invented in 1988 as
"fbk" driver (file emulates blockdevice).

> quack:/tmp/foo # ls bar2
> some_file
> quack:/tmp/foo # ls bar/baz/
> aquota.group  aquota.user  dir
> quack:/tmp/foo # mount -t ext2 -o loop ~jack/fs-images/ext2-image bar/
> quack:/tmp/foo # l bar
> total 18
> drwxr-xr-x 4 10005 users  1024 2011-02-24 12:18 ./
> drwxr-xr-x 4 root  root   4096 2011-12-13 16:55 ../
> drwx------ 2 root  root  12288 2010-03-10 12:21 lost+found/
> drwxrwxrwx 6 root  root   1024 2011-02-24 12:18 t/
> quack:/tmp/foo # 
>
>   So to summarize we have directory structure like:
> 		/tmp
> 		  |
> 		 foo
> 		/   \
> 	      bar   bar2
> 	       |     |
> here is ext2_image  here is subtree of 'dir' from ext3_image

If this is implemented the way it is implemented in Solaris, there is a
nearly empty vfs layer, specific to the loopback mount that indirectly
calls the vfs functions for the base FS. So on Solaris this is 
possible and it would work as expected. 

See: 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/lofs/lofs_vnops.c#937

Jörg

-- 
 EMail:joerg@schily.net                  (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Jan Kara]

On Tue 13-12-11 19:30:32, Joerg Schilling wrote:
> Jan Kara <jack@suse.cz> wrote:
> 
> > > AFAIK, Linux does getcwd() in kernel as Solaris does, so it should be possible 
> > > to get the path for a mount point, in case that path was not kept during the 
> > > mount(2) operation.
> >   Jorg, maybe I'm missing something but look:
> > quack:/tmp/foo # l
> > total 700
> > drwxr-xr-x  2 root root   4096 2011-12-13 16:54 ./
> > drwxrwxrwt 74 root root 712704 2011-12-13 16:54 ../
> > quack:/tmp/foo # mkdir -p bar/baz
> > quack:/tmp/foo # mkdir bar2
> > quack:/tmp/foo # mount -t ext3 -o loop ~jack/fs-images/ext3-image bar/baz/
> > quack:/tmp/foo # mkdir bar/baz/dir
> > quack:/tmp/foo # touch bar/baz/dir/some_file
> > quack:/tmp/foo # mount --bind bar/baz/dir/ bar2
> 
> This looks loks a loopback mount.
  The first mount is the loopback one, right. But that's not really
interesting. What is interesting is the second mount --bind which allows
you to mount arbitrary *directory* over some other directory in the system.
So you have two (consistent) copies of one directory and it's subtree in the
system.

> > quack:/tmp/foo # ls bar2
> > some_file
> > quack:/tmp/foo # ls bar/baz/
> > aquota.group  aquota.user  dir
> > quack:/tmp/foo # mount -t ext2 -o loop ~jack/fs-images/ext2-image bar/
> > quack:/tmp/foo # l bar
> > total 18
> > drwxr-xr-x 4 10005 users  1024 2011-02-24 12:18 ./
> > drwxr-xr-x 4 root  root   4096 2011-12-13 16:55 ../
> > drwx------ 2 root  root  12288 2010-03-10 12:21 lost+found/
> > drwxrwxrwx 6 root  root   1024 2011-02-24 12:18 t/
> > quack:/tmp/foo # 
> >
> >   So to summarize we have directory structure like:
> > 		/tmp
> > 		  |
> > 		 foo
> > 		/   \
> > 	      bar   bar2
> > 	       |     |
> > here is ext2_image  here is subtree of 'dir' from ext3_image
> 
> If this is implemented the way it is implemented in Solaris, there is a
> nearly empty vfs layer, specific to the loopback mount that indirectly
> calls the vfs functions for the base FS. So on Solaris this is 
> possible and it would work as expected. 
  Again, note that loopback mounts are not the hard part here. I could have
mounted standard block devices and nothing interesting would have changed
in my example. What is interesting is that I can create a directory tree
where root of some filesystem is not accessible (there is not a valid path
that would resolve to it) but it's subdirectory is accessible.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Joerg Schilling]

Jan Kara <jack@suse.cz> wrote:

> On Tue 13-12-11 19:30:32, Joerg Schilling wrote:
> > Jan Kara <jack@suse.cz> wrote:
> > 
> > > > AFAIK, Linux does getcwd() in kernel as Solaris does, so it should be possible 
> > > > to get the path for a mount point, in case that path was not kept during the 
> > > > mount(2) operation.
> > >   Jorg, maybe I'm missing something but look:
> > > quack:/tmp/foo # l
> > > total 700
> > > drwxr-xr-x  2 root root   4096 2011-12-13 16:54 ./
> > > drwxrwxrwt 74 root root 712704 2011-12-13 16:54 ../
> > > quack:/tmp/foo # mkdir -p bar/baz
> > > quack:/tmp/foo # mkdir bar2
> > > quack:/tmp/foo # mount -t ext3 -o loop ~jack/fs-images/ext3-image bar/baz/
> > > quack:/tmp/foo # mkdir bar/baz/dir
> > > quack:/tmp/foo # touch bar/baz/dir/some_file
> > > quack:/tmp/foo # mount --bind bar/baz/dir/ bar2
> > 
> > This looks loks a loopback mount.
>   The first mount is the loopback one, right. But that's not really

No, the first mount is an fbk mount.

> interesting. What is interesting is the second mount --bind which allows
> you to mount arbitrary *directory* over some other directory in the system.
> So you have two (consistent) copies of one directory and it's subtree in the
> system.

As explained before, the second mount is what looks like a loopback mount....


> > If this is implemented the way it is implemented in Solaris, there is a
> > nearly empty vfs layer, specific to the loopback mount that indirectly
> > calls the vfs functions for the base FS. So on Solaris this is 
> > possible and it would work as expected. 
>   Again, note that loopback mounts are not the hard part here. I could have
> mounted standard block devices and nothing interesting would have changed
> in my example. What is interesting is that I can create a directory tree
> where root of some filesystem is not accessible (there is not a valid path
> that would resolve to it) but it's subdirectory is accessible.

Please reread my previous mail. Your problem is that you use the wrong linux 
nomenclature while reading my text.

The pointer to the related "lofs" sourcecode explains how it works and why it 
works as expected on Solaris. 

Note that a mount point relative absolute path needs to be evaluated against 
the real original (first) mount and not against the second loopback mount. 

As mentioned in the previous mail, Solaris would work as expected as the 
loopback (lofs) VFS layer indirects to the original UDF VFS layer that 
will do the real work for the readlink operation.

Jörg

-- 
 EMail:joerg@schily.net                  (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Al Viro]

On Tue, Dec 13, 2011 at 08:25:26PM +0100, Jan Kara wrote:

> > If this is implemented the way it is implemented in Solaris, there is a
> > nearly empty vfs layer, specific to the loopback mount that indirectly
> > calls the vfs functions for the base FS. So on Solaris this is 
> > possible and it would work as expected. 
>   Again, note that loopback mounts are not the hard part here. I could have
> mounted standard block devices and nothing interesting would have changed
> in my example. What is interesting is that I can create a directory tree
> where root of some filesystem is not accessible (there is not a valid path
> that would resolve to it) but it's subdirectory is accessible.

For the sake of completeness:

; mount /dev/sda1 /mnt
; ls /mnt
foo	bar
; mkdir /tmp/splat
; mkdir /tmp/wank
; mount --bind /mnt/foo /tmp/splat
; mount --bind /mnt/bar /tmp/wank
; umount /mnt

and at that point we have fs on sda1 still active, with two subtrees bound
to /tmp/splat and /tmp/wank resp.  Root from /sda1, however, is _not_
available anywhere in your namespace.

As for the implementation, it's light-weight, all right - more so than
what you seem to describe as Solaris one.  We do *not* play with stacking
here; rather than doing that, we have a pair (vfsmount, dentry) used
to describe a point in namespace.  The latter is more or less a counterpart
of vnode; the former is a point in mount tree.  Note that it is *NOT*
a counterpart of e.g. BSD struct mount - that would be struct super_block.
There may be many vfsmounts refering to the same fs; all of them share
struct super_block, etc. - dentry tree is shared, not mirrored.

All IO is done as usual, pretty much ignoring the vfsmount side of things
(modulo things like per-mountpoint read-only/noatime/etc.).  Pathname
resolution acts on (vfsmount,dentry) pair, the first component getting
changed when we hit a mountpoint and when we walk .. from (mnt, mnt->mnt_root).
Process' current directory and root are (vfsmount,dentry) pairs, of course.

Each vfsmount refers to a subtree on some filesystem; those subtrees might
contain one another, etc. - no coherency problems, since the underlying
objects are shared.  Filesystem gets shut down when there's no vfsmounts
left.

Moreover, different processes might have completely unrelated mount trees and
any filesystem may have an arbitrary set of subtrees mounted in any namespaces.
Up to sysadmin...

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Al Viro]

On Tue, Dec 13, 2011 at 08:54:08PM +0100, Joerg Schilling wrote:
> Please reread my previous mail. Your problem is that you use the wrong linux 
> nomenclature while reading my text.
> 
> The pointer to the related "lofs" sourcecode explains how it works and why it 
> works as expected on Solaris. 
> 
> Note that a mount point relative absolute path needs to be evaluated against 
> the real original (first) mount and not against the second loopback mount. 

Except that unlike your lofs, bindings are symmetrical.  I.e. there is no
such thing as "real" mount - they are simply mounts refering to various
subtrees of given fs.  And mount --bind /foo/bar /baz does *not* render /foo
impossible to umount.  Not to mention that even on Solaris there's such
thing as chroot and you really don't want to open that can of worms...

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Joerg Schilling]

Al Viro <viro@ZenIV.linux.org.uk> wrote:

> > The pointer to the related "lofs" sourcecode explains how it works and why it 
> > works as expected on Solaris. 
> > 
> > Note that a mount point relative absolute path needs to be evaluated against 
> > the real original (first) mount and not against the second loopback mount. 
>
> Except that unlike your lofs, bindings are symmetrical.  I.e. there is no
> such thing as "real" mount - they are simply mounts refering to various
> subtrees of given fs.  And mount --bind /foo/bar /baz does *not* render /foo
> impossible to umount.  Not to mention that even on Solaris there's such
> thing as chroot and you really don't want to open that can of worms...

lofs has been introduced with NSE, the first project oriented revision control 
system (based on SCCS). lofs at that time was used to create chrooted tailored 
environments and of course, symlinks are important in such environments.

Absolute symlinks in such an environment either point to a local equivalent or 
they do not point to anything. Both is accepted behavior as this does not 
differ from the non-chrooted case.

Relative symlinks that point outside the chrooted environment are evaluated 
according to the rules for symlinks and for the root directory.

I see no can of worms here.

Let me look at the "binding"... I cannot see any advantage compared to loopback 
mounts. Do you know such an advantage?

If you allow to umount the original mount when there is another 
(non-overlapping) mount, then you of course have a problem with absolute mount 
point related symlinks. Given the fact that the Rock Ridge filesystem is older 
than Linux and that RR introduced mount point related absolute symlinks, isn't 
this bind mount something that could have been avoided?

Well, I have to admit that Solaris does not implement mount point related 
absolute paths in symlinks with Rock Ridge even though it could be done.
Seems like I should write than code ;-)

Jörg

-- 
 EMail:joerg@schily.net                  (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Al Viro]

On Tue, Dec 13, 2011 at 11:41:20PM +0100, Joerg Schilling wrote:

> lofs has been introduced with NSE, the first project oriented revision control 
> system (based on SCCS). lofs at that time was used to create chrooted tailored 
> environments and of course, symlinks are important in such environments.
> 
> Absolute symlinks in such an environment either point to a local equivalent or 
> they do not point to anything. Both is accepted behavior as this does not 
> differ from the non-chrooted case.
> 
> Relative symlinks that point outside the chrooted environment are evaluated 
> according to the rules for symlinks and for the root directory.
> 
> I see no can of worms here.

What happens when you chroot into a _subtree_ of that lofs?  How would
that kind of symlink look like to readlink(2) and what would following
it do to you?

> Let me look at the "binding"... I cannot see any advantage compared to loopback 
> mounts. Do you know such an advantage?

Why, yes - I do.  A bunch of those.
	* lack of coherency issues; we don't get to deal with separate set
of vnodes as Solaris does.
	* our variant is actually lighter, both in terms of memory
consumption and in runtime overhead.
	* it's conceptually simpler - there is (as in any Unix) a forest of
directory trees (one for each active fs) and there is a mount tree that
glues a unified namespace out of their pieces.  Each node in that tree bears
an arbitrary subtree of some active fs.  mount --bind simply creates a new
mount node refering to subtree rooted at given directory (or a non-directory,
while we are at it, in which case we want the mountpoint to be a non-directory
as well).
	* it plays well with namespaces - just make the mount tree
a property of process, just as the descriptor table, cwd/root, address
space, etc., with the only difference being that fork(2) shares that
one instead of copying; clone(2) does allow creating a copy (so does
unshare(2)).  That's a lot cleaner than chroot, BTW, and being able
to put a part of fs into such environment without plopping the whole
tree into it is very convenient.

> point related symlinks. Given the fact that the Rock Ridge filesystem is older 
> than Linux and that RR introduced mount point related absolute symlinks, isn't 
> this bind mount something that could have been avoided?
> 
> Well, I have to admit that Solaris does not implement mount point related 
> absolute paths in symlinks with Rock Ridge even though it could be done.
> Seems like I should write than code ;-)

Seems like somebody in Sun had enough taste to say "no, thanks" to that kind
of kludge, actually, but don't let that stop you - not our kernel, not our
headache and all such...

FWIW, mount --bind is loosely based on Plan 9 bind(8).  There are differences
in semantics (their variant leads to possibility of infinite mount trees,
which is _not_ something gracefully dealt with by existing Unix codebase)
and implementation differs as well, but the basic idea comes from there.

Re: FW: Symbolic link with absolute target path in UDF - not working properly?

[Joerg Schilling]

Al Viro <viro@ZenIV.linux.org.uk> wrote:

> What happens when you chroot into a _subtree_ of that lofs?  How would
> that kind of symlink look like to readlink(2) and what would following
> it do to you?

As mentioned, the readlink would be evaluated in the original filesystem and as 
the result is only a string, the interpretation has to be done later with 
respect to the chroot facts.

> > Let me look at the "binding"... I cannot see any advantage compared to loopback 
> > mounts. Do you know such an advantage?
>
> Why, yes - I do.  A bunch of those.
> 	* lack of coherency issues; we don't get to deal with separate set
> of vnodes as Solaris does.

I see more coherence isssues with the way this "bind" has been explained to me.
The separate set of vnodes creates a clean layering.

> 	* our variant is actually lighter, both in terms of memory
> consumption and in runtime overhead.

I would believe the converse: 

-	a function call does not cause much runtime overhead

-	the memory requirements for the Solaris model seems to lighter than
	something that needs to supply data structures even in case they are 
	not used.

> 	* it's conceptually simpler - there is (as in any Unix) a forest of
> directory trees (one for each active fs) and there is a mount tree that
> glues a unified namespace out of their pieces.  Each node in that tree bears
> an arbitrary subtree of some active fs.  mount --bind simply creates a new
> mount node refering to subtree rooted at given directory (or a non-directory,
> while we are at it, in which case we want the mountpoint to be a non-directory
> as well).

I see no advantage here.

> 	* it plays well with namespaces - just make the mount tree
> a property of process, just as the descriptor table, cwd/root, address
> space, etc., with the only difference being that fork(2) shares that
> one instead of copying; clone(2) does allow creating a copy (so does
> unshare(2)).  That's a lot cleaner than chroot, BTW, and being able
> to put a part of fs into such environment without plopping the whole
> tree into it is very convenient.

It is not about chroot but about a clean name space at fs level and I see a 
cleaner implementation if every mount results in an own related vfs.

> > Well, I have to admit that Solaris does not implement mount point related 
> > absolute paths in symlinks with Rock Ridge even though it could be done.
> > Seems like I should write than code ;-)
>
> Seems like somebody in Sun had enough taste to say "no, thanks" to that kind
> of kludge, actually, but don't let that stop you - not our kernel, not our
> headache and all such...

The Solaris RR implementation has been written in 1988 already and at that 
time, the kernel did not keep path names of mount points as it does now that 
getcwd() is a syscall and procfs2 provides path names for objects.

Nobody touched the path conversion code since then.....

But you remind me of a hsfs/iso9660 problem on Linux that has not been fixed 
since a long time:

Around 2004/2005, someone from FreeBSD discovered that FreeBSD, Linux and 
Solaris all paniced from mounting a iso9660 strange data. FreeBSD has been 
enhanced and together with a friend, I introduced range checks and other 
hardening for the Solaris implementation. While it is no longer possible to 
harm Solaris with a bad iso9660 media, this is still easy with Linux.


> FWIW, mount --bind is loosely based on Plan 9 bind(8).  There are differences
> in semantics (their variant leads to possibility of infinite mount trees,
> which is _not_ something gracefully dealt with by existing Unix codebase)
> and implementation differs as well, but the basic idea comes from there.

From the man page of bind(1) on Plan 9, the related idea is rather something 
like the SunOS translucent filesystem (tfs) that has been introduced together 
with lofs in 1986 for NSE. I cannot see a direct relation between Linux and 
Plan 9 here.

Jörg

-- 
 EMail:joerg@schily.net                  (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html