Race-free unlinking of directory entries

Race-free unlinking of directory entries

[Pali Rohár]

Hi!

Linux kernel currently does not provide any race-free way for calling
unlink() syscall on file entry which points to opened file descriptor.

On the other hand Linux kernel already provides race-free way for
creating file entry by linkat() syscall with AT_EMPTY_PATH or
AT_SYMLINK_FOLLOW flags. unlinkat() does not.

There was already discussion about unlink issue in bugzilla:
https://bugzilla.kernel.org/show_bug.cgi?id=93441

Because file descriptor describes inode number which can be stored in
more directories as hard links, there is a proposed funlinkat() syscall
with following API:

int funlinkat(int fd, int dirfd, const char *pathname, int flags);

It should atomically check if file descriptor fd and pathname (according
to dirfd) are same, and if then just unlinkat(dirfd, pathname, flags).
If are not same, throw error.

What userspace application basically needs:

Open file, test it stat (or probably content) and based on test result
decide if file needs to be removed or not.

Or delete a file behind a file descriptor opened with O_PATH.

Both cases are currently not possible without introducing race condition
between open/stat and unlink. Between those two calls, some other
process can exchange files.

-- 
Pali Rohár
pali.rohar@gmail.com

Re: Race-free unlinking of directory entries

[Pali Rohár]

Hi! I would like to remind this my older email about race free
unlinking. Is there any plan to provide such support?

On Wednesday 20 December 2017 20:18:44 Pali Rohár wrote:
> Hi!
> 
> Linux kernel currently does not provide any race-free way for calling
> unlink() syscall on file entry which points to opened file descriptor.
> 
> On the other hand Linux kernel already provides race-free way for
> creating file entry by linkat() syscall with AT_EMPTY_PATH or
> AT_SYMLINK_FOLLOW flags. unlinkat() does not.
> 
> There was already discussion about unlink issue in bugzilla:
> https://bugzilla.kernel.org/show_bug.cgi?id=93441
> 
> Because file descriptor describes inode number which can be stored in
> more directories as hard links, there is a proposed funlinkat() syscall
> with following API:
> 
> int funlinkat(int fd, int dirfd, const char *pathname, int flags);
> 
> It should atomically check if file descriptor fd and pathname (according
> to dirfd) are same, and if then just unlinkat(dirfd, pathname, flags).
> If are not same, throw error.
> 
> What userspace application basically needs:
> 
> Open file, test it stat (or probably content) and based on test result
> decide if file needs to be removed or not.
> 
> Or delete a file behind a file descriptor opened with O_PATH.
> 
> Both cases are currently not possible without introducing race condition
> between open/stat and unlink. Between those two calls, some other
> process can exchange files.

-- 
Pali Rohár
pali.rohar@gmail.com

Re: Race-free unlinking of directory entries

[Christoph Hellwig]

On Mon, Apr 09, 2018 at 09:42:41AM +0200, Pali Roh�r wrote:
> Hi! I would like to remind this my older email about race free
> unlinking. Is there any plan to provide such support?

Given that in Linux we know which directory entry a given fd refers to,
we shouldn't even need a new syscall, but simply support the
AT_EMPTY_PATH flag for funlinkat.

Re: Race-free unlinking of directory entries

[Pali Rohár]

On Monday 09 April 2018 02:59:19 Christoph Hellwig wrote:
> On Mon, Apr 09, 2018 at 09:42:41AM +0200, Pali Rohár wrote:
> > Hi! I would like to remind this my older email about race free
> > unlinking. Is there any plan to provide such support?
> 
> Given that in Linux we know which directory entry a given fd refers to,
> we shouldn't even need a new syscall, but simply support the
> AT_EMPTY_PATH flag for funlinkat.

I do not think it is enough, specially in cases when hard links are in
use. Example:

fd = open("/a")
link("/a", "/b")
link("/a", "/c")
unlink("/a")

And now I want to call funlink for "fd". Which file should be unlinked?
"/b" or "/c" or none or both?

Another example:

fd = open("/a")
link("/a", "/b")
unlink("/a")

Calling funlink for fd should unlink "/b" or it should fail?

And another example:

fd = open("/a")
rename("/a", "/b")

What should funlink do for fd now?

Note that there is difference which symlink is shown in /proc/<pid>/fd
between last two examples -- even those two examples results in the same
state (if there is no race). For last example <fd> points to "/b", for
previous <fd> points to "/a (deleted)".

-- 
Pali Rohár
pali.rohar@gmail.com

Re: Race-free unlinking of directory entries

[Christoph Hellwig]

On Mon, Apr 09, 2018 at 12:10:09PM +0200, Pali Roh�r wrote:
> I do not think it is enough, specially in cases when hard links are in
> use. Example:
> 
> fd = open("/a")
> link("/a", "/b")
> link("/a", "/c")
> unlink("/a")
> 
> And now I want to call funlink for "fd". Which file should be unlinked?
> "/b" or "/c" or none or both?

'/a' as that is what fd refers to.

> 
> Another example:
> 
> fd = open("/a")
> link("/a", "/b")
> unlink("/a")
> 
> Calling funlink for fd should unlink "/b" or it should fail?

It should fail, as '/a' doesn't refer to name that is visible in the
namespace.

> And another example:
> 
> fd = open("/a")
> rename("/a", "/b")
> 
> What should funlink do for fd now?

remove the directory entry refering to '/b' as that is what fd refers
to.

> Note that there is difference which symlink is shown in /proc/<pid>/fd
> between last two examples -- even those two examples results in the same
> state (if there is no race). For last example <fd> points to "/b", for
> previous <fd> points to "/a (deleted)".

Yes, that is fully expected.

Re: Race-free unlinking of directory entries

[Pali Rohár]

On Monday 09 April 2018 03:24:14 Christoph Hellwig wrote:
> On Mon, Apr 09, 2018 at 12:10:09PM +0200, Pali Rohár wrote:
> > Another example:
> > 
> > fd = open("/a")
> > link("/a", "/b")
> > unlink("/a")
> > 
> > Calling funlink for fd should unlink "/b" or it should fail?
> 
> It should fail, as '/a' doesn't refer to name that is visible in the
> namespace.
> 
> > And another example:
> > 
> > fd = open("/a")
> > rename("/a", "/b")
> > 
> > What should funlink do for fd now?
> 
> remove the directory entry refering to '/b' as that is what fd refers
> to.

Why it should differ in these two cases? Calling /bin/ln /a /b followed by
/bin/rm /a results in the same state as calling /bin/mv /a /b. This is
something which works in POSIX systems.

I think it is strange that new possible funlink call would work only if
external applications uses /bin/mv and would fail if /bin/ln and /bin/rm
are used.

This is reason why I suggested two parameters funlink, it takes fd for
unlinking and pathname which must contain same inode as fd. So when you
call it with fd+"/b" it unlink "/b" without failing.

-- 
Pali Rohár
pali.rohar@gmail.com

Re: Race-free unlinking of directory entries

[Pali Rohár]

[-- Attachment #1: Type: text/plain, Size: 1938 bytes --]

On Monday 09 April 2018 12:35:54 Pali Rohár wrote:
> On Monday 09 April 2018 03:24:14 Christoph Hellwig wrote:
> > On Mon, Apr 09, 2018 at 12:10:09PM +0200, Pali Rohár wrote:
> > > Another example:
> > > 
> > > fd = open("/a")
> > > link("/a", "/b")
> > > unlink("/a")
> > > 
> > > Calling funlink for fd should unlink "/b" or it should fail?
> > 
> > It should fail, as '/a' doesn't refer to name that is visible in the
> > namespace.
> > 
> > > And another example:
> > > 
> > > fd = open("/a")
> > > rename("/a", "/b")
> > > 
> > > What should funlink do for fd now?
> > 
> > remove the directory entry refering to '/b' as that is what fd refers
> > to.
> 
> Why it should differ in these two cases? Calling /bin/ln /a /b followed by
> /bin/rm /a results in the same state as calling /bin/mv /a /b. This is
> something which works in POSIX systems.
> 
> I think it is strange that new possible funlink call would work only if
> external applications uses /bin/mv and would fail if /bin/ln and /bin/rm
> are used.
> 
> This is reason why I suggested two parameters funlink, it takes fd for
> unlinking and pathname which must contain same inode as fd. So when you
> call it with fd+"/b" it unlink "/b" without failing.

Imagine that you have opened file descriptor in your program and you
want to unlink file behind that file descriptor.

Now some external process rename that file via rename() syscall and
based on your argument unlinking file descriptor in your program pass.

But if that external process do link() syscall followed by unlink()
syscall -- to simulate rename operation -- then based on your argument
unlinking file descriptor in your program fails.

I think this is really inconsistency which is not so good.

...

Has somebody else opinion on this problem?

Because I still think that funlinkat() syscall has its usage.

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]