archive mirror
 help / color / mirror / Atom feed
* Deadlock when using FUSE with PID namespaces
@ 2019-03-26 16:51 Sargun Dhillon
  0 siblings, 0 replies; only message in thread
From: Sargun Dhillon @ 2019-03-26 16:51 UTC (permalink / raw)
  To: linux-fsdevel, fuse-devel

We have a FUSE daemon, and users that access them that run in a pid
namespace. When the pid namespace is torn down, we end up in a
situation where in userspace, a user gets stuck. Specifically, they
get stuck in request_wait_answer.

PID: 3655410  TASK: ffff8bc9f87ada00  CPU: 55  COMMAND: "mount.objective"
 #0 [ffffb12820137ca8] __schedule at ffffffffb5aecf8d
 #1 [ffffb12820137d40] schedule at ffffffffb5aed492
 #2 [ffffb12820137d50] request_wait_answer at ffffffffb558ec1a
 #3 [ffffb12820137da8] __fuse_request_send at ffffffffb558ff1f
 #4 [ffffb12820137dc8] fuse_readdir at ffffffffb5594022
 #5 [ffffb12820137e78] iterate_dir at ffffffffb548a1ae
 #6 [ffffb12820137eb8] __x64_sys_getdents at ffffffffb548ad29
 #7 [ffffb12820137f38] do_syscall_64 at ffffffffb5204185
 #8 [ffffb12820137f50] entry_SYSCALL_64_after_hwframe at ffffffffb5c00088
    RIP: 00007fdae2d888eb  RSP: 00007fdadcc9fcf0  RFLAGS: 00000206
    RAX: ffffffffffffffda  RBX: 00007fda840008f0  RCX: 00007fdae2d888eb
    RDX: 0000000000020000  RSI: 00007fda840008f0  RDI: 0000000000000040
    RBP: 00007fda840008f0   R8: 00007fda84000000   R9: 0000000000000001
    R10: 0000000000000000  R11: 0000000000000206  R12: 00007fdadcc9fe20
    R13: 00007fda840008c0  R14: 0000000000000000  R15: 00007fdae3ba7250
    ORIG_RAX: 000000000000004e  CS: 0033  SS: 002b

I have a reproduction here:

Basically, what’s happening is that the same process is the fuse
daemon, and the filesystem client. There is a thread in that FUSE
daemon. let’s call the FUSE daemon P10, and the thread P11.

P10 wires up a FUSE filesystem on /tmp, it opens /dev/fuse with FD5.
P11 enacts an uninterruptible (blocking operation) on /tmp/foo, called
OP1, and /tmp/foo is FD6.
P10 reads the operation (OP1) from /dev/fuse, so now OP1 is in userspace

----The Pid 1 of the namespace is killed--

P10 just terminated, and cannot make progress. It will never respond
to OP1. FD5 and FD6 remain open, because P11 is in uninterruptible
sleep. P11 is in an uninterruptible disk sleep waiting for OP1 to
respond, and the fuse connection never aborts.

FUSE abortion doesn’t kick in because the FD of `/dev/fuse` that P10
originally opened as the FUSE daemon under FD5 will not be closed
until all threads as part of the process are terminated. The mount
namespace wont be torn down until P11 is torn down. P11 will never be
torn down because it’s waiting for someone to do something.

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2019-03-26 16:52 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-26 16:51 Deadlock when using FUSE with PID namespaces Sargun Dhillon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).