Deadlock (un-killable processes) in sys_futex

Deadlock (un-killable processes) in sys_futex

[Robert Święcki]

After fuzzing the linux kernel (3.12-rc4) I have two processes which
are stuck in an un-killable state. This is not specific to 3.12-rc4,
as I'm able to reproduce it on most modern kernels (e.g. Ubuntu's 3.5)
after a few minutes of fuzzing with a syscall fuzzer.

The debug data can be found here: http://alt.swiecki.net/linux/20327/
- process PIDs: 20327 and 13735

It includes..

ftrace report (probably the most useful):
I'm not expert in this kernel area (futex/mm), but it seems like a
constatnt loop between fault_in_user_writeable() and do_page_fault():
http://alt.swiecki.net/linux/20327/20327.trace.report.txt

/proc/pid/maps, /proc/pid/status:
http://alt.swiecki.net/linux/20327/20327.maps.txt
http://alt.swiecki.net/linux/20327/20327.status.txt

kdb stacktraces showing that both processes (single-threaded) are
stuck in sys_futex:
http://alt.swiecki.net/linux/20327/20327.kdb.txt
http://alt.swiecki.net/linux/20327/13735.kdb.txt

kgdb stacktraces displaying rather corrupted data:
http://alt.swiecki.net/linux/20327/20327.kgdb.txt
http://alt.swiecki.net/linux/20327/13735.kgdb.txt

kernel conf:
http://alt.swiecki.net/linux/20327/config-3.12-rc4.txt

-- 
Robert Święcki