All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/35 v7] fsnotify: Avoid SRCU stalls with fanotify permission events
@ 2017-04-03 15:33 Jan Kara
  2017-04-03 15:33 ` [PATCH 01/35] fsnotify: Remove unnecessary tests when showing fdinfo Jan Kara
                   ` (34 more replies)
  0 siblings, 35 replies; 43+ messages in thread
From: Jan Kara @ 2017-04-03 15:33 UTC (permalink / raw)
  To: linux-fsdevel; +Cc: Miklos Szeredi, Amir Goldstein, Paul Moore, Jan Kara

Hello,

This is the seventh revision of my patches to avoid SRCU stalls when fanotify
waits for response to permission events from userspace processes. Thanks to
Amir, Paul, and Miklos for review! It also passes a new LTP test that tries to
provoke hangs in fanotify subsystem when there are unanswered fanotify
permission events. If nobody has more objections, I'll push the changes to my
tree to queue them for the next merge window.

Changes since v6:
* Added Reviewed-by tags from Miklos
* Improved couple of comments suggested by Miklos
* Fixed possible NULL pointer dereference in audit_tree
* Cleaned up some patches based on Miklos' feedback

Changes since v5:
* Added Reviewed-by tags from Amir
* Fixed up __rcu annotation
* Fixed minor issues spotted by 0-day in the middle of the series
* Added fsnotify_attach_connector_to_object()
* Removed igrab()/iput() from fsnotify_recalc_mask()

Changes since v4:
* Further split up of patches as requested by Miklos
* Moved some hunks between patches to make things more logical
* Couple of smaller improvements suggested by Miklos
* Rebased on top of 4.11-rc2

Changes since v3:
* added Reviewed-by tags
* split adding of fsnotify_mark_connector into 4 smaller parts as Miklos asked
* simplified API of fsnotify_prepare/finish_user_wait()

Changes since v2:
* added Reviewed-by tags
* dropped fsnotify_put_list() abstraction
* use rcu_assign_pointer() where appropriate

Changes since v1:
* renamed fsnotify_mark_list to fsnotify_mark_connector and couple other
  things
* updated some comments and changelogs to better explain what is going on
* made audit use inode pointer as a key again
* added Reviewed-by tags
* dropped two audit fixes that got already merged
* added cleanup of mark destruction functions

Patch set overview
------------------

Currently, fanotify waits for response to a permission even from userspace
process while holding fsnotify_mark_srcu lock. That has a consequence that
when userspace process takes long to respond or does not respond at all,
fsnotify_mark_srcu period cannot ever complete blocking reclaim of any
notification marks and also blocking any process that did synchronize_srcu()
on fsnotify_mark_srcu. Effectively, this eventually blocks anybody interacting
with the notification subsystem. Miklos has some real world reports of this
happening. Although this in principle a problem of broken userspace
application (which futhermore has to have CAP_SYS_ADMIN in init_user_ns, so
it is not a security problem), it is still nasty that a simple error can
block the kernel like this.

This patch set solves this problem. The basic idea of the solution is that
when fanotify needs to wait for response from userspace process, it grabs
reference to the mark which generated the event and drops fsnotify_mark_srcu
lock. When userspace responds, we grab fsnotify_mark_srcu again, drop
the mark reference, and continue iterating the list of marks attached to the
inode / vfsmount delivering the event to other notification groups. What
complicates this simple approach is that the mark for which we wait for
response has to stay pinned in the list of marks attached to the inode /
vfsmount so that we can resume iteration of the list when userspace responds
but on the other hand when the inode gets unlinked while we wait for userspace
reponse, we need to destroy the mark (or at least detach it from the inode).

The first 5 patches contain some initial fixes and cleanups. Patches 6-17
implement attaching of marks to inode / vfsmount via a dedicated structure
which allows us to detach list of marks from the object without having to
destroy the list itself. Patches 18-20 implement removal of mark from the
list of marks attached to an object when last mark reference is dropped.
Patches 21-24 then implement dropping of SRCU lock when waiting on response
from userspace. Patches 25-33 are mostly trivial cleanups that get rid of
trivial wrappers and one pointer in the mark structure.

Patches have survived testing with inotify/fanotify tests in LTP.

Finally, to ease experimenting with the patches I've pushed them out to
git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git fsnotify

								Honza

^ permalink raw reply	[flat|nested] 43+ messages in thread

end of thread, other threads:[~2017-04-11 20:27 UTC | newest]

Thread overview: 43+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-03 15:33 [PATCH 0/35 v7] fsnotify: Avoid SRCU stalls with fanotify permission events Jan Kara
2017-04-03 15:33 ` [PATCH 01/35] fsnotify: Remove unnecessary tests when showing fdinfo Jan Kara
2017-04-03 15:33 ` [PATCH 02/35] inotify: Remove inode pointers from debug messages Jan Kara
2017-04-03 15:33 ` [PATCH 03/35] fanotify: Move recalculation of inode / vfsmount mask under mark_mutex Jan Kara
2017-04-03 15:33 ` [PATCH 04/35] audit: Abstract hash key handling Jan Kara
2017-04-04 20:38   ` Paul Moore
2017-04-03 15:33 ` [PATCH 05/35] audit_tree: Use mark flags to check whether mark is alive Jan Kara
2017-04-04 20:47   ` Paul Moore
2017-04-05  7:38     ` Jan Kara
2017-04-06 11:51       ` Paul Moore
2017-04-10 15:31         ` Jan Kara
2017-04-10 15:43           ` Jan Kara
2017-04-11 20:27             ` Paul Moore
2017-04-03 15:33 ` [PATCH 06/35] fsnotify: Update comments Jan Kara
2017-04-03 15:33 ` [PATCH 07/35] fsnotify: Move mark list head from object into dedicated structure Jan Kara
2017-04-03 15:33 ` [PATCH 08/35] fsnotify: Move object pointer to fsnotify_mark_connector Jan Kara
2017-04-03 15:33 ` [PATCH 09/35] fsnotify: Make fsnotify_mark_connector hold inode reference Jan Kara
2017-04-03 15:33 ` [PATCH 10/35] fsnotify: Remove indirection from mark list addition Jan Kara
2017-04-03 15:34 ` [PATCH 11/35] fsnotify: Move fsnotify_destroy_marks() Jan Kara
2017-04-03 15:34 ` [PATCH 12/35] fsnotify: Move locking into fsnotify_recalc_mask() Jan Kara
2017-04-03 15:34 ` [PATCH 13/35] fsnotify: Move locking into fsnotify_find_mark() Jan Kara
2017-04-03 15:34 ` [PATCH 14/35] fsnotify: Determine lock in fsnotify_destroy_marks() Jan Kara
2017-04-03 15:34 ` [PATCH 15/35] fsnotify: Remove indirection from fsnotify_detach_mark() Jan Kara
2017-04-03 15:34 ` [PATCH 16/35] fsnotify: Avoid double locking in fsnotify_detach_from_object() Jan Kara
2017-04-03 15:34 ` [PATCH 17/35] fsnotify: Remove useless list deletion and comment Jan Kara
2017-04-03 15:34 ` [PATCH 18/35] fsnotify: Lock object list with connector lock Jan Kara
2017-04-03 15:34 ` [PATCH 19/35] fsnotify: Free fsnotify_mark_connector when there is no mark attached Jan Kara
2017-04-03 15:34 ` [PATCH 20/35] inotify: Do not drop mark reference under idr_lock Jan Kara
2017-04-03 15:34 ` [PATCH 21/35] fsnotify: Move queueing of mark for destruction into fsnotify_put_mark() Jan Kara
2017-04-03 15:34 ` [PATCH 22/35] fsnotify: Detach mark from object list when last reference is dropped Jan Kara
2017-04-03 15:34 ` [PATCH 23/35] fsnotify: Remove special handling of mark destruction on group shutdown Jan Kara
2017-04-03 15:34 ` [PATCH 24/35] fsnotify: Provide framework for dropping SRCU lock in ->handle_event Jan Kara
2017-04-03 15:34 ` [PATCH 25/35] fsnotify: Pass fsnotify_iter_info into handle_event handler Jan Kara
2017-04-03 15:34 ` [PATCH 26/35] fanotify: Release SRCU lock when waiting for userspace response Jan Kara
2017-04-03 15:34 ` [PATCH 27/35] fsnotify: Remove fsnotify_set_mark_{,ignored_}mask_locked() Jan Kara
2017-04-03 15:34 ` [PATCH 28/35] fsnotify: Remove fsnotify_recalc_{inode|vfsmount}_mask() Jan Kara
2017-04-03 15:34 ` [PATCH 29/35] fsnotify: Inline fsnotify_clear_{inode|vfsmount}_mark_group() Jan Kara
2017-04-03 15:34 ` [PATCH 30/35] fsnotify: Rename fsnotify_clear_marks_by_group_flags() Jan Kara
2017-04-03 15:34 ` [PATCH 31/35] fsnotify: Remove fsnotify_detach_group_marks() Jan Kara
2017-04-03 15:34 ` [PATCH 32/35] fsnotify: Remove fsnotify_find_{inode|vfsmount}_mark() Jan Kara
2017-04-03 15:34 ` [PATCH 33/35] fsnotify: Drop inode_mark.c Jan Kara
2017-04-03 15:34 ` [PATCH 34/35] fsnotify: Add group pointer in fsnotify_init_mark() Jan Kara
2017-04-03 15:34 ` [PATCH 35/35] fsnotify: Move ->free_mark callback to fsnotify_ops Jan Kara

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.