From: Michal Hocko <mhocko@kernel.org> To: Jonathan Corbet <corbet@lwn.net> Cc: LKML <linux-kernel@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>, <linux-mm@kvack.org>, Michal Hocko <mhocko@suse.com>, "Darrick J. Wong" <darrick.wong@oracle.com>, David Sterba <dsterba@suse.cz> Subject: [PATCH] doc: document scope NOFS, NOIO APIs Date: Thu, 24 May 2018 13:43:41 +0200 [thread overview] Message-ID: <20180524114341.1101-1-mhocko@kernel.org> (raw) In-Reply-To: <20180424183536.GF30619@thunk.org> From: Michal Hocko <mhocko@suse.com> Although the api is documented in the source code Ted has pointed out that there is no mention in the core-api Documentation and there are people looking there to find answers how to use a specific API. Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: David Sterba <dsterba@suse.cz> Requested-by: "Theodore Y. Ts'o" <tytso@mit.edu> Signed-off-by: Michal Hocko <mhocko@suse.com> --- Hi Johnatan, Ted has proposed this at LSFMM and then we discussed that briefly on the mailing list [1]. I received some useful feedback from Darrick and Dave which has been (hopefully) integrated. Then the thing fall off my radar rediscovering it now when doing some cleanup. Could you take the patch please? [1] http://lkml.kernel.org/r/20180424183536.GF30619@thunk.org .../core-api/gfp_mask-from-fs-io.rst | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/core-api/gfp_mask-from-fs-io.rst diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst new file mode 100644 index 000000000000..e8b2678e959b --- /dev/null +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst @@ -0,0 +1,55 @@ +================================= +GFP masks used from FS/IO context +================================= + +:Date: Mapy, 2018 +:Author: Michal Hocko <mhocko@kernel.org> + +Introduction +============ + +Code paths in the filesystem and IO stacks must be careful when +allocating memory to prevent recursion deadlocks caused by direct +memory reclaim calling back into the FS or IO paths and blocking on +already held resources (e.g. locks - most commonly those used for the +transaction context). + +The traditional way to avoid this deadlock problem is to clear __GFP_FS +resp. __GFP_IO (note the later implies clearing the first as well) in +the gfp mask when calling an allocator. GFP_NOFS resp. GFP_NOIO can be +used as shortcut. It turned out though that above approach has led to +abuses when the restricted gfp mask is used "just in case" without a +deeper consideration which leads to problems because an excessive use +of GFP_NOFS/GFP_NOIO can lead to memory over-reclaim or other memory +reclaim issues. + +New API +======== + +Since 4.12 we do have a generic scope API for both NOFS and NOIO context +``memalloc_nofs_save``, ``memalloc_nofs_restore`` resp. ``memalloc_noio_save``, +``memalloc_noio_restore`` which allow to mark a scope to be a critical +section from the memory reclaim recursion into FS/IO POV. Any allocation +from that scope will inherently drop __GFP_FS resp. __GFP_IO from the given +mask so no memory allocation can recurse back in the FS/IO. + +FS/IO code then simply calls the appropriate save function right at the +layer where a lock taken from the reclaim context (e.g. shrinker) and +the corresponding restore function when the lock is released. All that +ideally along with an explanation what is the reclaim context for easier +maintenance. + +What about __vmalloc(GFP_NOFS) +============================== + +vmalloc doesn't support GFP_NOFS semantic because there are hardcoded +GFP_KERNEL allocations deep inside the allocator which are quite non-trivial +to fix up. That means that calling ``vmalloc`` with GFP_NOFS/GFP_NOIO is +almost always a bug. The good news is that the NOFS/NOIO semantic can be +achieved by the scope api. + +In the ideal world, upper layers should already mark dangerous contexts +and so no special care is required and vmalloc should be called without +any problems. Sometimes if the context is not really clear or there are +layering violations then the recommended way around that is to wrap ``vmalloc`` +by the scope API with a comment explaining the problem. -- 2.17.0
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org> To: Jonathan Corbet <corbet@lwn.net> Cc: LKML <linux-kernel@vger.kernel.org>, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko <mhocko@suse.com>, "Darrick J. Wong" <darrick.wong@oracle.com>, David Sterba <dsterba@suse.cz> Subject: [PATCH] doc: document scope NOFS, NOIO APIs Date: Thu, 24 May 2018 13:43:41 +0200 [thread overview] Message-ID: <20180524114341.1101-1-mhocko@kernel.org> (raw) In-Reply-To: <20180424183536.GF30619@thunk.org> From: Michal Hocko <mhocko@suse.com> Although the api is documented in the source code Ted has pointed out that there is no mention in the core-api Documentation and there are people looking there to find answers how to use a specific API. Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: David Sterba <dsterba@suse.cz> Requested-by: "Theodore Y. Ts'o" <tytso@mit.edu> Signed-off-by: Michal Hocko <mhocko@suse.com> --- Hi Johnatan, Ted has proposed this at LSFMM and then we discussed that briefly on the mailing list [1]. I received some useful feedback from Darrick and Dave which has been (hopefully) integrated. Then the thing fall off my radar rediscovering it now when doing some cleanup. Could you take the patch please? [1] http://lkml.kernel.org/r/20180424183536.GF30619@thunk.org .../core-api/gfp_mask-from-fs-io.rst | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/core-api/gfp_mask-from-fs-io.rst diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst new file mode 100644 index 000000000000..e8b2678e959b --- /dev/null +++ b/Documentation/core-api/gfp_mask-from-fs-io.rst @@ -0,0 +1,55 @@ +================================= +GFP masks used from FS/IO context +================================= + +:Date: Mapy, 2018 +:Author: Michal Hocko <mhocko@kernel.org> + +Introduction +============ + +Code paths in the filesystem and IO stacks must be careful when +allocating memory to prevent recursion deadlocks caused by direct +memory reclaim calling back into the FS or IO paths and blocking on +already held resources (e.g. locks - most commonly those used for the +transaction context). + +The traditional way to avoid this deadlock problem is to clear __GFP_FS +resp. __GFP_IO (note the later implies clearing the first as well) in +the gfp mask when calling an allocator. GFP_NOFS resp. GFP_NOIO can be +used as shortcut. It turned out though that above approach has led to +abuses when the restricted gfp mask is used "just in case" without a +deeper consideration which leads to problems because an excessive use +of GFP_NOFS/GFP_NOIO can lead to memory over-reclaim or other memory +reclaim issues. + +New API +======== + +Since 4.12 we do have a generic scope API for both NOFS and NOIO context +``memalloc_nofs_save``, ``memalloc_nofs_restore`` resp. ``memalloc_noio_save``, +``memalloc_noio_restore`` which allow to mark a scope to be a critical +section from the memory reclaim recursion into FS/IO POV. Any allocation +from that scope will inherently drop __GFP_FS resp. __GFP_IO from the given +mask so no memory allocation can recurse back in the FS/IO. + +FS/IO code then simply calls the appropriate save function right at the +layer where a lock taken from the reclaim context (e.g. shrinker) and +the corresponding restore function when the lock is released. All that +ideally along with an explanation what is the reclaim context for easier +maintenance. + +What about __vmalloc(GFP_NOFS) +============================== + +vmalloc doesn't support GFP_NOFS semantic because there are hardcoded +GFP_KERNEL allocations deep inside the allocator which are quite non-trivial +to fix up. That means that calling ``vmalloc`` with GFP_NOFS/GFP_NOIO is +almost always a bug. The good news is that the NOFS/NOIO semantic can be +achieved by the scope api. + +In the ideal world, upper layers should already mark dangerous contexts +and so no special care is required and vmalloc should be called without +any problems. Sometimes if the context is not really clear or there are +layering violations then the recommended way around that is to wrap ``vmalloc`` +by the scope API with a comment explaining the problem. -- 2.17.0
next prev parent reply other threads:[~2018-05-24 11:43 UTC|newest] Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-04-24 16:27 vmalloc with GFP_NOFS Michal Hocko 2018-04-24 16:27 ` [Cluster-devel] " Michal Hocko 2018-04-24 16:27 ` Michal Hocko 2018-04-24 16:27 ` Michal Hocko 2018-04-24 16:46 ` Mikulas Patocka 2018-04-24 16:46 ` [Cluster-devel] " Mikulas Patocka 2018-04-24 16:46 ` Mikulas Patocka 2018-04-24 16:55 ` Michal Hocko 2018-04-24 16:55 ` [Cluster-devel] " Michal Hocko 2018-04-24 16:55 ` Michal Hocko 2018-04-24 17:05 ` Mikulas Patocka 2018-04-24 17:05 ` [Cluster-devel] " Mikulas Patocka 2018-04-24 17:05 ` Mikulas Patocka 2018-04-24 18:35 ` Theodore Y. Ts'o 2018-04-24 18:35 ` [Cluster-devel] " Theodore Y. Ts'o 2018-04-24 18:35 ` Theodore Y. Ts'o 2018-04-24 19:25 ` Michal Hocko 2018-04-24 19:25 ` [Cluster-devel] " Michal Hocko 2018-04-24 19:25 ` Michal Hocko 2018-05-09 13:42 ` Michal Hocko 2018-05-09 13:42 ` [Cluster-devel] " Michal Hocko 2018-05-09 13:42 ` Michal Hocko 2018-05-09 14:13 ` David Sterba 2018-05-09 14:13 ` [Cluster-devel] " David Sterba 2018-05-09 14:13 ` David Sterba 2018-05-09 15:13 ` Darrick J. Wong 2018-05-09 15:13 ` [Cluster-devel] " Darrick J. Wong 2018-05-09 15:13 ` Darrick J. Wong 2018-05-09 16:24 ` Mike Rapoport 2018-05-09 16:24 ` [Cluster-devel] " Mike Rapoport 2018-05-09 16:24 ` Mike Rapoport 2018-05-09 21:06 ` Michal Hocko 2018-05-09 21:06 ` [Cluster-devel] " Michal Hocko 2018-05-09 21:06 ` Michal Hocko 2018-05-09 21:04 ` Michal Hocko 2018-05-09 21:04 ` [Cluster-devel] " Michal Hocko 2018-05-09 21:04 ` Michal Hocko 2018-05-09 22:02 ` Darrick J. Wong 2018-05-09 22:02 ` [Cluster-devel] " Darrick J. Wong 2018-05-09 22:02 ` Darrick J. Wong 2018-05-10 5:58 ` Michal Hocko 2018-05-10 5:58 ` [Cluster-devel] " Michal Hocko 2018-05-10 5:58 ` Michal Hocko 2018-05-10 7:18 ` Michal Hocko 2018-05-10 7:18 ` [Cluster-devel] " Michal Hocko 2018-05-10 7:18 ` Michal Hocko 2018-05-24 11:43 ` Michal Hocko [this message] 2018-05-24 11:43 ` [PATCH] doc: document scope NOFS, NOIO APIs Michal Hocko 2018-05-24 14:33 ` Shakeel Butt 2018-05-24 14:47 ` Michal Hocko 2018-05-24 16:37 ` Randy Dunlap 2018-05-25 7:52 ` Michal Hocko 2018-05-28 7:21 ` Nikolay Borisov 2018-05-29 8:22 ` Michal Hocko 2018-05-28 11:32 ` Vlastimil Babka 2018-05-24 20:52 ` Jonathan Corbet 2018-05-24 20:52 ` Jonathan Corbet 2018-05-25 8:11 ` Michal Hocko 2018-05-24 22:17 ` Dave Chinner 2018-05-24 23:25 ` Theodore Y. Ts'o 2018-05-25 8:16 ` Michal Hocko 2018-05-27 12:47 ` Mike Rapoport 2018-05-28 9:21 ` Michal Hocko 2018-05-28 16:10 ` Randy Dunlap 2018-05-29 8:21 ` Michal Hocko 2018-05-27 23:48 ` Dave Chinner 2018-05-28 9:19 ` Michal Hocko 2018-05-28 22:32 ` Dave Chinner 2018-05-29 8:18 ` Michal Hocko 2018-05-29 8:26 ` [PATCH v2] " Michal Hocko 2018-05-29 8:26 ` Michal Hocko 2018-05-29 10:22 ` Dave Chinner 2018-05-29 11:50 ` Mike Rapoport 2018-05-29 11:51 ` Jonathan Corbet 2018-05-29 11:51 ` Jonathan Corbet 2018-05-29 12:37 ` Michal Hocko 2018-07-17 12:49 ` vmalloc with GFP_NOFS Michal Hocko 2018-07-17 12:49 ` [Cluster-devel] " Michal Hocko 2018-07-17 12:49 ` Michal Hocko 2018-04-24 19:03 ` Richard Weinberger 2018-04-24 19:03 ` [Cluster-devel] " Richard Weinberger 2018-04-24 19:03 ` Richard Weinberger 2018-04-24 19:28 ` Michal Hocko 2018-04-24 19:28 ` [Cluster-devel] " Michal Hocko 2018-04-24 19:28 ` Michal Hocko 2018-04-24 22:18 ` Richard Weinberger 2018-04-24 22:18 ` [Cluster-devel] " Richard Weinberger 2018-04-24 22:18 ` Richard Weinberger 2018-04-24 23:09 ` Michal Hocko 2018-04-24 23:09 ` [Cluster-devel] " Michal Hocko 2018-04-24 23:09 ` Michal Hocko 2018-04-24 23:17 ` Mikulas Patocka 2018-04-24 23:17 ` [Cluster-devel] " Mikulas Patocka 2018-04-24 23:17 ` Mikulas Patocka 2018-04-24 23:25 ` Michal Hocko 2018-04-24 23:25 ` [Cluster-devel] " Michal Hocko 2018-04-24 23:25 ` Michal Hocko 2018-04-25 12:43 ` Mikulas Patocka 2018-04-25 12:43 ` [Cluster-devel] " Mikulas Patocka 2018-04-25 12:43 ` Mikulas Patocka 2018-04-25 14:45 ` Michal Hocko 2018-04-25 14:45 ` [Cluster-devel] " Michal Hocko 2018-04-25 14:45 ` Michal Hocko 2018-04-25 15:25 ` Mikulas Patocka 2018-04-25 15:25 ` [Cluster-devel] " Mikulas Patocka 2018-04-25 15:25 ` Mikulas Patocka 2018-04-25 16:56 ` Michal Hocko 2018-04-25 16:56 ` [Cluster-devel] " Michal Hocko 2018-04-25 16:56 ` Michal Hocko 2018-07-17 12:47 ` Michal Hocko 2018-07-17 12:47 ` [Cluster-devel] " Michal Hocko 2018-07-17 12:47 ` Michal Hocko 2018-04-24 19:05 ` Richard Weinberger 2018-04-24 19:05 ` [Cluster-devel] " Richard Weinberger 2018-04-24 19:05 ` Richard Weinberger 2018-04-24 19:10 ` Richard Weinberger 2018-04-24 19:10 ` [Cluster-devel] " Richard Weinberger 2018-04-24 19:10 ` Richard Weinberger 2018-04-24 19:26 ` Steven Whitehouse 2018-04-24 19:26 ` [Cluster-devel] " Steven Whitehouse 2018-04-24 19:26 ` Steven Whitehouse 2018-04-24 20:09 ` Michal Hocko 2018-04-24 20:09 ` [Cluster-devel] " Michal Hocko 2018-04-24 20:09 ` Michal Hocko 2018-07-17 12:50 ` Michal Hocko 2018-07-17 12:50 ` [Cluster-devel] " Michal Hocko 2018-07-17 12:50 ` Michal Hocko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180524114341.1101-1-mhocko@kernel.org \ --to=mhocko@kernel.org \ --cc=corbet@lwn.net \ --cc=darrick.wong@oracle.com \ --cc=dsterba@suse.cz \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@suse.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.