* [PATCH v3 1/3] docs: core-api/gfp_mask-from-fs-io: add a label for cross-referencing
2018-08-17 14:47 [PATCH v3 0/3] docs/core-api: add memory allocation guide Mike Rapoport
@ 2018-08-17 14:47 ` Mike Rapoport
2018-08-17 14:47 ` [PATCH v3 2/3] docs: core-api/mm-api: add a lable for GFP flags section Mike Rapoport
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Mike Rapoport @ 2018-08-17 14:47 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel, Mike Rapoport
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
Documentation/core-api/gfp_mask-from-fs-io.rst | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/core-api/gfp_mask-from-fs-io.rst b/Documentation/core-api/gfp_mask-from-fs-io.rst
index e0df8f4..e7c32a8 100644
--- a/Documentation/core-api/gfp_mask-from-fs-io.rst
+++ b/Documentation/core-api/gfp_mask-from-fs-io.rst
@@ -1,3 +1,5 @@
+.. _gfp_mask_from_fs_io:
+
=================================
GFP masks used from FS/IO context
=================================
--
2.7.4
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 2/3] docs: core-api/mm-api: add a lable for GFP flags section
2018-08-17 14:47 [PATCH v3 0/3] docs/core-api: add memory allocation guide Mike Rapoport
2018-08-17 14:47 ` [PATCH v3 1/3] docs: core-api/gfp_mask-from-fs-io: add a label for cross-referencing Mike Rapoport
@ 2018-08-17 14:47 ` Mike Rapoport
2018-08-17 14:47 ` [PATCH v3 3/3] docs: core-api: add memory allocation guide Mike Rapoport
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Mike Rapoport @ 2018-08-17 14:47 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel, Mike Rapoport
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
Documentation/core-api/mm-api.rst | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/core-api/mm-api.rst b/Documentation/core-api/mm-api.rst
index 46ae353..5ce1ec1 100644
--- a/Documentation/core-api/mm-api.rst
+++ b/Documentation/core-api/mm-api.rst
@@ -14,6 +14,8 @@ User Space Memory Access
.. kernel-doc:: mm/util.c
:functions: get_user_pages_fast
+.. _mm-api-gfp-flags:
+
Memory Allocation Controls
==========================
--
2.7.4
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 3/3] docs: core-api: add memory allocation guide
2018-08-17 14:47 [PATCH v3 0/3] docs/core-api: add memory allocation guide Mike Rapoport
2018-08-17 14:47 ` [PATCH v3 1/3] docs: core-api/gfp_mask-from-fs-io: add a label for cross-referencing Mike Rapoport
2018-08-17 14:47 ` [PATCH v3 2/3] docs: core-api/mm-api: add a lable for GFP flags section Mike Rapoport
@ 2018-08-17 14:47 ` Mike Rapoport
2018-09-11 17:55 ` Jonathan Corbet
2018-09-03 5:12 ` [PATCH v3 0/3] docs/core-api: " Mike Rapoport
2018-09-11 16:24 ` Mike Rapoport
4 siblings, 1 reply; 9+ messages in thread
From: Mike Rapoport @ 2018-08-17 14:47 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel, Mike Rapoport
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
---
Documentation/core-api/index.rst | 1 +
Documentation/core-api/memory-allocation.rst | 124 +++++++++++++++++++++++++++
2 files changed, 125 insertions(+)
create mode 100644 Documentation/core-api/memory-allocation.rst
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index cdc2020..8afc0da 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -27,6 +27,7 @@ Core utilities
errseq
printk-formats
circular-buffers
+ memory-allocation
mm-api
gfp_mask-from-fs-io
timekeeping
diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
new file mode 100644
index 0000000..3c56543
--- /dev/null
+++ b/Documentation/core-api/memory-allocation.rst
@@ -0,0 +1,124 @@
+=======================
+Memory Allocation Guide
+=======================
+
+Linux provides a variety of APIs for memory allocation. You can
+allocate small chunks using `kmalloc` or `kmem_cache_alloc` families,
+large virtually contiguous areas using `vmalloc` and its derivatives,
+or you can directly request pages from the page allocator with
+`alloc_pages`. It is also possible to use more specialized allocators,
+for instance `cma_alloc` or `zs_malloc`.
+
+Most of the memory allocation APIs use GFP flags to express how that
+memory should be allocated. The GFP acronym stands for "get free
+pages", the underlying memory allocation function.
+
+Diversity of the allocation APIs combined with the numerous GFP flags
+makes the question "How should I allocate memory?" not that easy to
+answer, although very likely you should use
+
+::
+
+ kzalloc(<size>, GFP_KERNEL);
+
+Of course there are cases when other allocation APIs and different GFP
+flags must be used.
+
+Get Free Page flags
+===================
+
+The GFP flags control the allocators behavior. They tell what memory
+zones can be used, how hard the allocator should try to find free
+memory, whether the memory can be accessed by the userspace etc. The
+:ref:`Documentation/core-api/mm-api.rst <mm-api-gfp-flags>` provides
+reference documentation for the GFP flags and their combinations and
+here we briefly outline their recommended usage:
+
+ * Most of the time ``GFP_KERNEL`` is what you need. Memory for the
+ kernel data structures, DMAable memory, inode cache, all these and
+ many other allocations types can use ``GFP_KERNEL``. Note, that
+ using ``GFP_KERNEL`` implies ``GFP_RECLAIM``, which means that
+ direct reclaim may be triggered under memory pressure; the calling
+ context must be allowed to sleep.
+ * If the allocation is performed from an atomic context, e.g interrupt
+ handler, use ``GFP_NOWAIT``. This flag prevents direct reclaim and
+ IO or filesystem operations. Consequently, under memory pressure
+ ``GFP_NOWAIT`` allocation is likely to fail. Allocations which
+ have a reasonable fallback should be using ``GFP_NOWARN``.
+ * If you think that accessing memory reserves is justified and the kernel
+ will be stressed unless allocation succeeds, you may use ``GFP_ATOMIC``.
+ * Untrusted allocations triggered from userspace should be a subject
+ of kmem accounting and must have ``__GFP_ACCOUNT`` bit set. There
+ is the handy ``GFP_KERNEL_ACCOUNT`` shortcut for ``GFP_KERNEL``
+ allocations that should be accounted.
+ * Userspace allocations should use either of the ``GFP_USER``,
+ ``GFP_HIGHUSER`` or ``GFP_HIGHUSER_MOVABLE`` flags. The longer
+ the flag name the less restrictive it is.
+
+ ``GFP_HIGHUSER_MOVABLE`` does not require that allocated memory
+ will be directly accessible by the kernel or the hardware and
+ implies that the data is movable.
+
+ ``GFP_HIGHUSER`` means that the allocated memory is not movable,
+ but it is not required to be directly accessible by the kernel or
+ the hardware. An example may be a hardware allocation that maps
+ data directly into userspace but has no addressing limitations.
+
+ ``GFP_USER`` means that the allocated memory is not movable and it
+ must be directly accessible by the kernel or the hardware. It is
+ typically used by hardware for buffers that are mapped to
+ userspace (e.g. graphics) that hardware still must DMA to.
+
+You may notice that quite a few allocations in the existing code
+specify ``GFP_NOIO`` or ``GFP_NOFS``. Historically, they were used to
+prevent recursion deadlocks caused by direct memory reclaim calling
+back into the FS or IO paths and blocking on already held
+resources. Since 4.12 the preferred way to address this issue is to
+use new scope APIs described in
+:ref:`Documentation/core-api/gfp_mask-from-fs-io.rst <gfp_mask_from_fs_io>`.
+
+Other legacy GFP flags are ``GFP_DMA`` and ``GFP_DMA32``. They are
+used to ensure that the allocated memory is accessible by hardware
+with limited addressing capabilities. So unless you are writing a
+driver for a device with such restrictions, avoid using these flags.
+And even with hardware with restrictions it is preferable to use
+`dma_alloc*` APIs.
+
+Selecting memory allocator
+==========================
+
+The most straightforward way to allocate memory is to use a function
+from the :c:func:`kmalloc` family. And, to be on the safe size it's
+best to use routines that set memory to zero, like
+:c:func:`kzalloc`. If you need to allocate memory for an array, there
+are :c:func:`kmalloc_array` and :c:func:`kcalloc` helpers.
+
+The maximal size of a chunk that can be allocated with `kmalloc` is
+limited. The actual limit depends on the hardware and the kernel
+configuration, but it is a good practice to use `kmalloc` for objects
+smaller than page size.
+
+For large allocations you can use :c:func:`vmalloc` and
+:c:func:`vzalloc`, or directly request pages from the page
+allocator. The memory allocated by `vmalloc` and related functions is
+not physically contiguous.
+
+If you are not sure whether the allocation size is too large for
+`kmalloc`, it is possible to use :c:func:`kvmalloc` and its
+derivatives. It will try to allocate memory with `kmalloc` and if the
+allocation fails it will be retried with `vmalloc`. There are
+restrictions on which GFP flags can be used with `kvmalloc`; please
+see :c:func:`kvmalloc_node` reference documentation. Note that
+`kvmalloc` may return memory that is not physically contiguous.
+
+If you need to allocate many identical objects you can use the slab
+cache allocator. The cache should be set up with
+:c:func:`kmem_cache_create` before it can be used. Afterwards
+:c:func:`kmem_cache_alloc` and its convenience wrappers can allocate
+memory from that cache.
+
+When the allocated memory is no longer needed it must be freed. You
+can use :c:func:`kvfree` for the memory allocated with `kmalloc`,
+`vmalloc` and `kvmalloc`. The slab caches should be freed with
+:c:func:`kmem_cache_free`. And don't forget to destroy the cache with
+:c:func:`kmem_cache_destroy`.
--
2.7.4
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v3 3/3] docs: core-api: add memory allocation guide
2018-08-17 14:47 ` [PATCH v3 3/3] docs: core-api: add memory allocation guide Mike Rapoport
@ 2018-09-11 17:55 ` Jonathan Corbet
2018-09-12 10:33 ` Mike Rapoport
0 siblings, 1 reply; 9+ messages in thread
From: Jonathan Corbet @ 2018-09-11 17:55 UTC (permalink / raw)
To: Mike Rapoport
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel
Sorry for being so slow to get to this...it fell into a dark crack in my
rickety email folder hierarchy. I do have one question...
On Fri, 17 Aug 2018 17:47:16 +0300
Mike Rapoport <rppt@linux.vnet.ibm.com> wrote:
> + ``GFP_HIGHUSER_MOVABLE`` does not require that allocated memory
> + will be directly accessible by the kernel or the hardware and
> + implies that the data is movable.
> +
> + ``GFP_HIGHUSER`` means that the allocated memory is not movable,
> + but it is not required to be directly accessible by the kernel or
> + the hardware. An example may be a hardware allocation that maps
> + data directly into userspace but has no addressing limitations.
> +
> + ``GFP_USER`` means that the allocated memory is not movable and it
> + must be directly accessible by the kernel or the hardware. It is
> + typically used by hardware for buffers that are mapped to
> + userspace (e.g. graphics) that hardware still must DMA to.
I realize that this is copied from elsewhere, but still...as I understand
it, the "HIGH" part means that the allocation can be satisfied from high
memory, nothing more. So...it's irrelevant on 64-bit machines to start
with, right? And it has nothing to do with DMA, I would think. That would
be handled by the DMA infrastructure and, perhaps, the DMA* zones. Right?
I ask because high memory is an artifact of how things are laid out on
32-bit systems; hardware can often DMA quite easily into memory that the
kernel sees as "high". So, to me, this description seems kind of
confusing; I wouldn't mention hardware at all. But maybe I'm missing
something?
Thanks,
jon
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 3/3] docs: core-api: add memory allocation guide
2018-09-11 17:55 ` Jonathan Corbet
@ 2018-09-12 10:33 ` Mike Rapoport
2018-09-13 22:41 ` Jonathan Corbet
0 siblings, 1 reply; 9+ messages in thread
From: Mike Rapoport @ 2018-09-12 10:33 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel
On Tue, Sep 11, 2018 at 11:55:55AM -0600, Jonathan Corbet wrote:
> Sorry for being so slow to get to this...it fell into a dark crack in my
> rickety email folder hierarchy. I do have one question...
>
> On Fri, 17 Aug 2018 17:47:16 +0300
> Mike Rapoport <rppt@linux.vnet.ibm.com> wrote:
>
> > + ``GFP_HIGHUSER_MOVABLE`` does not require that allocated memory
> > + will be directly accessible by the kernel or the hardware and
> > + implies that the data is movable.
> > +
> > + ``GFP_HIGHUSER`` means that the allocated memory is not movable,
> > + but it is not required to be directly accessible by the kernel or
> > + the hardware. An example may be a hardware allocation that maps
> > + data directly into userspace but has no addressing limitations.
> > +
> > + ``GFP_USER`` means that the allocated memory is not movable and it
> > + must be directly accessible by the kernel or the hardware. It is
> > + typically used by hardware for buffers that are mapped to
> > + userspace (e.g. graphics) that hardware still must DMA to.
>
> I realize that this is copied from elsewhere, but still...as I understand
> it, the "HIGH" part means that the allocation can be satisfied from high
> memory, nothing more. So...it's irrelevant on 64-bit machines to start
> with, right? And it has nothing to do with DMA, I would think. That would
> be handled by the DMA infrastructure and, perhaps, the DMA* zones. Right?
>
> I ask because high memory is an artifact of how things are laid out on
> 32-bit systems; hardware can often DMA quite easily into memory that the
> kernel sees as "high". So, to me, this description seems kind of
> confusing; I wouldn't mention hardware at all. But maybe I'm missing
> something?
Well, I've amended the original text from gfp.h in attempt to make it more
"user friendly". The GFP_HIGHUSER became really confusing :)
I think that we can drop mentions of hardware from GFP_HIGHUSER_MOVABLE and
GFP_USER, but it makes sense to leave the example in the GFP_HIGHUSER
description.
How about:
``GFP_HIGHUSER_MOVABLE`` does not require that allocated memory
will be directly accessible by the kernel and implies that the
data is movable.
``GFP_HIGHUSER`` means that the allocated memory is not movable,
but it is not required to be directly accessible by the kernel. An
example may be a hardware allocation that maps data directly into
userspace but has no addressing limitations.
``GFP_USER`` means that the allocated memory is not movable and it
must be directly accessible by the kernel
> Thanks,
>
> jon
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 3/3] docs: core-api: add memory allocation guide
2018-09-12 10:33 ` Mike Rapoport
@ 2018-09-13 22:41 ` Jonathan Corbet
0 siblings, 0 replies; 9+ messages in thread
From: Jonathan Corbet @ 2018-09-13 22:41 UTC (permalink / raw)
To: Mike Rapoport
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel
On Wed, 12 Sep 2018 13:33:06 +0300
Mike Rapoport <rppt@linux.vnet.ibm.com> wrote:
> How about:
>
> ``GFP_HIGHUSER_MOVABLE`` does not require that allocated memory
> will be directly accessible by the kernel and implies that the
> data is movable.
>
> ``GFP_HIGHUSER`` means that the allocated memory is not movable,
> but it is not required to be directly accessible by the kernel. An
> example may be a hardware allocation that maps data directly into
> userspace but has no addressing limitations.
>
> ``GFP_USER`` means that the allocated memory is not movable and it
> must be directly accessible by the kernel
Sounds good to me.
Thanks,
jon
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] docs/core-api: add memory allocation guide
2018-08-17 14:47 [PATCH v3 0/3] docs/core-api: add memory allocation guide Mike Rapoport
` (2 preceding siblings ...)
2018-08-17 14:47 ` [PATCH v3 3/3] docs: core-api: add memory allocation guide Mike Rapoport
@ 2018-09-03 5:12 ` Mike Rapoport
2018-09-11 16:24 ` Mike Rapoport
4 siblings, 0 replies; 9+ messages in thread
From: Mike Rapoport @ 2018-09-03 5:12 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel
Any updates on this?
On Fri, Aug 17, 2018 at 05:47:13PM +0300, Mike Rapoport wrote:
> Hi,
>
> As Vlastimil mentioned at [1], it would be nice to have some guide about
> memory allocation. This set adds such guide that summarizes the "best
> practices".
>
> The changes from the RFC include additions and corrections from Michal and
> Randy. I've also added markup to cross-reference the kernel-doc
> documentation.
>
> I've split the patch into three to separate labels addition to the exiting
> files from the new contents.
>
> Note that the second patch depends on the mm docs update [2] that Andrew
> took to the -mm tree.
>
> v2 -> v3:
> * s/HW/hardware
>
> [1] https://www.spinics.net/lists/netfilter-devel/msg55542.html
> [2] https://lkml.org/lkml/2018/7/26/684
>
> Mike Rapoport (3):
> docs: core-api/gfp_mask-from-fs-io: add a label for cross-referencing
> docs: core-api/mm-api: add a lable for GFP flags section
> docs: core-api: add memory allocation guide
>
> Documentation/core-api/gfp_mask-from-fs-io.rst | 2 +
> Documentation/core-api/index.rst | 1 +
> Documentation/core-api/memory-allocation.rst | 124 +++++++++++++++++++++++++
> Documentation/core-api/mm-api.rst | 2 +
> 4 files changed, 129 insertions(+)
> create mode 100644 Documentation/core-api/memory-allocation.rst
>
> --
> 2.7.4
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/3] docs/core-api: add memory allocation guide
2018-08-17 14:47 [PATCH v3 0/3] docs/core-api: add memory allocation guide Mike Rapoport
` (3 preceding siblings ...)
2018-09-03 5:12 ` [PATCH v3 0/3] docs/core-api: " Mike Rapoport
@ 2018-09-11 16:24 ` Mike Rapoport
4 siblings, 0 replies; 9+ messages in thread
From: Mike Rapoport @ 2018-09-11 16:24 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Michal Hocko, Randy Dunlap, Matthew Wilcox, Vlastimil Babka,
linux-mm, linux-doc, linux-kernel
Ping?
On Fri, Aug 17, 2018 at 05:47:13PM +0300, Mike Rapoport wrote:
> Hi,
>
> As Vlastimil mentioned at [1], it would be nice to have some guide about
> memory allocation. This set adds such guide that summarizes the "best
> practices".
>
> The changes from the RFC include additions and corrections from Michal and
> Randy. I've also added markup to cross-reference the kernel-doc
> documentation.
>
> I've split the patch into three to separate labels addition to the exiting
> files from the new contents.
>
> Note that the second patch depends on the mm docs update [2] that Andrew
> took to the -mm tree.
>
> v2 -> v3:
> * s/HW/hardware
>
> [1] https://www.spinics.net/lists/netfilter-devel/msg55542.html
> [2] https://lkml.org/lkml/2018/7/26/684
>
> Mike Rapoport (3):
> docs: core-api/gfp_mask-from-fs-io: add a label for cross-referencing
> docs: core-api/mm-api: add a lable for GFP flags section
> docs: core-api: add memory allocation guide
>
> Documentation/core-api/gfp_mask-from-fs-io.rst | 2 +
> Documentation/core-api/index.rst | 1 +
> Documentation/core-api/memory-allocation.rst | 124 +++++++++++++++++++++++++
> Documentation/core-api/mm-api.rst | 2 +
> 4 files changed, 129 insertions(+)
> create mode 100644 Documentation/core-api/memory-allocation.rst
>
> --
> 2.7.4
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 9+ messages in thread