[PATCH v3 0/3] docs/core-api: memory-allocation: minor cleanups

[PATCH v3 0/3] docs/core-api: memory-allocation: minor cleanups

[Chris Packham]

Clean up some formatting and add references to helpers for calculating sizes
safely.

This series is based of v5.4-rc4.

There was a merge conflict with commit 59bb47985c1d ("mm, sl[aou]b: guarantee
natural alignment for kmalloc(power-of-two)") and the c:func: patch is
dependent on the typo fix. The former was resolved with a rebase, the latter by
actually sending it as part of the series.

Chris Packham (3):
  docs/core-api: memory-allocation: fix typo
  docs/core-api: memory-allocation: remove uses of c:func:
  docs/core-api: memory-allocation: mention size helpers

 Documentation/core-api/memory-allocation.rst | 50 ++++++++++----------
 1 file changed, 24 insertions(+), 26 deletions(-)

-- 
2.23.0

[PATCH v3 1/3] docs/core-api: memory-allocation: fix typo

[Chris Packham]

"on the safe size" should be "on the safe side".

Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
---
 Documentation/core-api/memory-allocation.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
index 939e3dfc86e9..dcf851b4520f 100644
--- a/Documentation/core-api/memory-allocation.rst
+++ b/Documentation/core-api/memory-allocation.rst
@@ -88,7 +88,7 @@ Selecting memory allocator
 ==========================
 
 The most straightforward way to allocate memory is to use a function
-from the :c:func:`kmalloc` family. And, to be on the safe size it's
+from the :c:func:`kmalloc` family. And, to be on the safe side it's
 best to use routines that set memory to zero, like
 :c:func:`kzalloc`. If you need to allocate memory for an array, there
 are :c:func:`kmalloc_array` and :c:func:`kcalloc` helpers.
-- 
2.23.0

[PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Chris Packham]

These are no longer needed as the documentation build will automatically
add the cross references.

Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
---

Notes:
    It should be noted that kvmalloc() and kmem_cache_destroy() lack a
    kerneldoc header, a side-effect of this change is that the :c:func:
    fallback of making them bold is lost. This is probably best fixed by
    adding a kerneldoc header to their source.
    
    Changes in v2:
    - new

 Documentation/core-api/memory-allocation.rst | 49 +++++++++-----------
 1 file changed, 23 insertions(+), 26 deletions(-)

diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
index dcf851b4520f..e47d48655085 100644
--- a/Documentation/core-api/memory-allocation.rst
+++ b/Documentation/core-api/memory-allocation.rst
@@ -88,10 +88,10 @@ Selecting memory allocator
 ==========================
 
 The most straightforward way to allocate memory is to use a function
-from the :c:func:`kmalloc` family. And, to be on the safe side it's
-best to use routines that set memory to zero, like
-:c:func:`kzalloc`. If you need to allocate memory for an array, there
-are :c:func:`kmalloc_array` and :c:func:`kcalloc` helpers.
+from the kmalloc() family. And, to be on the safe side it's best to use
+routines that set memory to zero, like kzalloc(). If you need to
+allocate memory for an array, there are kmalloc_array() and kcalloc()
+helpers.
 
 The maximal size of a chunk that can be allocated with `kmalloc` is
 limited. The actual limit depends on the hardware and the kernel
@@ -102,29 +102,26 @@ The address of a chunk allocated with `kmalloc` is aligned to at least
 ARCH_KMALLOC_MINALIGN bytes.  For sizes which are a power of two, the
 alignment is also guaranteed to be at least the respective size.
 
-For large allocations you can use :c:func:`vmalloc` and
-:c:func:`vzalloc`, or directly request pages from the page
-allocator. The memory allocated by `vmalloc` and related functions is
-not physically contiguous.
+For large allocations you can use vmalloc() and vzalloc(), or directly
+request pages from the page allocator. The memory allocated by `vmalloc`
+and related functions is not physically contiguous.
 
 If you are not sure whether the allocation size is too large for
-`kmalloc`, it is possible to use :c:func:`kvmalloc` and its
-derivatives. It will try to allocate memory with `kmalloc` and if the
-allocation fails it will be retried with `vmalloc`. There are
-restrictions on which GFP flags can be used with `kvmalloc`; please
-see :c:func:`kvmalloc_node` reference documentation. Note that
-`kvmalloc` may return memory that is not physically contiguous.
+`kmalloc`, it is possible to use kvmalloc() and its derivatives. It will
+try to allocate memory with `kmalloc` and if the allocation fails it
+will be retried with `vmalloc`. There are restrictions on which GFP
+flags can be used with `kvmalloc`; please see kvmalloc_node() reference
+documentation. Note that `kvmalloc` may return memory that is not
+physically contiguous.
 
 If you need to allocate many identical objects you can use the slab
-cache allocator. The cache should be set up with
-:c:func:`kmem_cache_create` or :c:func:`kmem_cache_create_usercopy`
-before it can be used. The second function should be used if a part of
-the cache might be copied to the userspace.  After the cache is
-created :c:func:`kmem_cache_alloc` and its convenience wrappers can
-allocate memory from that cache.
-
-When the allocated memory is no longer needed it must be freed. You
-can use :c:func:`kvfree` for the memory allocated with `kmalloc`,
-`vmalloc` and `kvmalloc`. The slab caches should be freed with
-:c:func:`kmem_cache_free`. And don't forget to destroy the cache with
-:c:func:`kmem_cache_destroy`.
+cache allocator. The cache should be set up with kmem_cache_create() or
+kmem_cache_create_usercopy() before it can be used. The second function
+should be used if a part of the cache might be copied to the userspace.
+After the cache is created kmem_cache_alloc() and its convenience
+wrappers can allocate memory from that cache.
+
+When the allocated memory is no longer needed it must be freed. You can
+use kvfree() for the memory allocated with `kmalloc`, `vmalloc` and
+`kvmalloc`. The slab caches should be freed with kmem_cache_free(). And
+don't forget to destroy the cache with kmem_cache_destroy().
-- 
2.23.0

[PATCH v3 3/3] docs/core-api: memory-allocation: mention size helpers

[Chris Packham]

Mention struct_size(), array_size() and array3_size() in the same place
as kmalloc() and friends.

Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
---

Notes:
    Changes in v3:
    - rebase against v5.4-rc4
    Changes in v2:
    - Drop use of c:func:

 Documentation/core-api/memory-allocation.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
index e47d48655085..4aa82ddd01b8 100644
--- a/Documentation/core-api/memory-allocation.rst
+++ b/Documentation/core-api/memory-allocation.rst
@@ -91,7 +91,8 @@ The most straightforward way to allocate memory is to use a function
 from the kmalloc() family. And, to be on the safe side it's best to use
 routines that set memory to zero, like kzalloc(). If you need to
 allocate memory for an array, there are kmalloc_array() and kcalloc()
-helpers.
+helpers. The helpers struct_size(), array_size() and array3_size() can
+be used to safely calculate object sizes without overflowing.
 
 The maximal size of a chunk that can be allocated with `kmalloc` is
 limited. The actual limit depends on the hardware and the kernel
-- 
2.23.0

Re: [PATCH v3 0/3] docs/core-api: memory-allocation: minor cleanups

[Matthew Wilcox]

On Fri, Oct 25, 2019 at 08:50:13AM +1300, Chris Packham wrote:
> Clean up some formatting and add references to helpers for calculating sizes
> safely.

For the whole series:

Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Re: [PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Jonathan Corbet]

On Fri, 25 Oct 2019 08:50:15 +1300
Chris Packham <chris.packham@alliedtelesis.co.nz> wrote:

> These are no longer needed as the documentation build will automatically
> add the cross references.
> 
> Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
> ---
> 
> Notes:
>     It should be noted that kvmalloc() and kmem_cache_destroy() lack a
>     kerneldoc header, a side-effect of this change is that the :c:func:
>     fallback of making them bold is lost. This is probably best fixed by
>     adding a kerneldoc header to their source.
>     
>     Changes in v2:
>     - new
> 
>  Documentation/core-api/memory-allocation.rst | 49 +++++++++-----------
>  1 file changed, 23 insertions(+), 26 deletions(-)

This one still doesn't apply; have you verified that you can apply the
whole series to docs-next?

Thanks,

jon

Re: [PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Chris Packham]

Hi Jon,

On Thu, 2019-10-24 at 14:29 -0600, Jonathan Corbet wrote:
> On Fri, 25 Oct 2019 08:50:15 +1300
> Chris Packham <chris.packham@alliedtelesis.co.nz> wrote:
> 
> > These are no longer needed as the documentation build will automatically
> > add the cross references.
> > 
> > Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
> > ---
> > 
> > Notes:
> >     It should be noted that kvmalloc() and kmem_cache_destroy() lack a
> >     kerneldoc header, a side-effect of this change is that the :c:func:
> >     fallback of making them bold is lost. This is probably best fixed by
> >     adding a kerneldoc header to their source.
> >     
> >     Changes in v2:
> >     - new
> > 
> >  Documentation/core-api/memory-allocation.rst | 49 +++++++++-----------
> >  1 file changed, 23 insertions(+), 26 deletions(-)
> 
> This one still doesn't apply; have you verified that you can apply the
> whole series to docs-next?

I can see the problem. I'll rebase against docs-next and send a v4.

When I do actually get a series that applies to docs-next it'll
conflict with 59bb47985c1d ("mm, sl[aou]b: guarantee
natural alignment for kmalloc(power-of-two)") in Linus's tree.



> 
> Thanks,
> 
> jon

Re: [PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Jonathan Corbet]

On Thu, 24 Oct 2019 20:51:23 +0000
Chris Packham <Chris.Packham@alliedtelesis.co.nz> wrote:

> When I do actually get a series that applies to docs-next it'll
> conflict with 59bb47985c1d ("mm, sl[aou]b: guarantee
> natural alignment for kmalloc(power-of-two)") in Linus's tree.

Alternatively, if I sync up to -rc4, does the problem go away?  I should
be able to explain that to Linus without too much trouble...

Thanks,

jon

Re: [PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Chris Packham]

On Thu, 2019-10-24 at 14:57 -0600, Jonathan Corbet wrote:
> On Thu, 24 Oct 2019 20:51:23 +0000
> Chris Packham <Chris.Packham@alliedtelesis.co.nz> wrote:
> 
> > When I do actually get a series that applies to docs-next it'll
> > conflict with 59bb47985c1d ("mm, sl[aou]b: guarantee
> > natural alignment for kmalloc(power-of-two)") in Linus's tree.
> 
> Alternatively, if I sync up to -rc4, does the problem go away?  I should
> be able to explain that to Linus without too much trouble...
> 

Yes the series applies cleanly once docs-next is merged with v5.4-rc4.

There is a merge conflict between 6ee0fac199e1 ("docs: fix memory.low
description in cgroup-v2.rst") in your tree and 9783aa9917f8 ("mm,
memcg: proportional memory.{low,min} reclaim") in Linus's tree. Here's
the resolution I came up with in my test merge

diff --cc Documentation/admin-guide/cgroup-v2.rst
index 26d1cde6b34a,5361ebec3361..515c2d9eef84
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@@ -1117,8 -1120,11 +1120,11 @@@ PAGE_SIZE multiple when read back
  
        Best-effort memory protection.  If the memory usage of a
        cgroup is within its effective low boundary, the cgroup's
 -      memory won't be reclaimed unless memory can be reclaimed
 +      memory won't be reclaimed unless there is no reclaimable
-       memory available in unprotected cgroups.
+       from unprotected cgroups.  Above the effective low boundary (or
+       effective min boundary if it is higher), pages are reclaimed
+       proportionally to the overage, reducing reclaim pressure for
+       smaller overages.
  
        Effective low boundary is limited by memory.low values of
        all ancestor cgroups. If there is memory.low overcommitment

Re: [PATCH v3 0/3] docs/core-api: memory-allocation: minor cleanups

[Mike Rapoport]

On Fri, Oct 25, 2019 at 08:50:13AM +1300, Chris Packham wrote:
> Clean up some formatting and add references to helpers for calculating sizes
> safely.
> 
> This series is based of v5.4-rc4.
> 
> There was a merge conflict with commit 59bb47985c1d ("mm, sl[aou]b: guarantee
> natural alignment for kmalloc(power-of-two)") and the c:func: patch is
> dependent on the typo fix. The former was resolved with a rebase, the latter by
> actually sending it as part of the series.
> 
> Chris Packham (3):
>   docs/core-api: memory-allocation: fix typo
>   docs/core-api: memory-allocation: remove uses of c:func:
>   docs/core-api: memory-allocation: mention size helpers

For the series:

Acked-by: Mike Rapoport <rppt@linux.ibm.com>
 
>  Documentation/core-api/memory-allocation.rst | 50 ++++++++++----------
>  1 file changed, 24 insertions(+), 26 deletions(-)
> 
> -- 
> 2.23.0
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH v3 2/3] docs/core-api: memory-allocation: remove uses of c:func:

[Mike Rapoport]

On Fri, Oct 25, 2019 at 08:50:15AM +1300, Chris Packham wrote:
> These are no longer needed as the documentation build will automatically
> add the cross references.
> 
> Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
> ---
> 
> Notes:
>     It should be noted that kvmalloc() and kmem_cache_destroy() lack a
>     kerneldoc header, a side-effect of this change is that the :c:func:
>     fallback of making them bold is lost. This is probably best fixed by
>     adding a kerneldoc header to their source.

You are more than welcome to add them ;-)
     
>     Changes in v2:
>     - new
> 
>  Documentation/core-api/memory-allocation.rst | 49 +++++++++-----------
>  1 file changed, 23 insertions(+), 26 deletions(-)
> 
> diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
> index dcf851b4520f..e47d48655085 100644
> --- a/Documentation/core-api/memory-allocation.rst
> +++ b/Documentation/core-api/memory-allocation.rst
> @@ -88,10 +88,10 @@ Selecting memory allocator
>  ==========================
>  
>  The most straightforward way to allocate memory is to use a function
> -from the :c:func:`kmalloc` family. And, to be on the safe side it's
> -best to use routines that set memory to zero, like
> -:c:func:`kzalloc`. If you need to allocate memory for an array, there
> -are :c:func:`kmalloc_array` and :c:func:`kcalloc` helpers.
> +from the kmalloc() family. And, to be on the safe side it's best to use
> +routines that set memory to zero, like kzalloc(). If you need to
> +allocate memory for an array, there are kmalloc_array() and kcalloc()
> +helpers.
>  
>  The maximal size of a chunk that can be allocated with `kmalloc` is
>  limited. The actual limit depends on the hardware and the kernel
> @@ -102,29 +102,26 @@ The address of a chunk allocated with `kmalloc` is aligned to at least
>  ARCH_KMALLOC_MINALIGN bytes.  For sizes which are a power of two, the
>  alignment is also guaranteed to be at least the respective size.
>  
> -For large allocations you can use :c:func:`vmalloc` and
> -:c:func:`vzalloc`, or directly request pages from the page
> -allocator. The memory allocated by `vmalloc` and related functions is
> -not physically contiguous.
> +For large allocations you can use vmalloc() and vzalloc(), or directly
> +request pages from the page allocator. The memory allocated by `vmalloc`
> +and related functions is not physically contiguous.
>  
>  If you are not sure whether the allocation size is too large for
> -`kmalloc`, it is possible to use :c:func:`kvmalloc` and its
> -derivatives. It will try to allocate memory with `kmalloc` and if the
> -allocation fails it will be retried with `vmalloc`. There are
> -restrictions on which GFP flags can be used with `kvmalloc`; please
> -see :c:func:`kvmalloc_node` reference documentation. Note that
> -`kvmalloc` may return memory that is not physically contiguous.
> +`kmalloc`, it is possible to use kvmalloc() and its derivatives. It will
> +try to allocate memory with `kmalloc` and if the allocation fails it
> +will be retried with `vmalloc`. There are restrictions on which GFP
> +flags can be used with `kvmalloc`; please see kvmalloc_node() reference
> +documentation. Note that `kvmalloc` may return memory that is not
> +physically contiguous.
>  
>  If you need to allocate many identical objects you can use the slab
> -cache allocator. The cache should be set up with
> -:c:func:`kmem_cache_create` or :c:func:`kmem_cache_create_usercopy`
> -before it can be used. The second function should be used if a part of
> -the cache might be copied to the userspace.  After the cache is
> -created :c:func:`kmem_cache_alloc` and its convenience wrappers can
> -allocate memory from that cache.
> -
> -When the allocated memory is no longer needed it must be freed. You
> -can use :c:func:`kvfree` for the memory allocated with `kmalloc`,
> -`vmalloc` and `kvmalloc`. The slab caches should be freed with
> -:c:func:`kmem_cache_free`. And don't forget to destroy the cache with
> -:c:func:`kmem_cache_destroy`.
> +cache allocator. The cache should be set up with kmem_cache_create() or
> +kmem_cache_create_usercopy() before it can be used. The second function
> +should be used if a part of the cache might be copied to the userspace.
> +After the cache is created kmem_cache_alloc() and its convenience
> +wrappers can allocate memory from that cache.
> +
> +When the allocated memory is no longer needed it must be freed. You can
> +use kvfree() for the memory allocated with `kmalloc`, `vmalloc` and
> +`kvmalloc`. The slab caches should be freed with kmem_cache_free(). And
> +don't forget to destroy the cache with kmem_cache_destroy().
> -- 
> 2.23.0
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH v3 0/3] docs/core-api: memory-allocation: minor cleanups

[Jonathan Corbet]

On Fri, 25 Oct 2019 08:50:13 +1300
Chris Packham <chris.packham@alliedtelesis.co.nz> wrote:

> Clean up some formatting and add references to helpers for calculating sizes
> safely.
> 
> This series is based of v5.4-rc4.
> 
> There was a merge conflict with commit 59bb47985c1d ("mm, sl[aou]b: guarantee
> natural alignment for kmalloc(power-of-two)") and the c:func: patch is
> dependent on the typo fix. The former was resolved with a rebase, the latter by
> actually sending it as part of the series.
> 
> Chris Packham (3):
>   docs/core-api: memory-allocation: fix typo
>   docs/core-api: memory-allocation: remove uses of c:func:
>   docs/core-api: memory-allocation: mention size helpers
> 
>  Documentation/core-api/memory-allocation.rst | 50 ++++++++++----------
>  1 file changed, 24 insertions(+), 26 deletions(-)

OK, this series is (finally) applied, thanks for your patience...

jon