From: Michal Hocko <mhocko@suse.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Matthew Wilcox <willy@infradead.org>,
Hugh Dickins <hughd@google.com>,
Zhou Guanghui <zhouguanghui1@huawei.com>, Zi Yan <ziy@nvidia.com>,
Shakeel Butt <shakeelb@google.com>, Roman Gushchin <guro@fb.com>,
linux-mm@kvack.org, cgroups@vger.kernel.org,
linux-kernel@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH] mm: page_alloc: fix memcg accounting leak in speculative cache lookup
Date: Mon, 22 Mar 2021 10:55:30 +0100 [thread overview]
Message-ID: <YFhpkrK7MZ1mDQhi@dhcp22.suse.cz> (raw)
In-Reply-To: <20210319071547.60973-1-hannes@cmpxchg.org>
On Fri 19-03-21 03:15:47, Johannes Weiner wrote:
> When the freeing of a higher-order page block (non-compound) races
> with a speculative page cache lookup, __free_pages() needs to leave
> the first order-0 page in the chunk to the lookup but free the buddy
> pages that the lookup doesn't know about separately.
>
> However, if such a higher-order page is charged to a memcg (e.g. !vmap
> kernel stack)), only the first page of the block has page->memcg
> set. That means we'll uncharge only one order-0 page from the entire
> block, and leak the remainder.
>
> Add a split_page_memcg() to __free_pages() right before it starts
> taking the higher-order page apart and freeing its individual
> constituent pages. This ensures all of them will have the memcg
> linkage set up for correct uncharging. Also update the comments a bit
> to clarify what exactly is happening to the page during that race.
>
> This bug is old and has its roots in the speculative page cache patch
> and adding cgroup accounting of kernel pages. There are no known user
> reports. A backport to stable is therefor not warranted.
>
> Reported-by: Matthew Wilcox <willy@infradead.org>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Thanks!
> ---
> mm/page_alloc.c | 33 +++++++++++++++++++++++++++------
> 1 file changed, 27 insertions(+), 6 deletions(-)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index c53fe4fa10bf..f4bd56656402 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5112,10 +5112,9 @@ static inline void free_the_page(struct page *page, unsigned int order)
> * the allocation, so it is easy to leak memory. Freeing more memory
> * than was allocated will probably emit a warning.
> *
> - * If the last reference to this page is speculative, it will be released
> - * by put_page() which only frees the first page of a non-compound
> - * allocation. To prevent the remaining pages from being leaked, we free
> - * the subsequent pages here. If you want to use the page's reference
> + * This function isn't a put_page(). Don't let the put_page_testzero()
> + * fool you, it's only to deal with speculative cache references. It
> + * WILL free pages directly. If you want to use the page's reference
> * count to decide when to free the allocation, you should allocate a
> * compound page, and use put_page() instead of __free_pages().
> *
> @@ -5124,11 +5123,33 @@ static inline void free_the_page(struct page *page, unsigned int order)
> */
> void __free_pages(struct page *page, unsigned int order)
> {
> - if (put_page_testzero(page))
> + /*
> + * Drop the base reference from __alloc_pages and free. In
> + * case there is an outstanding speculative reference, from
> + * e.g. the page cache, it will put and free the page later.
> + */
> + if (likely(put_page_testzero(page))) {
> free_the_page(page, order);
> - else if (!PageHead(page))
> + return;
> + }
> +
> + /*
> + * The speculative reference will put and free the page.
> + *
> + * However, if the speculation was into a higher-order page
> + * chunk that isn't marked compound, the other side will know
> + * nothing about our buddy pages and only free the order-0
> + * page at the start of our chunk! We must split off and free
> + * the buddy pages here.
> + *
> + * The buddy pages aren't individually refcounted, so they
> + * can't have any pending speculative references themselves.
> + */
> + if (!PageHead(page) && order > 0) {
> + split_page_memcg(page, 1 << order);
> while (order-- > 0)
> free_the_page(page + (1 << order), order);
> + }
> }
> EXPORT_SYMBOL(__free_pages);
>
> --
> 2.30.1
>
--
Michal Hocko
SUSE Labs
prev parent reply other threads:[~2021-03-22 9:55 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-19 7:15 [PATCH] mm: page_alloc: fix memcg accounting leak in speculative cache lookup Johannes Weiner
2021-03-19 13:21 ` Matthew Wilcox
2021-03-20 1:52 ` Hugh Dickins
2021-03-20 3:25 ` Matthew Wilcox
2021-03-23 19:02 ` Johannes Weiner
2021-03-23 19:12 ` Matthew Wilcox
2021-03-23 20:10 ` Hugh Dickins
2021-03-26 1:55 ` Hugh Dickins
2021-03-26 2:51 ` Matthew Wilcox
2021-03-26 4:04 ` Hugh Dickins
2021-03-26 12:07 ` Matthew Wilcox
2021-03-24 8:58 ` Michal Hocko
2021-03-22 9:55 ` Michal Hocko [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YFhpkrK7MZ1mDQhi@dhcp22.suse.cz \
--to=mhocko@suse.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=shakeelb@google.com \
--cc=willy@infradead.org \
--cc=zhouguanghui1@huawei.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).