All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.com>
To: Muchun Song <songmuchun@bytedance.com>
Cc: corbet@lwn.net, mike.kravetz@oracle.com, tglx@linutronix.de,
	mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com,
	dave.hansen@linux.intel.com, luto@kernel.org,
	peterz@infradead.org, viro@zeniv.linux.org.uk,
	akpm@linux-foundation.org, paulmck@kernel.org,
	mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com,
	rdunlap@infradead.org, oneukum@suse.com,
	anshuman.khandual@arm.com, jroedel@suse.de,
	almasrymina@google.com, rientjes@google.com, willy@infradead.org,
	osalvador@suse.de, song.bao.hua@hisilicon.com,
	duanxiongchun@bytedance.com, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH v5 00/21] Free some vmemmap pages of hugetlb page
Date: Fri, 20 Nov 2020 09:42:02 +0100	[thread overview]
Message-ID: <20201120084202.GJ3200@dhcp22.suse.cz> (raw)
In-Reply-To: <20201120064325.34492-1-songmuchun@bytedance.com>

On Fri 20-11-20 14:43:04, Muchun Song wrote:
[...]

Thanks for improving the cover letter and providing some numbers. I have
only glanced through the patchset because I didn't really have more time
to dive depply into them.

Overall it looks promissing. To summarize. I would prefer to not have
the feature enablement controlled by compile time option and the kernel
command line option should be opt-in. I also do not like that freeing
the pool can trigger the oom killer or even shut the system down if no
oom victim is eligible.

One thing that I didn't really get to think hard about is what is the
effect of vmemmap manipulation wrt pfn walkers. pfn_to_page can be
invalid when racing with the split. How do we enforce that this won't
blow up?

I have also asked in a previous version whether the vmemmap manipulation
should be really unconditional. E.g. shortlived hugetlb pages allocated
from the buddy allocator directly rather than for a pool. Maybe it
should be restricted for the pool allocation as those are considered
long term and therefore the overhead will be amortized and freeing path
restrictions better understandable.

>  Documentation/admin-guide/kernel-parameters.txt |   9 +
>  Documentation/admin-guide/mm/hugetlbpage.rst    |   3 +
>  arch/x86/include/asm/hugetlb.h                  |  17 +
>  arch/x86/include/asm/pgtable_64_types.h         |   8 +
>  arch/x86/mm/init_64.c                           |   7 +-
>  fs/Kconfig                                      |  14 +
>  include/linux/bootmem_info.h                    |  78 +++
>  include/linux/hugetlb.h                         |  19 +
>  include/linux/hugetlb_cgroup.h                  |  15 +-
>  include/linux/memory_hotplug.h                  |  27 -
>  mm/Makefile                                     |   2 +
>  mm/bootmem_info.c                               | 124 ++++
>  mm/hugetlb.c                                    | 163 ++++-
>  mm/hugetlb_vmemmap.c                            | 765 ++++++++++++++++++++++++
>  mm/hugetlb_vmemmap.h                            | 103 ++++

I will need to look closer but I suspect that a non-trivial part of the
vmemmap manipulation really belongs to mm/sparse-vmemmap.c because the
split and remapping shouldn't really be hugetlb specific. Sure hugetlb
knows how to split but all the splitting should be implemented in
vmemmap proper.

>  mm/memory_hotplug.c                             | 116 ----
>  mm/sparse.c                                     |   5 +-
>  17 files changed, 1295 insertions(+), 180 deletions(-)
>  create mode 100644 include/linux/bootmem_info.h
>  create mode 100644 mm/bootmem_info.c
>  create mode 100644 mm/hugetlb_vmemmap.c
>  create mode 100644 mm/hugetlb_vmemmap.h

Thanks!
-- 
Michal Hocko
SUSE Labs

  parent reply	other threads:[~2020-11-20  8:42 UTC|newest]

Thread overview: 98+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-20  6:43 [PATCH v5 00/21] Free some vmemmap pages of hugetlb page Muchun Song
2020-11-20  6:43 ` [PATCH v5 01/21] mm/memory_hotplug: Move bootmem info registration API to bootmem_info.c Muchun Song
2020-11-20  6:43 ` [PATCH v5 02/21] mm/memory_hotplug: Move {get,put}_page_bootmem() " Muchun Song
2020-11-20  6:43 ` [PATCH v5 03/21] mm/hugetlb: Introduce a new config HUGETLB_PAGE_FREE_VMEMMAP Muchun Song
2020-11-20  7:49   ` Michal Hocko
2020-11-20  8:35     ` [External] " Muchun Song
2020-11-20  8:35       ` Muchun Song
2020-11-20  8:47       ` Michal Hocko
2020-11-20  8:53         ` Muchun Song
2020-11-20  8:53           ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 04/21] mm/hugetlb: Introduce nr_free_vmemmap_pages in the struct hstate Muchun Song
2020-11-20  6:43 ` [PATCH v5 05/21] mm/hugetlb: Introduce pgtable allocation/freeing helpers Muchun Song
2020-11-20  6:43 ` [PATCH v5 06/21] mm/bootmem_info: Introduce {free,prepare}_vmemmap_page() Muchun Song
2020-11-20  6:43 ` [PATCH v5 07/21] mm/bootmem_info: Combine bootmem info and type into page->freelist Muchun Song
2020-11-20  6:43 ` [PATCH v5 08/21] mm/hugetlb: Initialize page table lock for vmemmap Muchun Song
2020-11-20  6:43 ` [PATCH v5 09/21] mm/hugetlb: Free the vmemmap pages associated with each hugetlb page Muchun Song
2020-11-20  6:43 ` [PATCH v5 10/21] mm/hugetlb: Defer freeing of hugetlb pages Muchun Song
2020-11-20  6:43 ` [PATCH v5 11/21] mm/hugetlb: Allocate the vmemmap pages associated with each hugetlb page Muchun Song
2020-11-20  8:11   ` Michal Hocko
2020-11-20  8:51     ` [External] " Muchun Song
2020-11-20  8:51       ` Muchun Song
2020-11-20  9:28       ` Michal Hocko
2020-11-20  9:37         ` Muchun Song
2020-11-20  9:37           ` Muchun Song
2020-11-20 11:10           ` Michal Hocko
2020-11-20 11:56             ` Muchun Song
2020-11-20 11:56               ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 12/21] mm/hugetlb: Introduce remap_huge_page_pmd_vmemmap helper Muchun Song
2020-11-20  6:43 ` [PATCH v5 13/21] mm/hugetlb: Use PG_slab to indicate split pmd Muchun Song
2020-11-20  8:16   ` Michal Hocko
2020-11-20  9:30     ` [External] " Muchun Song
2020-11-20  9:30       ` Muchun Song
2020-11-23  7:48       ` Michal Hocko
2020-11-23  8:01         ` Muchun Song
2020-11-23  8:01           ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 14/21] mm/hugetlb: Support freeing vmemmap pages of gigantic page Muchun Song
2020-11-20  6:43 ` [PATCH v5 15/21] mm/hugetlb: Set the PageHWPoison to the raw error page Muchun Song
2020-11-20  8:19   ` Michal Hocko
2020-11-20 10:32     ` [External] " Muchun Song
2020-11-20 10:32       ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 16/21] mm/hugetlb: Flush work when dissolving hugetlb page Muchun Song
2020-11-20  8:20   ` Michal Hocko
2020-11-20  6:43 ` [PATCH v5 17/21] mm/hugetlb: Add a kernel parameter hugetlb_free_vmemmap Muchun Song
2020-11-20  8:22   ` Michal Hocko
2020-11-20 10:39     ` [External] " Muchun Song
2020-11-20 10:39       ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 18/21] mm/hugetlb: Merge pte to huge pmd only for gigantic page Muchun Song
2020-11-20  8:23   ` Michal Hocko
2020-11-20 10:41     ` [External] " Muchun Song
2020-11-20 10:41       ` Muchun Song
2020-11-20  6:43 ` [PATCH v5 19/21] mm/hugetlb: Gather discrete indexes of tail page Muchun Song
2020-11-20  6:43 ` [PATCH v5 20/21] mm/hugetlb: Add BUILD_BUG_ON to catch invalid usage of tail struct page Muchun Song
2020-11-20  6:43 ` [PATCH v5 21/21] mm/hugetlb: Disable freeing vmemmap if struct page size is not power of two Muchun Song
2020-11-20  8:25   ` Michal Hocko
2020-11-20  9:15     ` David Hildenbrand
2020-11-22 13:30       ` Mike Rapoport
2020-11-22 19:00     ` Matthew Wilcox
2020-11-23  3:14       ` [External] " Muchun Song
2020-11-23  3:14         ` Muchun Song
2020-11-20  9:16   ` David Hildenbrand
2020-11-20 10:42     ` [External] " Muchun Song
2020-11-20 10:42       ` Muchun Song
2020-11-20  8:42 ` Michal Hocko [this message]
2020-11-20  9:27   ` [PATCH v5 00/21] Free some vmemmap pages of hugetlb page David Hildenbrand
2020-11-20  9:39     ` Michal Hocko
2020-11-20  9:43       ` David Hildenbrand
2020-11-20 17:45         ` Mike Kravetz
2020-11-20 18:00           ` David Hildenbrand
2020-11-22  7:29           ` [External] " Muchun Song
2020-11-22  7:29             ` Muchun Song
2020-11-23  7:38           ` Michal Hocko
2020-11-23 21:52             ` Mike Kravetz
2020-11-23 22:01               ` Matthew Wilcox
2020-11-20 12:40   ` [External] " Muchun Song
2020-11-20 12:40     ` Muchun Song
2020-11-20 13:11     ` Michal Hocko
2020-11-20 15:44       ` Muchun Song
2020-11-20 15:44         ` Muchun Song
2020-11-23  7:40         ` Michal Hocko
2020-11-23  8:53           ` Muchun Song
2020-11-23  8:53             ` Muchun Song
2020-11-23  9:43             ` Michal Hocko
2020-11-23 10:36               ` Muchun Song
2020-11-23 10:36                 ` Muchun Song
2020-11-23 10:42                 ` Michal Hocko
2020-11-23 11:16                   ` Muchun Song
2020-11-23 11:16                     ` Muchun Song
2020-11-23 11:32                     ` Michal Hocko
2020-11-23 12:07                       ` Muchun Song
2020-11-23 12:07                         ` Muchun Song
2020-11-23 12:18                         ` Michal Hocko
2020-11-23 12:40                           ` Muchun Song
2020-11-23 12:40                             ` Muchun Song
2020-11-23 12:48                             ` Michal Hocko
2020-11-23 12:45                   ` Matthew Wilcox
2020-11-23 13:05                     ` Muchun Song
2020-11-23 13:05                       ` Muchun Song
2020-11-23 13:13                     ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201120084202.GJ3200@dhcp22.suse.cz \
    --to=mhocko@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=almasrymina@google.com \
    --cc=anshuman.khandual@arm.com \
    --cc=bp@alien8.de \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=duanxiongchun@bytedance.com \
    --cc=hpa@zytor.com \
    --cc=jroedel@suse.de \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@kernel.org \
    --cc=mchehab+huawei@kernel.org \
    --cc=mike.kravetz@oracle.com \
    --cc=mingo@redhat.com \
    --cc=oneukum@suse.com \
    --cc=osalvador@suse.de \
    --cc=paulmck@kernel.org \
    --cc=pawan.kumar.gupta@linux.intel.com \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=rientjes@google.com \
    --cc=song.bao.hua@hisilicon.com \
    --cc=songmuchun@bytedance.com \
    --cc=tglx@linutronix.de \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.