From: Miaohe Lin <linmiaohe@huawei.com>
To: Muchun Song <songmuchun@bytedance.com>
Cc: <duanxiongchun@bytedance.com>, <linux-doc@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>,
<linux-fsdevel@vger.kernel.org>, <corbet@lwn.net>,
<mike.kravetz@oracle.com>, <tglx@linutronix.de>,
<mingo@redhat.com>, <bp@alien8.de>, <x86@kernel.org>,
<hpa@zytor.com>, <dave.hansen@linux.intel.com>, <luto@kernel.org>,
<peterz@infradead.org>, <viro@zeniv.linux.org.uk>,
<akpm@linux-foundation.org>, <paulmck@kernel.org>,
<mchehab+huawei@kernel.org>, <pawan.kumar.gupta@linux.intel.com>,
<rdunlap@infradead.org>, <oneukum@suse.com>,
<anshuman.khandual@arm.com>, <jroedel@suse.de>,
<almasrymina@google.com>, <rientjes@google.com>,
<willy@infradead.org>, <osalvador@suse.de>, <mhocko@suse.com>,
<song.bao.hua@hisilicon.com>, <david@redhat.com>,
<naoya.horiguchi@nec.com>
Subject: Re: [PATCH v14 7/8] mm: hugetlb: gather discrete indexes of tail page
Date: Fri, 5 Feb 2021 15:30:17 +0800 [thread overview]
Message-ID: <1312358b-f065-4525-bbdf-25d011c72395@huawei.com> (raw)
In-Reply-To: <20210204035043.36609-8-songmuchun@bytedance.com>
On 2021/2/4 11:50, Muchun Song wrote:
> For HugeTLB page, there are more metadata to save in the struct page.
> But the head struct page cannot meet our needs, so we have to abuse
> other tail struct page to store the metadata. In order to avoid
> conflicts caused by subsequent use of more tail struct pages, we can
> gather these discrete indexes of tail struct page. In this case, it
> will be easier to add a new tail page index later.
>
> There are only (RESERVE_VMEMMAP_SIZE / sizeof(struct page)) struct
> page structs that can be used when CONFIG_HUGETLB_PAGE_FREE_VMEMMAP,
> so add a BUILD_BUG_ON to catch invalid usage of the tail struct page.
>
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> Reviewed-by: Oscar Salvador <osalvador@suse.de>
Thanks.
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> ---
> include/linux/hugetlb.h | 20 ++++++++++++++++++--
> include/linux/hugetlb_cgroup.h | 19 +++++++++++--------
> mm/hugetlb_vmemmap.c | 8 ++++++++
> 3 files changed, 37 insertions(+), 10 deletions(-)
>
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 775aea53669a..822ab2f5542a 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -28,6 +28,22 @@ typedef struct { unsigned long pd; } hugepd_t;
> #include <linux/shm.h>
> #include <asm/tlbflush.h>
>
> +/*
> + * For HugeTLB page, there are more metadata to save in the struct page. But
> + * the head struct page cannot meet our needs, so we have to abuse other tail
> + * struct page to store the metadata. In order to avoid conflicts caused by
> + * subsequent use of more tail struct pages, we gather these discrete indexes
> + * of tail struct page here.
> + */
> +enum {
> + SUBPAGE_INDEX_SUBPOOL = 1, /* reuse page->private */
> +#ifdef CONFIG_CGROUP_HUGETLB
> + SUBPAGE_INDEX_CGROUP, /* reuse page->private */
> + SUBPAGE_INDEX_CGROUP_RSVD, /* reuse page->private */
> +#endif
> + NR_USED_SUBPAGE,
> +};
> +
> struct hugepage_subpool {
> spinlock_t lock;
> long count;
> @@ -607,13 +623,13 @@ extern unsigned int default_hstate_idx;
> */
> static inline struct hugepage_subpool *hugetlb_page_subpool(struct page *hpage)
> {
> - return (struct hugepage_subpool *)(hpage+1)->private;
> + return (void *)page_private(hpage + SUBPAGE_INDEX_SUBPOOL);
> }
>
> static inline void hugetlb_set_page_subpool(struct page *hpage,
> struct hugepage_subpool *subpool)
> {
> - set_page_private(hpage+1, (unsigned long)subpool);
> + set_page_private(hpage + SUBPAGE_INDEX_SUBPOOL, (unsigned long)subpool);
> }
>
> static inline struct hstate *hstate_file(struct file *f)
> diff --git a/include/linux/hugetlb_cgroup.h b/include/linux/hugetlb_cgroup.h
> index 2ad6e92f124a..c0cae6a704f2 100644
> --- a/include/linux/hugetlb_cgroup.h
> +++ b/include/linux/hugetlb_cgroup.h
> @@ -21,15 +21,16 @@ struct hugetlb_cgroup;
> struct resv_map;
> struct file_region;
>
> +#ifdef CONFIG_CGROUP_HUGETLB
> /*
> * Minimum page order trackable by hugetlb cgroup.
> * At least 4 pages are necessary for all the tracking information.
> - * The second tail page (hpage[2]) is the fault usage cgroup.
> - * The third tail page (hpage[3]) is the reservation usage cgroup.
> + * The second tail page (hpage[SUBPAGE_INDEX_CGROUP]) is the fault
> + * usage cgroup. The third tail page (hpage[SUBPAGE_INDEX_CGROUP_RSVD])
> + * is the reservation usage cgroup.
> */
> -#define HUGETLB_CGROUP_MIN_ORDER 2
> +#define HUGETLB_CGROUP_MIN_ORDER order_base_2(NR_USED_SUBPAGE)
>
> -#ifdef CONFIG_CGROUP_HUGETLB
> enum hugetlb_memory_event {
> HUGETLB_MAX,
> HUGETLB_NR_MEMORY_EVENTS,
> @@ -66,9 +67,9 @@ __hugetlb_cgroup_from_page(struct page *page, bool rsvd)
> if (compound_order(page) < HUGETLB_CGROUP_MIN_ORDER)
> return NULL;
> if (rsvd)
> - return (struct hugetlb_cgroup *)page[3].private;
> + return (void *)page_private(page + SUBPAGE_INDEX_CGROUP_RSVD);
> else
> - return (struct hugetlb_cgroup *)page[2].private;
> + return (void *)page_private(page + SUBPAGE_INDEX_CGROUP);
> }
>
> static inline struct hugetlb_cgroup *hugetlb_cgroup_from_page(struct page *page)
> @@ -90,9 +91,11 @@ static inline int __set_hugetlb_cgroup(struct page *page,
> if (compound_order(page) < HUGETLB_CGROUP_MIN_ORDER)
> return -1;
> if (rsvd)
> - page[3].private = (unsigned long)h_cg;
> + set_page_private(page + SUBPAGE_INDEX_CGROUP_RSVD,
> + (unsigned long)h_cg);
> else
> - page[2].private = (unsigned long)h_cg;
> + set_page_private(page + SUBPAGE_INDEX_CGROUP,
> + (unsigned long)h_cg);
> return 0;
> }
>
> diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
> index 36ebd677e606..8efad9978821 100644
> --- a/mm/hugetlb_vmemmap.c
> +++ b/mm/hugetlb_vmemmap.c
> @@ -272,6 +272,14 @@ void __init hugetlb_vmemmap_init(struct hstate *h)
> unsigned int nr_pages = pages_per_huge_page(h);
> unsigned int vmemmap_pages;
>
> + /*
> + * There are only (RESERVE_VMEMMAP_SIZE / sizeof(struct page)) struct
> + * page structs that can be used when CONFIG_HUGETLB_PAGE_FREE_VMEMMAP,
> + * so add a BUILD_BUG_ON to catch invalid usage of the tail struct page.
> + */
> + BUILD_BUG_ON(NR_USED_SUBPAGE >=
> + RESERVE_VMEMMAP_SIZE / sizeof(struct page));
> +
> if (!hugetlb_free_vmemmap_enabled)
> return;
>
>
next prev parent reply other threads:[~2021-02-05 7:32 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-04 3:50 [PATCH v14 0/8] Free some vmemmap pages of HugeTLB page Muchun Song
2021-02-04 3:50 ` [PATCH v14 1/8] mm: memory_hotplug: factor out bootmem core functions to bootmem_info.c Muchun Song
2021-02-04 3:50 ` [PATCH v14 2/8] mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP Muchun Song
2021-02-04 11:44 ` Miaohe Lin
2021-02-04 3:50 ` [PATCH v14 3/8] mm: hugetlb: free the vmemmap pages associated with each HugeTLB page Muchun Song
2021-02-05 8:54 ` Oscar Salvador
2021-02-05 16:01 ` [External] " Muchun Song
2021-02-04 3:50 ` [PATCH v14 4/8] mm: hugetlb: alloc " Muchun Song
2021-02-05 9:29 ` Muchun Song
2021-02-05 11:54 ` Oscar Salvador
2021-02-06 8:01 ` [External] " Muchun Song
2021-02-04 3:50 ` [PATCH v14 5/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Muchun Song
2021-02-05 7:25 ` Miaohe Lin
2021-02-04 3:50 ` [PATCH v14 6/8] mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate Muchun Song
2021-02-05 7:29 ` Miaohe Lin
2021-02-05 8:22 ` Oscar Salvador
2021-02-05 8:39 ` Miaohe Lin
2021-02-05 8:56 ` Oscar Salvador
2021-02-05 9:12 ` Miaohe Lin
2021-02-04 3:50 ` [PATCH v14 7/8] mm: hugetlb: gather discrete indexes of tail page Muchun Song
2021-02-05 7:30 ` Miaohe Lin [this message]
2021-02-04 3:50 ` [PATCH v14 8/8] mm: hugetlb: optimize the code with the help of the compiler Muchun Song
2021-02-04 6:33 ` Miaohe Lin
2021-02-05 9:09 ` Oscar Salvador
2021-02-05 9:16 ` [External] " Muchun Song
2021-02-05 8:59 ` [PATCH v14 0/8] Free some vmemmap pages of HugeTLB page Oscar Salvador
2021-02-05 9:30 ` [External] " Muchun Song
2021-02-05 16:00 ` Joao Martins
2021-02-05 16:13 ` [External] " Muchun Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1312358b-f065-4525-bbdf-25d011c72395@huawei.com \
--to=linmiaohe@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=almasrymina@google.com \
--cc=anshuman.khandual@arm.com \
--cc=bp@alien8.de \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=david@redhat.com \
--cc=duanxiongchun@bytedance.com \
--cc=hpa@zytor.com \
--cc=jroedel@suse.de \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=mchehab+huawei@kernel.org \
--cc=mhocko@suse.com \
--cc=mike.kravetz@oracle.com \
--cc=mingo@redhat.com \
--cc=naoya.horiguchi@nec.com \
--cc=oneukum@suse.com \
--cc=osalvador@suse.de \
--cc=paulmck@kernel.org \
--cc=pawan.kumar.gupta@linux.intel.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rientjes@google.com \
--cc=song.bao.hua@hisilicon.com \
--cc=songmuchun@bytedance.com \
--cc=tglx@linutronix.de \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).