From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B82DC4338F for ; Mon, 16 Aug 2021 23:29:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E9BA860F4B for ; Mon, 16 Aug 2021 23:29:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232792AbhHPXaN (ORCPT ); Mon, 16 Aug 2021 19:30:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:56250 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234060AbhHPX35 (ORCPT ); Mon, 16 Aug 2021 19:29:57 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 946D960F4B; Mon, 16 Aug 2021 23:29:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1629156565; bh=ldTqrIbTIun9NTy9rQVypaKbvvjavp3irOHg3pOWqRM=; h=Date:From:To:Subject:From; b=Rx38q0l1of+zu7vjipgV3Y9s9njiFj/c4pYYUscBu0uLuSSkPf9FK2MUh00jm1mW6 NigFZRCmc+wkdYR4G0EU2rJk/myBHl9q2sg5XF6ST9b6z7yoLW1A0e4conkxbeJ4OR U4sKbBb1uEe1l7ZAIp6tQbRSqsCF768BjpbLS9t0= Date: Mon, 16 Aug 2021 16:29:25 -0700 From: akpm@linux-foundation.org To: mm-commits@vger.kernel.org, ziy@nvidia.com, songmuchun@bytedance.com, rientjes@google.com, osalvador@suse.de, naoya.horiguchi@linux.dev, mhocko@suse.com, david@redhat.com, mike.kravetz@oracle.com Subject: + hugetlb-add-hugetlb-demote-page-support.patch added to -mm tree Message-ID: <20210816232925.FUady%akpm@linux-foundation.org> User-Agent: s-nail v14.9.10 Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org The patch titled Subject: hugetlb: add hugetlb demote page support has been added to the -mm tree. Its filename is hugetlb-add-hugetlb-demote-page-support.patch This patch should soon appear at https://ozlabs.org/~akpm/mmots/broken-out/hugetlb-add-hugetlb-demote-page-support.patch and later at https://ozlabs.org/~akpm/mmotm/broken-out/hugetlb-add-hugetlb-demote-page-support.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Mike Kravetz Subject: hugetlb: add hugetlb demote page support Demote page functionality will split a huge page into a number of huge pages of a smaller size. For example, on x86 a 1GB huge page can be demoted into 512 2M huge pages. Demotion is done 'in place' by simply splitting the huge page. Added '*_for_demote' wrappers for remove_hugetlb_page, destroy_compound_gigantic_page and prep_compound_gigantic_page for use by demote code. Link: https://lkml.kernel.org/r/20210816224953.157796-5-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Cc: David Hildenbrand Cc: David Rientjes Cc: Michal Hocko Cc: Muchun Song Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/hugetlb.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 73 insertions(+), 3 deletions(-) --- a/mm/hugetlb.c~hugetlb-add-hugetlb-demote-page-support +++ a/mm/hugetlb.c @@ -1279,6 +1279,12 @@ static void destroy_compound_gigantic_pa __destroy_compound_gigantic_page(page, order, false); } +static void destroy_compound_gigantic_page_for_demote(struct page *page, + unsigned int order) +{ + __destroy_compound_gigantic_page(page, order, true); +} + static void free_gigantic_page(struct page *page, unsigned int order) { #ifdef CONFIG_CMA @@ -1418,6 +1424,12 @@ static void remove_hugetlb_page(struct h __remove_hugetlb_page(h, page, adjust_surplus, false); } +static void remove_hugetlb_page_for_demote(struct hstate *h, struct page *page, + bool adjust_surplus) +{ + __remove_hugetlb_page(h, page, adjust_surplus, true); +} + static void add_hugetlb_page(struct hstate *h, struct page *page, bool adjust_surplus) { @@ -1767,6 +1779,12 @@ static bool prep_compound_gigantic_page( return __prep_compound_gigantic_page(page, order, false); } +static bool prep_compound_gigantic_page_for_demote(struct page *page, + unsigned int order) +{ + return __prep_compound_gigantic_page(page, order, true); +} + /* * PageHuge() only returns true for hugetlbfs pages, but not for normal or * transparent huge pages. See the PageTransHuge() documentation for more @@ -3274,9 +3292,55 @@ out: return 0; } +static int demote_free_huge_page(struct hstate *h, struct page *page) +{ + int i, nid = page_to_nid(page); + struct hstate *target_hstate; + bool cma_page = HPageCma(page); + + target_hstate = size_to_hstate(PAGE_SIZE << h->demote_order); + + remove_hugetlb_page_for_demote(h, page, false); + spin_unlock_irq(&hugetlb_lock); + + if (alloc_huge_page_vmemmap(h, page)) { + /* Allocation of vmemmmap failed, we can not demote page */ + spin_lock_irq(&hugetlb_lock); + set_page_refcounted(page); + add_hugetlb_page(h, page, false); + return 1; + } + + /* + * Use destroy_compound_gigantic_page_for_demote for all huge page + * sizes as it will not ref count pages. + */ + destroy_compound_gigantic_page_for_demote(page, huge_page_order(h)); + + for (i = 0; i < pages_per_huge_page(h); + i += pages_per_huge_page(target_hstate)) { + if (hstate_is_gigantic(target_hstate)) + prep_compound_gigantic_page_for_demote(page + i, + target_hstate->order); + else + prep_compound_page(page + i, target_hstate->order); + set_page_private(page + i, 0); + set_page_refcounted(page + i); + prep_new_huge_page(target_hstate, page + i, nid); + if (cma_page) + SetHPageCma(page + i); + put_page(page + i); + } + + spin_lock_irq(&hugetlb_lock); + return 0; +} + static int demote_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed) __must_hold(&hugetlb_lock) { + int nr_nodes, node; + struct page *page; int rc = 0; lockdep_assert_held(&hugetlb_lock); @@ -3292,9 +3356,15 @@ static int demote_pool_huge_page(struct return 1; } - /* - * TODO - demote fucntionality will be added in subsequent patch - */ + for_each_node_mask_to_free(h, nr_nodes, node, nodes_allowed) { + if (!list_empty(&h->hugepage_freelists[node])) { + page = list_entry(h->hugepage_freelists[node].next, + struct page, lru); + rc = !demote_free_huge_page(h, page); + break; + } + } + return rc; } _ Patches currently in -mm which might be from mike.kravetz@oracle.com are hugetlb-simplify-prep_compound_gigantic_page-ref-count-racing-code.patch hugetlb-drop-ref-count-earlier-after-page-allocation.patch hugetlb-before-freeing-hugetlb-page-set-dtor-to-appropriate-value.patch hugetlb-add-demote-hugetlb-page-sysfs-interfaces.patch hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch hugetlb-add-demote-bool-to-gigantic-page-routines.patch hugetlb-add-hugetlb-demote-page-support.patch hugetlb-document-the-demote-sysfs-interfaces.patch hugetlb-vmemmap-optimizations-when-demoting-hugetlb-pages.patch hugetlb-prepare-destroy-and-prep-routines-for-vmemmap-optimized-pages.patch hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch