Re: [RFC PATCH 2/8] hugetlb: recompute min_count when dropping hugetlb_lock

From: Michal Hocko <mhocko@suse.com>
To: Mike Kravetz <mike.kravetz@oracle.com>,
	Peter Zijlstra <peterz@infradead.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	Shakeel Butt <shakeelb@google.com>,
	Oscar Salvador <osalvador@suse.de>,
	David Hildenbrand <david@redhat.com>,
	Muchun Song <songmuchun@bytedance.com>,
	David Rientjes <rientjes@google.com>,
	Miaohe Lin <linmiaohe@huawei.com>,
	Matthew Wilcox <willy@infradead.org>,
	HORIGUCHI NAOYA <naoya.horiguchi@nec.com>,
	"Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
	Waiman Long <longman@redhat.com>, Peter Xu <peterx@redhat.com>,
	Mina Almasry <almasrymina@google.com>,
	Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [RFC PATCH 2/8] hugetlb: recompute min_count when dropping hugetlb_lock
Date: Tue, 23 Mar 2021 08:50:53 +0100	[thread overview]
Message-ID: <YFmd3d5B2VT4GkiG@dhcp22.suse.cz> (raw)
In-Reply-To: <a7d90d58-fa6a-7fa1-77c9-a08515746018@oracle.com>

On Mon 22-03-21 16:07:29, Mike Kravetz wrote:
> On 3/22/21 7:07 AM, Michal Hocko wrote:
> > On Fri 19-03-21 15:42:03, Mike Kravetz wrote:
> >> The routine set_max_huge_pages reduces the number of hugetlb_pages,
> >> by calling free_pool_huge_page in a loop.  It does this as long as
> >> persistent_huge_pages() is above a calculated min_count value.
> >> However, this loop can conditionally drop hugetlb_lock and in some
> >> circumstances free_pool_huge_page can drop hugetlb_lock.  If the
> >> lock is dropped, counters could change the calculated min_count
> >> value may no longer be valid.
> > 
> > OK, this one looks like a real bug fix introduced by 55f67141a8927.
> > Unless I am missing something we could release pages which are reserved
> > already.
> >  
> >> The routine try_to_free_low has the same issue.
> >>
> >> Recalculate min_count in each loop iteration as hugetlb_lock may have
> >> been dropped.
> >>
> >> Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
> >> ---
> >>  mm/hugetlb.c | 25 +++++++++++++++++++++----
> >>  1 file changed, 21 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> >> index d5be25f910e8..c537274c2a38 100644
> >> --- a/mm/hugetlb.c
> >> +++ b/mm/hugetlb.c
> >> @@ -2521,11 +2521,20 @@ static void __init report_hugepages(void)
> >>  	}
> >>  }
> >>  
> >> +static inline unsigned long min_hp_count(struct hstate *h, unsigned long count)
> >> +{
> >> +	unsigned long min_count;
> >> +
> >> +	min_count = h->resv_huge_pages + h->nr_huge_pages - h->free_huge_pages;
> >> +	return max(count, min_count);
> > 
> > Just out of curiousity, is compiler allowed to inline this piece of code
> > and then cache the value? In other words do we need to make these
> > READ_ONCE or otherwise enforce the no-caching behavior?
> 
> I honestly do not know if the compiler is allowed to do that.  The
> assembly code generated by my compiler does not cache the value, but
> that does not guarantee anything.  I can add READ_ONCE to make the
> function look something like:
> 
> static inline unsigned long min_hp_count(struct hstate *h, unsigned long count)
> {
> 	unsigned long min_count;
> 
> 	min_count = READ_ONCE(h->resv_huge_pages) + READ_ONCE(h->nr_huge_pages)
> 					- READ_ONCE(h->free_huge_pages);
> 	return max(count, min_count);
> }

Maybe just forcing to never inline the function should be sufficient.
This is not a hot path to micro optimize for no function call. But there
are much more qualified people on the CC list on this matter who could
clarify. Peter?
-- 
Michal Hocko
SUSE Labs