From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C29AAC433E5 for ; Tue, 28 Jul 2020 14:17:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76683206F5 for ; Tue, 28 Jul 2020 14:17:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="1SLkZtZe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76683206F5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F101C6B007B; Tue, 28 Jul 2020 10:17:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE7D76B007E; Tue, 28 Jul 2020 10:17:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD70D6B0080; Tue, 28 Jul 2020 10:17:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id B0A026B007B for ; Tue, 28 Jul 2020 10:17:13 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5138E8248076 for ; Tue, 28 Jul 2020 14:17:13 +0000 (UTC) X-FDA: 77087686746.28.dock98_55057c726f6a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 646B26D8B for ; Tue, 28 Jul 2020 14:17:09 +0000 (UTC) X-HE-Tag: dock98_55057c726f6a X-Filterd-Recvd-Size: 9220 Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Tue, 28 Jul 2020 14:17:06 +0000 (UTC) Received: by mail-pj1-f67.google.com with SMTP id lx9so844483pjb.2 for ; Tue, 28 Jul 2020 07:17:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=IGpofeA17dXnW8p+LpiNOYhDoLesFaEWeH9vhRlfQhI=; b=1SLkZtZeMIMwxIMuDjStl9UAdHtwooExe6Rg1P9sSgrqYPofvwdOniEsXPalrEj4ZE SJQpMRBb50bWQ0ZdwPNj1dtSaPXfhTFI5BCRMCzSEPPv+81mgGaKiuMq5unUV+0nlYPI HH5R90J3R709NQZxkyZ27sLlnbU2L/FvxJqIO6YQF9JLHd3b97q6hYFizpPSKODW9UKM lr+YFeqchqLmnIyvgl8no1etDAVJ0suvmIEtmovs7fC/fRpm7r0wDSq/l8YLQcsLWPrQ 6nGU/Gy7mD4RTtr0WDhs+QFa6+Xh0qq3gkpvXEfXWGd2B+8+PAJQ2RIjeuskL9Or6nRJ 1a0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=IGpofeA17dXnW8p+LpiNOYhDoLesFaEWeH9vhRlfQhI=; b=W4CtP6WcAfv0ULjV57WWbuJyuAsuRPC8oZUPf9KTXto6Dk0FCkuj2LFUYBI0stfcON rqrwBb9M0NCEo50Lyw3MCMDB6UNViisBVRvsP70mthUa0WJDNVG3kEggI1146oTdCKsT 2V+4qPH0CcnP+hMCIZWhjlY1Dg5ZXI8gKsG+mloICx3XPIzjbZWo7lj9UM3HlyrfBRfA oe5IpuNNQ6Jo2COq2TzJzX1djigan3Kef83K21wDCXQy0HlJ1PXnQQLS3J6D6LAxgqfe gVvMKsLEs/eBMWmNsoU9Tgdd+zSajNUEbqHlD++b0RUBYBMSyFJng1tPS6vLFwPWeoXq BcnA== X-Gm-Message-State: AOAM533EY/6cOrZn/pG9Lw8ybUmjby38KN7sRalna1g7XOEG37GWJG65 +G2aiGyYgZYdY980YQOp99SX2Lt69ub71TYbbfBG6w== X-Google-Smtp-Source: ABdhPJwJnfO4QjYiJywtGTs1VlxrNe5Tnx9vLxLG3J8+PobA6MnrnJatSc00k18wNmb7BHPagESfluqvmEh18mSXavQ= X-Received: by 2002:a17:90b:4005:: with SMTP id ie5mr4805430pjb.147.1595945825188; Tue, 28 Jul 2020 07:17:05 -0700 (PDT) MIME-Version: 1.0 References: <20200728034938.14993-1-songmuchun@bytedance.com> <20200728132453.GB14854@MiWiFi-R3L-srv> In-Reply-To: <20200728132453.GB14854@MiWiFi-R3L-srv> From: Muchun Song Date: Tue, 28 Jul 2020 22:16:29 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v4] mm/hugetlb: add mempolicy check in the reservation routine To: Baoquan He Cc: mike.kravetz@oracle.com, Andrew Morton , Michal Hocko , David Rientjes , mgorman@suse.de, Michel Lespinasse , Linux Memory Management List , LKML , Jianchao Guo Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 646B26D8B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 28, 2020 at 9:25 PM Baoquan He wrote: > > Hi Muchun, > > On 07/28/20 at 11:49am, Muchun Song wrote: > > In the reservation routine, we only check whether the cpuset meets > > the memory allocation requirements. But we ignore the mempolicy of > > MPOL_BIND case. If someone mmap hugetlb succeeds, but the subsequent > > memory allocation may fail due to mempolicy restrictions and receives > > the SIGBUS signal. This can be reproduced by the follow steps. > > > > 1) Compile the test case. > > cd tools/testing/selftests/vm/ > > gcc map_hugetlb.c -o map_hugetlb > > > > 2) Pre-allocate huge pages. Suppose there are 2 numa nodes in the > > system. Each node will pre-allocate one huge page. > > echo 2 > /proc/sys/vm/nr_hugepages > > > > 3) Run test case(mmap 4MB). We receive the SIGBUS signal. > > numactl --membind=0 ./map_hugetlb 4 > > I think supporting the mempolicy of MPOL_BIND case is a good idea. > I am wondering what about the other mempolicy cases, e.g MPOL_INTERLEAVE, > MPOL_PREFERRED. Asking these because we already have similar handling in > sysfs, proc nr_hugepages_mempolicy writting. Please see > __nr_hugepages_store_common() for detail. Yeah, I know the nr_hugepages_mempolicy. But this new code will help produce a quick failure as described in the commit message instead of waiting until the page fault routine(and receive a SIGBUG signal). > > Thanks > Baoquan > > > > > With this patch applied, the mmap will fail in the step 3) and throw > > "mmap: Cannot allocate memory". > > > > Signed-off-by: Muchun Song > > Reported-by: Jianchao Guo > > Suggested-by: Michal Hocko > > Reviewed-by: Mike Kravetz > > --- > > changelog in v4: > > 1) Fix compilation errors with !CONFIG_NUMA. > > > > changelog in v3: > > 1) Do not allocate nodemask on the stack. > > 2) Update comment. > > > > changelog in v2: > > 1) Reuse policy_nodemask(). > > > > include/linux/mempolicy.h | 14 ++++++++++++++ > > mm/hugetlb.c | 22 ++++++++++++++++++---- > > mm/mempolicy.c | 2 +- > > 3 files changed, 33 insertions(+), 5 deletions(-) > > > > diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h > > index ea9c15b60a96..0656ece1ccf1 100644 > > --- a/include/linux/mempolicy.h > > +++ b/include/linux/mempolicy.h > > @@ -152,6 +152,15 @@ extern int huge_node(struct vm_area_struct *vma, > > extern bool init_nodemask_of_mempolicy(nodemask_t *mask); > > extern bool mempolicy_nodemask_intersects(struct task_struct *tsk, > > const nodemask_t *mask); > > +extern nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy); > > + > > +static inline nodemask_t *policy_nodemask_current(gfp_t gfp) > > +{ > > + struct mempolicy *mpol = get_task_policy(current); > > + > > + return policy_nodemask(gfp, mpol); > > +} > > + > > extern unsigned int mempolicy_slab_node(void); > > > > extern enum zone_type policy_zone; > > @@ -281,5 +290,10 @@ static inline int mpol_misplaced(struct page *page, struct vm_area_struct *vma, > > static inline void mpol_put_task_policy(struct task_struct *task) > > { > > } > > + > > +static inline nodemask_t *policy_nodemask_current(gfp_t gfp) > > +{ > > + return NULL; > > +} > > #endif /* CONFIG_NUMA */ > > #endif > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > > index 589c330df4db..a34458f6a475 100644 > > --- a/mm/hugetlb.c > > +++ b/mm/hugetlb.c > > @@ -3463,13 +3463,21 @@ static int __init default_hugepagesz_setup(char *s) > > } > > __setup("default_hugepagesz=", default_hugepagesz_setup); > > > > -static unsigned int cpuset_mems_nr(unsigned int *array) > > +static unsigned int allowed_mems_nr(struct hstate *h) > > { > > int node; > > unsigned int nr = 0; > > + nodemask_t *mpol_allowed; > > + unsigned int *array = h->free_huge_pages_node; > > + gfp_t gfp_mask = htlb_alloc_mask(h); > > + > > + mpol_allowed = policy_nodemask_current(gfp_mask); > > > > - for_each_node_mask(node, cpuset_current_mems_allowed) > > - nr += array[node]; > > + for_each_node_mask(node, cpuset_current_mems_allowed) { > > + if (!mpol_allowed || > > + (mpol_allowed && node_isset(node, *mpol_allowed))) > > + nr += array[node]; > > + } > > > > return nr; > > } > > @@ -3648,12 +3656,18 @@ static int hugetlb_acct_memory(struct hstate *h, long delta) > > * we fall back to check against current free page availability as > > * a best attempt and hopefully to minimize the impact of changing > > * semantics that cpuset has. > > + * > > + * Apart from cpuset, we also have memory policy mechanism that > > + * also determines from which node the kernel will allocate memory > > + * in a NUMA system. So similar to cpuset, we also should consider > > + * the memory policy of the current task. Similar to the description > > + * above. > > */ > > if (delta > 0) { > > if (gather_surplus_pages(h, delta) < 0) > > goto out; > > > > - if (delta > cpuset_mems_nr(h->free_huge_pages_node)) { > > + if (delta > allowed_mems_nr(h)) { > > return_unused_surplus_pages(h, delta); > > goto out; > > } > > diff --git a/mm/mempolicy.c b/mm/mempolicy.c > > index 93fcfc1f2fa2..fce14c3f4f38 100644 > > --- a/mm/mempolicy.c > > +++ b/mm/mempolicy.c > > @@ -1873,7 +1873,7 @@ static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone) > > * Return a nodemask representing a mempolicy for filtering nodes for > > * page allocation > > */ > > -static nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) > > +nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) > > { > > /* Lower zones don't get a nodemask applied for MPOL_BIND */ > > if (unlikely(policy->mode == MPOL_BIND) && > > -- > > 2.11.0 > > > > > -- Yours, Muchun