From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CD97C433E0 for ; Mon, 1 Jun 2020 23:00:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B4B6D2073B for ; Mon, 1 Jun 2020 23:00:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Xbisw2gF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4B6D2073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4AB9E80007; Mon, 1 Jun 2020 19:00:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45C958E0006; Mon, 1 Jun 2020 19:00:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34A9380007; Mon, 1 Jun 2020 19:00:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 18EFF8E0006 for ; Mon, 1 Jun 2020 19:00:21 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BB8C5180AD815 for ; Mon, 1 Jun 2020 23:00:20 +0000 (UTC) X-FDA: 76882163400.30.test35_3d24d270d363a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 8E843180B3C8B for ; Mon, 1 Jun 2020 23:00:20 +0000 (UTC) X-HE-Tag: test35_3d24d270d363a X-Filterd-Recvd-Size: 9203 Received: from mail-ej1-f65.google.com (mail-ej1-f65.google.com [209.85.218.65]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Mon, 1 Jun 2020 23:00:20 +0000 (UTC) Received: by mail-ej1-f65.google.com with SMTP id f7so10829929ejq.6 for ; Mon, 01 Jun 2020 16:00:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=RXVWrJM0qErwoZtOlN6QuMQ8iwvgbXwoJOqbVdWAFHY=; b=Xbisw2gFUJ2KuoD5GrY5EdEz3hKD2LSX+FXSyf19uB3hWSqL2cyCBBdVSJDzm24eZA yRitsX00wWVhri4dWL84YRH/5TPgHP3Or58GNWyhiRsXH4TDc/wnSXC/EfSrc+cQ9uPY kmwJr99NcWpT18FIVNRJoylvm9IGChTJIsh7qn8UBxaya6e4j4Pwr/r9CVbhZbZa9ZAg jB1b5g4PtmBw+KdloZ4sU05ggtCAKh9Ra2xSDynQSEqQ8IUSTr2usZBUjODc4bw/tCIB UzGCGUybJ9iQEN5fa/Pv75q77zFAyuCUpgamDrFOLp2m/m80kfa0XboA4qYvItBNIC5B wz9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=RXVWrJM0qErwoZtOlN6QuMQ8iwvgbXwoJOqbVdWAFHY=; b=r230eofbQmiecFqckYqRK66Vy3B9Df2SvnRSgNc1OF5oplmnd9Ji8sG5Wsll+wg/oI EueYsAwBAD5E/aCFQALh4lGBQbvUP++giQhFTPyJ5WjsIx+Uo8zUNF+o4qGJff3lamNL dud9Y0PFQSjKlG0TPxOEv01wCv0wcRNud8kozy91I5dBMUeLSPvS78KF/Sbqg8v6Iqk/ ePizRLxQY8WirO0iUMkVSJG4464nwDaYuJdyvw/yokCMMxU91jTarGf+zyIvGpvIfNOU EK+uOPMZUasDtuDTHg/mPHkK4EArEnwJ0KPcF2WZYGjnqpgoiWvWM1mDklTM6bNEeTYB jejA== X-Gm-Message-State: AOAM5329TIei90vQxe/a7U13RTbYuZ/PpMls0XI4lHk7pPWX67kbKKAq csa4GYVgnzNZcBB9LOTjqEnlksjjBF7Q395vaGQ= X-Google-Smtp-Source: ABdhPJx71vSqtqi5aHP1BnaPIdrc7rF0WI6Cb4Hk3hNYRuJ+fBfDfxvh0/qTsskaqLy1c/pfzyBqrGbANg415hraRcs= X-Received: by 2002:a17:906:ae85:: with SMTP id md5mr15533786ejb.213.1591052418869; Mon, 01 Jun 2020 16:00:18 -0700 (PDT) MIME-Version: 1.0 References: <20200601032204.124624-1-gthelen@google.com> In-Reply-To: <20200601032204.124624-1-gthelen@google.com> From: Yang Shi Date: Mon, 1 Jun 2020 15:59:56 -0700 Message-ID: Subject: Re: [PATCH] shmem, memcg: enable memcg aware shrinker To: Greg Thelen Cc: Hugh Dickins , Andrew Morton , Kirill Tkhai , Linux MM , Linux Kernel Mailing List , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 8E843180B3C8B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, May 31, 2020 at 8:22 PM Greg Thelen wrote: > > Since v4.19 commit b0dedc49a2da ("mm/vmscan.c: iterate only over charged > shrinkers during memcg shrink_slab()") a memcg aware shrinker is only > called when the per-memcg per-node shrinker_map indicates that the > shrinker may have objects to release to the memcg and node. > > shmem_unused_huge_count and shmem_unused_huge_scan support the per-tmpfs > shrinker which advertises per memcg and numa awareness. The shmem > shrinker releases memory by splitting hugepages that extend beyond > i_size. > > Shmem does not currently set bits in shrinker_map. So, starting with > b0dedc49a2da, memcg reclaim avoids calling the shmem shrinker under > pressure. This leads to undeserved memcg OOM kills. > Example that reliably sees memcg OOM kill in unpatched kernel: > FS=/tmp/fs > CONTAINER=/cgroup/memory/tmpfs_shrinker > mkdir -p $FS > mount -t tmpfs -o huge=always nodev $FS > # Create 1000 MB container, which shouldn't suffer OOM. > mkdir $CONTAINER > echo 1000M > $CONTAINER/memory.limit_in_bytes > echo $BASHPID >> $CONTAINER/cgroup.procs > # Create 4000 files. Ideally each file uses 4k data page + a little > # metadata. Assume 8k total per-file, 32MB (4000*8k) should easily > # fit within container's 1000 MB. But if data pages use 2MB > # hugepages (due to aggressive huge=always) then files consume 8GB, > # which hits memcg 1000 MB limit. > for i in {1..4000}; do > echo . > $FS/$i > done It looks all the inodes which have tail THP beyond i_size are on one single list, then the shrinker actually just splits the first nr_to_scan inodes. But since the list is not memcg aware, so it seems it may split the THPs which are not charged to the victim memcg and the victim memcg still may suffer from pre-mature oom, right? > > v5.4 commit 87eaceb3faa5 ("mm: thp: make deferred split shrinker memcg > aware") maintains the per-node per-memcg shrinker bitmap for THP > shrinker. But there's no such logic in shmem. Make shmem set the > per-memcg per-node shrinker bits when it modifies inodes to have > shrinkable pages. > > Fixes: b0dedc49a2da ("mm/vmscan.c: iterate only over charged shrinkers during memcg shrink_slab()") > Cc: # 4.19+ > Signed-off-by: Greg Thelen > --- > mm/shmem.c | 61 +++++++++++++++++++++++++++++++----------------------- > 1 file changed, 35 insertions(+), 26 deletions(-) > > diff --git a/mm/shmem.c b/mm/shmem.c > index bd8840082c94..e11090f78cb5 100644 > --- a/mm/shmem.c > +++ b/mm/shmem.c > @@ -1002,6 +1002,33 @@ static int shmem_getattr(const struct path *path, struct kstat *stat, > return 0; > } > > +/* > + * Expose inode and optional page to shrinker as having a possibly splittable > + * hugepage that reaches beyond i_size. > + */ > +static void shmem_shrinker_add(struct shmem_sb_info *sbinfo, > + struct inode *inode, struct page *page) > +{ > + struct shmem_inode_info *info = SHMEM_I(inode); > + > + spin_lock(&sbinfo->shrinklist_lock); > + /* > + * _careful to defend against unlocked access to ->shrink_list in > + * shmem_unused_huge_shrink() > + */ > + if (list_empty_careful(&info->shrinklist)) { > + list_add_tail(&info->shrinklist, &sbinfo->shrinklist); > + sbinfo->shrinklist_len++; > + } > + spin_unlock(&sbinfo->shrinklist_lock); > + > +#ifdef CONFIG_MEMCG > + if (page && PageTransHuge(page)) > + memcg_set_shrinker_bit(page->mem_cgroup, page_to_nid(page), > + inode->i_sb->s_shrink.id); > +#endif > +} > + > static int shmem_setattr(struct dentry *dentry, struct iattr *attr) > { > struct inode *inode = d_inode(dentry); > @@ -1048,17 +1075,13 @@ static int shmem_setattr(struct dentry *dentry, struct iattr *attr) > * to shrink under memory pressure. > */ > if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { > - spin_lock(&sbinfo->shrinklist_lock); > - /* > - * _careful to defend against unlocked access to > - * ->shrink_list in shmem_unused_huge_shrink() > - */ > - if (list_empty_careful(&info->shrinklist)) { > - list_add_tail(&info->shrinklist, > - &sbinfo->shrinklist); > - sbinfo->shrinklist_len++; > - } > - spin_unlock(&sbinfo->shrinklist_lock); > + struct page *page; > + > + page = find_get_page(inode->i_mapping, > + (newsize & HPAGE_PMD_MASK) >> PAGE_SHIFT); > + shmem_shrinker_add(sbinfo, inode, page); > + if (page) > + put_page(page); > } > } > } > @@ -1889,21 +1912,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, > if (PageTransHuge(page) && > DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE) < > hindex + HPAGE_PMD_NR - 1) { > - /* > - * Part of the huge page is beyond i_size: subject > - * to shrink under memory pressure. > - */ > - spin_lock(&sbinfo->shrinklist_lock); > - /* > - * _careful to defend against unlocked access to > - * ->shrink_list in shmem_unused_huge_shrink() > - */ > - if (list_empty_careful(&info->shrinklist)) { > - list_add_tail(&info->shrinklist, > - &sbinfo->shrinklist); > - sbinfo->shrinklist_len++; > - } > - spin_unlock(&sbinfo->shrinklist_lock); > + shmem_shrinker_add(sbinfo, inode, page); > } > > /* > -- > 2.27.0.rc0.183.gde8f92d652-goog > >