From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48C0AC433DF for ; Wed, 19 Aug 2020 07:58:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDD6E20772 for ; Wed, 19 Aug 2020 07:58:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD6E20772 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8BBDA8D000C; Wed, 19 Aug 2020 03:58:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8459F8D0001; Wed, 19 Aug 2020 03:58:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70DA78D000C; Wed, 19 Aug 2020 03:58:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id 5755A8D0001 for ; Wed, 19 Aug 2020 03:58:17 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 22641362E for ; Wed, 19 Aug 2020 07:58:17 +0000 (UTC) X-FDA: 77166565434.18.wish82_3c14e1427026 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id E6891100EC664 for ; Wed, 19 Aug 2020 07:58:16 +0000 (UTC) X-HE-Tag: wish82_3c14e1427026 X-Filterd-Recvd-Size: 6807 Received: from out30-56.freemail.mail.aliyun.com (out30-56.freemail.mail.aliyun.com [115.124.30.56]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Wed, 19 Aug 2020 07:58:06 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R881e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07488;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=19;SR=0;TI=SMTPD_---0U6D5.Xk_1597823875; Received: from IT-FVFX43SYHV2H.local(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0U6D5.Xk_1597823875) by smtp.aliyun-inc.com(127.0.0.1); Wed, 19 Aug 2020 15:57:57 +0800 Subject: Re: [RFC PATCH v2 5/5] mm: Split move_pages_to_lru into 3 separate passes To: Alexander Duyck Cc: yang.shi@linux.alibaba.com, lkp@intel.com, rong.a.chen@intel.com, khlebnikov@yandex-team.ru, kirill@shutemov.name, hughd@google.com, linux-kernel@vger.kernel.org, daniel.m.jordan@oracle.com, linux-mm@kvack.org, shakeelb@google.com, willy@infradead.org, hannes@cmpxchg.org, tj@kernel.org, cgroups@vger.kernel.org, akpm@linux-foundation.org, richard.weiyang@gmail.com, mgorman@techsingularity.net, iamjoonsoo.kim@lge.com References: <20200819041852.23414.95939.stgit@localhost.localdomain> <20200819042738.23414.60815.stgit@localhost.localdomain> From: Alex Shi Message-ID: <084c58a7-7aac-820c-9606-19391c35b9b5@linux.alibaba.com> Date: Wed, 19 Aug 2020 15:56:43 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200819042738.23414.60815.stgit@localhost.localdomain> Content-Type: text/plain; charset=utf-8 X-Rspamd-Queue-Id: E6891100EC664 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: =E5=9C=A8 2020/8/19 =E4=B8=8B=E5=8D=8812:27, Alexander Duyck =E5=86=99=E9= =81=93: > From: Alexander Duyck >=20 > The current code for move_pages_to_lru is meant to release the LRU lock > every time it encounters an unevictable page or a compound page that mu= st > be freed. This results in a fair amount of code bulk because the lruvec= has > to be reacquired every time the lock is released and reacquired. >=20 > Instead of doing this I believe we can break the code up into 3 passes.= The > first pass will identify the pages we can move to LRU and move those. I= n > addition it will sort the list out leaving the unevictable pages in the > list and moving those pages that have dropped to a reference count of 0= to > pages_to_free. The second pass will return the unevictable pages to the > LRU. The final pass will free any compound pages we have in the > pages_to_free list before we merge it back with the original list and > return from the function. >=20 > The advantage of doing it this way is that we only have to release the = lock > between pass 1 and 2, and then we reacquire the lock after pass 3 after= we > merge the pages_to_free back into the original list. As such we only ha= ve > to release the lock at most once in an entire call instead of having to > test to see if we need to relock with each page. >=20 > Signed-off-by: Alexander Duyck > --- > mm/vmscan.c | 68 ++++++++++++++++++++++++++++++++++-----------------= -------- > 1 file changed, 39 insertions(+), 29 deletions(-) >=20 > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 3ebe3f9b653b..6a2bdbc1a9eb 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -1850,22 +1850,21 @@ static unsigned noinline_for_stack move_pages_t= o_lru(struct lruvec *lruvec, > { > int nr_pages, nr_moved =3D 0; > LIST_HEAD(pages_to_free); > - struct page *page; > - struct lruvec *orig_lruvec =3D lruvec; > + struct page *page, *next; > enum lru_list lru; > =20 > - while (!list_empty(list)) { > - page =3D lru_to_page(list); > + list_for_each_entry_safe(page, next, list, lru) { > VM_BUG_ON_PAGE(PageLRU(page), page); > - list_del(&page->lru); > - if (unlikely(!page_evictable(page))) { > - if (lruvec) { > - spin_unlock_irq(&lruvec->lru_lock); > - lruvec =3D NULL; > - } > - putback_lru_page(page); > + > + /* > + * if page is unevictable leave it on the list to be returned > + * to the LRU after we have finished processing the other > + * entries in the list. > + */ > + if (unlikely(!page_evictable(page))) > continue; > - } > + > + list_del(&page->lru); > =20 > /* > * The SetPageLRU needs to be kept here for list intergrity. > @@ -1878,20 +1877,14 @@ static unsigned noinline_for_stack move_pages_t= o_lru(struct lruvec *lruvec, > * list_add(&page->lru,) > * list_add(&page->lru,) > */ > - lruvec =3D relock_page_lruvec_irq(page, lruvec); It's actually changed the meaning from current func. which I had seen a b= ug if no relock. but after move to 5.9 kernel, I can not reprodce the bug any more. I am n= ot sure if 5.9 fixed=20 the problem, and we don't need relock here.=20 For the rest of this patch.=20 Reviewed-by: Alex Shi > SetPageLRU(page); > =20 > if (unlikely(put_page_testzero(page))) { > __ClearPageLRU(page); > __ClearPageActive(page); > =20 > - if (unlikely(PageCompound(page))) { > - spin_unlock_irq(&lruvec->lru_lock); > - lruvec =3D NULL; > - destroy_compound_page(page); > - } else > - list_add(&page->lru, &pages_to_free); > - > + /* defer freeing until we can release lru_lock */ > + list_add(&page->lru, &pages_to_free); > continue; > } > =20 > @@ -1904,16 +1897,33 @@ static unsigned noinline_for_stack move_pages_t= o_lru(struct lruvec *lruvec, > if (PageActive(page)) > workingset_age_nonresident(lruvec, nr_pages); > } > - if (orig_lruvec !=3D lruvec) { > - if (lruvec) > - spin_unlock_irq(&lruvec->lru_lock); > - spin_lock_irq(&orig_lruvec->lru_lock); > - } > =20 > - /* > - * To save our caller's stack, now use input list for pages to free. > - */ > - list_splice(&pages_to_free, list); > + if (unlikely(!list_empty(list) || !list_empty(&pages_to_free))) { > + spin_unlock_irq(&lruvec->lru_lock); > + > + /* return any unevictable pages to the LRU list */ > + while (!list_empty(list)) { > + page =3D lru_to_page(list); > + list_del(&page->lru); > + putback_lru_page(page); > + } > + > + /* > + * To save our caller's stack use input > + * list for pages to free. > + */ > + list_splice(&pages_to_free, list); > + > + /* free any compound pages we have in the list */ > + list_for_each_entry_safe(page, next, list, lru) { > + if (likely(!PageCompound(page))) > + continue; > + list_del(&page->lru); > + destroy_compound_page(page); > + } > + > + spin_lock_irq(&lruvec->lru_lock); > + } > =20 > return nr_moved; > } >=20