From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D214C433E3 for ; Tue, 21 Jul 2020 09:26:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7DBE720657 for ; Tue, 21 Jul 2020 09:26:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728852AbgGUJ0m (ORCPT ); Tue, 21 Jul 2020 05:26:42 -0400 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]:48716 "EHLO out30-45.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726892AbgGUJ0l (ORCPT ); Tue, 21 Jul 2020 05:26:41 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R481e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01422;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0U3Oezh-_1595323596; Received: from IT-FVFX43SYHV2H.local(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0U3Oezh-_1595323596) by smtp.aliyun-inc.com(127.0.0.1); Tue, 21 Jul 2020 17:26:36 +0800 Subject: Re: [PATCH v16 16/22] mm/mlock: reorder isolation sequence during munlock To: Alexander Duyck Cc: Andrew Morton , Mel Gorman , Tejun Heo , Hugh Dickins , Konstantin Khlebnikov , Daniel Jordan , Yang Shi , Matthew Wilcox , Johannes Weiner , kbuild test robot , linux-mm , LKML , cgroups@vger.kernel.org, Shakeel Butt , Joonsoo Kim , Wei Yang , "Kirill A. Shutemov" References: <1594429136-20002-1-git-send-email-alex.shi@linux.alibaba.com> <1594429136-20002-17-git-send-email-alex.shi@linux.alibaba.com> <6e37ee32-c6c5-fcc5-3cad-74f7ae41fb67@linux.alibaba.com> From: Alex Shi Message-ID: <7a931661-e096-29ee-d97d-8bf96ba6c972@linux.alibaba.com> Date: Tue, 21 Jul 2020 17:26:34 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2020/7/21 上午2:51, Alexander Duyck 写道: >> Look into the __split_huge_page_tail, there is a tiny gap between tail page >> get PG_mlocked, and it is added into lru list. >> The TestClearPageLRU could blocked memcg changes of the page from stopping >> isolate_lru_page. > I get that there is a gap between the two in __split_huge_page_tail. > My concern is more the fact that you are pulling the bit testing > outside of the locked region when I don't think it needs to be. The > lock is being taken unconditionally, so why pull the testing out when > you could just do it inside the lock anyway? My worry is that you > might be addressing __split_huge_page_tail but in the process you > might be introducing a new race with something like > __pagevec_lru_add_fn. Yes, the page maybe interfered by clear_page_mlock and add pages to wrong lru list. > > If I am not mistaken the Mlocked flag can still be cleared regardless > of if the LRU bit is set or not. So you can still clear the LRU bit > before you pull the page out of the list, but it can be done after > clearing the Mlocked flag instead of before you have even taken the > LRU lock. In that way it would function more similar to how you > handled pagevec_lru_move_fn() as all this function is really doing is > moving the pages out of the unevictable list into one of the other LRU > lists anyway since the Mlocked flag was cleared. > Without the lru bit guard, the page may be moved between memcgs, luckly, lock_page would stop the mem_cgroup_move_account with BUSY state cost. whole new change would like the following, I will testing/resend again. Thanks! Alex @@ -182,7 +179,7 @@ static void __munlock_isolation_failed(struct page *page) unsigned int munlock_vma_page(struct page *page) { int nr_pages; - pg_data_t *pgdat = page_pgdat(page); + struct lruvec *lruvec; /* For try_to_munlock() and to serialize with page migration */ BUG_ON(!PageLocked(page)); @@ -190,11 +187,11 @@ unsigned int munlock_vma_page(struct page *page) VM_BUG_ON_PAGE(PageTail(page), page); /* - * Serialize with any parallel __split_huge_page_refcount() which + * Serialize split tail pages in __split_huge_page_tail() which * might otherwise copy PageMlocked to part of the tail pages before * we clear it in the head page. It also stabilizes hpage_nr_pages(). */ - spin_lock_irq(&pgdat->lru_lock); + lruvec = lock_page_lruvec_irq(page); if (!TestClearPageMlocked(page)) { /* Potentially, PTE-mapped THP: do not skip the rest PTEs */ @@ -205,15 +202,15 @@ unsigned int munlock_vma_page(struct page *page) nr_pages = hpage_nr_pages(page); __mod_zone_page_state(page_zone(page), NR_MLOCK, -nr_pages); - if (__munlock_isolate_lru_page(page, true)) { - spin_unlock_irq(&pgdat->lru_lock); + if (__munlock_isolate_lru_page(page, lruvec, true)) { + unlock_page_lruvec_irq(lruvec); __munlock_isolated_page(page); goto out; } __munlock_isolation_failed(page); unlock_out: - spin_unlock_irq(&pgdat->lru_lock); + unlock_page_lruvec_irq(lruvec); out: return nr_pages - 1; @@ -293,23 +290,27 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) int nr = pagevec_count(pvec); int delta_munlocked = -nr; struct pagevec pvec_putback; + struct lruvec *lruvec = NULL; int pgrescued = 0; pagevec_init(&pvec_putback); /* Phase 1: page isolation */ - spin_lock_irq(&zone->zone_pgdat->lru_lock); for (i = 0; i < nr; i++) { struct page *page = pvec->pages[i]; + /* block memcg change in mem_cgroup_move_account */ + lock_page(page); + lruvec = relock_page_lruvec_irq(page, lruvec); if (TestClearPageMlocked(page)) { /* * We already have pin from follow_page_mask() * so we can spare the get_page() here. */ - if (__munlock_isolate_lru_page(page, false)) + if (__munlock_isolate_lru_page(page, lruvec, false)) { + unlock_page(page); continue; - else + } else __munlock_isolation_failed(page); } else { delta_munlocked++; @@ -321,11 +322,14 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) * pin. We cannot do it under lru_lock however. If it's * the last pin, __page_cache_release() would deadlock. */ + unlock_page(page); pagevec_add(&pvec_putback, pvec->pages[i]); pvec->pages[i] = NULL; } - __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); - spin_unlock_irq(&zone->zone_pgdat->lru_lock); + if (lruvec) { + __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); + unlock_page_lruvec_irq(lruvec); + } /* Now we can release pins of pages that we are not munlocking */ pagevec_release(&pvec_putback); From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Shi Subject: Re: [PATCH v16 16/22] mm/mlock: reorder isolation sequence during munlock Date: Tue, 21 Jul 2020 17:26:34 +0800 Message-ID: <7a931661-e096-29ee-d97d-8bf96ba6c972@linux.alibaba.com> References: <1594429136-20002-1-git-send-email-alex.shi@linux.alibaba.com> <1594429136-20002-17-git-send-email-alex.shi@linux.alibaba.com> <6e37ee32-c6c5-fcc5-3cad-74f7ae41fb67@linux.alibaba.com> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: Sender: cgroups-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-ID: Content-Type: text/plain; charset="utf-8" To: Alexander Duyck Cc: Andrew Morton , Mel Gorman , Tejun Heo , Hugh Dickins , Konstantin Khlebnikov , Daniel Jordan , Yang Shi , Matthew Wilcox , Johannes Weiner , kbuild test robot , linux-mm , LKML , cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Shakeel Butt , Joonsoo Kim , Wei Yang , "Kirill A. Shutemov" 在 2020/7/21 上午2:51, Alexander Duyck 写道: >> Look into the __split_huge_page_tail, there is a tiny gap between tail page >> get PG_mlocked, and it is added into lru list. >> The TestClearPageLRU could blocked memcg changes of the page from stopping >> isolate_lru_page. > I get that there is a gap between the two in __split_huge_page_tail. > My concern is more the fact that you are pulling the bit testing > outside of the locked region when I don't think it needs to be. The > lock is being taken unconditionally, so why pull the testing out when > you could just do it inside the lock anyway? My worry is that you > might be addressing __split_huge_page_tail but in the process you > might be introducing a new race with something like > __pagevec_lru_add_fn. Yes, the page maybe interfered by clear_page_mlock and add pages to wrong lru list. > > If I am not mistaken the Mlocked flag can still be cleared regardless > of if the LRU bit is set or not. So you can still clear the LRU bit > before you pull the page out of the list, but it can be done after > clearing the Mlocked flag instead of before you have even taken the > LRU lock. In that way it would function more similar to how you > handled pagevec_lru_move_fn() as all this function is really doing is > moving the pages out of the unevictable list into one of the other LRU > lists anyway since the Mlocked flag was cleared. > Without the lru bit guard, the page may be moved between memcgs, luckly, lock_page would stop the mem_cgroup_move_account with BUSY state cost. whole new change would like the following, I will testing/resend again. Thanks! Alex @@ -182,7 +179,7 @@ static void __munlock_isolation_failed(struct page *page) unsigned int munlock_vma_page(struct page *page) { int nr_pages; - pg_data_t *pgdat = page_pgdat(page); + struct lruvec *lruvec; /* For try_to_munlock() and to serialize with page migration */ BUG_ON(!PageLocked(page)); @@ -190,11 +187,11 @@ unsigned int munlock_vma_page(struct page *page) VM_BUG_ON_PAGE(PageTail(page), page); /* - * Serialize with any parallel __split_huge_page_refcount() which + * Serialize split tail pages in __split_huge_page_tail() which * might otherwise copy PageMlocked to part of the tail pages before * we clear it in the head page. It also stabilizes hpage_nr_pages(). */ - spin_lock_irq(&pgdat->lru_lock); + lruvec = lock_page_lruvec_irq(page); if (!TestClearPageMlocked(page)) { /* Potentially, PTE-mapped THP: do not skip the rest PTEs */ @@ -205,15 +202,15 @@ unsigned int munlock_vma_page(struct page *page) nr_pages = hpage_nr_pages(page); __mod_zone_page_state(page_zone(page), NR_MLOCK, -nr_pages); - if (__munlock_isolate_lru_page(page, true)) { - spin_unlock_irq(&pgdat->lru_lock); + if (__munlock_isolate_lru_page(page, lruvec, true)) { + unlock_page_lruvec_irq(lruvec); __munlock_isolated_page(page); goto out; } __munlock_isolation_failed(page); unlock_out: - spin_unlock_irq(&pgdat->lru_lock); + unlock_page_lruvec_irq(lruvec); out: return nr_pages - 1; @@ -293,23 +290,27 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) int nr = pagevec_count(pvec); int delta_munlocked = -nr; struct pagevec pvec_putback; + struct lruvec *lruvec = NULL; int pgrescued = 0; pagevec_init(&pvec_putback); /* Phase 1: page isolation */ - spin_lock_irq(&zone->zone_pgdat->lru_lock); for (i = 0; i < nr; i++) { struct page *page = pvec->pages[i]; + /* block memcg change in mem_cgroup_move_account */ + lock_page(page); + lruvec = relock_page_lruvec_irq(page, lruvec); if (TestClearPageMlocked(page)) { /* * We already have pin from follow_page_mask() * so we can spare the get_page() here. */ - if (__munlock_isolate_lru_page(page, false)) + if (__munlock_isolate_lru_page(page, lruvec, false)) { + unlock_page(page); continue; - else + } else __munlock_isolation_failed(page); } else { delta_munlocked++; @@ -321,11 +322,14 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) * pin. We cannot do it under lru_lock however. If it's * the last pin, __page_cache_release() would deadlock. */ + unlock_page(page); pagevec_add(&pvec_putback, pvec->pages[i]); pvec->pages[i] = NULL; } - __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); - spin_unlock_irq(&zone->zone_pgdat->lru_lock); + if (lruvec) { + __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); + unlock_page_lruvec_irq(lruvec); + } /* Now we can release pins of pages that we are not munlocking */ pagevec_release(&pvec_putback);