From: Alex Shi <alex.shi@linux.alibaba.com> To: Alexander Duyck <alexander.duyck@gmail.com> Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@techsingularity.net>, Tejun Heo <tj@kernel.org>, Hugh Dickins <hughd@google.com>, Konstantin Khlebnikov <khlebnikov@yandex-team.ru>, Daniel Jordan <daniel.m.jordan@oracle.com>, Yang Shi <yang.shi@linux.alibaba.com>, Matthew Wilcox <willy@infradead.org>, Johannes Weiner <hannes@cmpxchg.org>, kbuild test robot <lkp@intel.com>, linux-mm <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org>, cgroups@vger.kernel.org, Shakeel Butt <shakeelb@google.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Wei Yang <richard.weiyang@gmail.com>, "Kirill A. Shutemov" <kirill@shutemov.name>, Michal Hocko <mhocko@kernel.org>, Vladimir Davydov <vdavydov.dev@gmail.com>, Rong Chen <rong.a.chen@intel.com> Subject: Re: [PATCH v16 18/22] mm/lru: replace pgdat lru_lock with lruvec lock Date: Sun, 19 Jul 2020 17:12:23 +0800 [thread overview] Message-ID: <c339f46e-ae04-4e65-2713-a5c8be56051a@linux.alibaba.com> (raw) In-Reply-To: <62dfd262-a7ac-d18e-216a-2988c690b256@linux.alibaba.com> 在 2020/7/18 下午10:15, Alex Shi 写道: >>> >>> struct wb_domain *mem_cgroup_wb_domain(struct bdi_writeback *wb); >>> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h >>> index 14c668b7e793..36c1680efd90 100644 >>> --- a/include/linux/mmzone.h >>> +++ b/include/linux/mmzone.h >>> @@ -261,6 +261,8 @@ struct lruvec { >>> atomic_long_t nonresident_age; >>> /* Refaults at the time of last reclaim cycle */ >>> unsigned long refaults; >>> + /* per lruvec lru_lock for memcg */ >>> + spinlock_t lru_lock; >>> /* Various lruvec state flags (enum lruvec_flags) */ >>> unsigned long flags; >> Any reason for placing this here instead of at the end of the >> structure? From what I can tell it looks like lruvec is already 128B >> long so placing the lock on the end would put it into the next >> cacheline which may provide some performance benefit since it is >> likely to be bounced quite a bit. > Rong Chen(Cced) once reported a performance regression when the lock at > the end of struct, and move here could remove it. > Although I can't not reproduce. But I trust his report. > Oops, Rong's report is on another member which is different with current struct. Compare to move to tail, how about to move it to head of struct, which is close to lru list? Did you have some data of the place change? Thanks Alex > ... > >>> putback: >>> - spin_unlock_irq(&zone->zone_pgdat->lru_lock); >>> pagevec_add(&pvec_putback, pvec->pages[i]); >>> pvec->pages[i] = NULL; >>> } >>> - /* tempary disable irq, will remove later */ >>> - local_irq_disable(); >>> __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); >>> - local_irq_enable(); >>> + if (lruvec) >>> + unlock_page_lruvec_irq(lruvec); >> So I am not a fan of this change. You went to all the trouble of >> reducing the lock scope just to bring it back out here again. In >> addition it implies there is a path where you might try to update the >> page state without disabling interrupts. > Right. but any idea to avoid this except a extra local_irq_disable? > The following changes would resolve the problem. Is this ok? @@ -324,7 +322,8 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) pagevec_add(&pvec_putback, pvec->pages[i]); pvec->pages[i] = NULL; } - __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); + if (delta_munlocked) + __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); if (lruvec) unlock_page_lruvec_irq(lruvec);
WARNING: multiple messages have this Message-ID (diff)
From: Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> To: Alexander Duyck <alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Mel Gorman <mgorman-3eNAlZScCAx27rWaFMvyedHuzzzSOjJt@public.gmane.org>, Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Konstantin Khlebnikov <khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org>, Daniel Jordan <daniel.m.jordan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>, Yang Shi <yang.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>, Matthew Wilcox <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>, kbuild test robot <lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>, linux-mm <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Joonsoo Kim <iamjoonsoo.kim-Hm3cg6mZ9cc@public.gmane.org>, Wei Yang <richard.weiyang-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, "Kirill A. Shutemov" <kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org>, Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Vladimir Davydov <vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Rong Chen <rong.a.chen-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> Subject: Re: [PATCH v16 18/22] mm/lru: replace pgdat lru_lock with lruvec lock Date: Sun, 19 Jul 2020 17:12:23 +0800 [thread overview] Message-ID: <c339f46e-ae04-4e65-2713-a5c8be56051a@linux.alibaba.com> (raw) In-Reply-To: <62dfd262-a7ac-d18e-216a-2988c690b256-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> 在 2020/7/18 下午10:15, Alex Shi 写道: >>> >>> struct wb_domain *mem_cgroup_wb_domain(struct bdi_writeback *wb); >>> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h >>> index 14c668b7e793..36c1680efd90 100644 >>> --- a/include/linux/mmzone.h >>> +++ b/include/linux/mmzone.h >>> @@ -261,6 +261,8 @@ struct lruvec { >>> atomic_long_t nonresident_age; >>> /* Refaults at the time of last reclaim cycle */ >>> unsigned long refaults; >>> + /* per lruvec lru_lock for memcg */ >>> + spinlock_t lru_lock; >>> /* Various lruvec state flags (enum lruvec_flags) */ >>> unsigned long flags; >> Any reason for placing this here instead of at the end of the >> structure? From what I can tell it looks like lruvec is already 128B >> long so placing the lock on the end would put it into the next >> cacheline which may provide some performance benefit since it is >> likely to be bounced quite a bit. > Rong Chen(Cced) once reported a performance regression when the lock at > the end of struct, and move here could remove it. > Although I can't not reproduce. But I trust his report. > Oops, Rong's report is on another member which is different with current struct. Compare to move to tail, how about to move it to head of struct, which is close to lru list? Did you have some data of the place change? Thanks Alex > ... > >>> putback: >>> - spin_unlock_irq(&zone->zone_pgdat->lru_lock); >>> pagevec_add(&pvec_putback, pvec->pages[i]); >>> pvec->pages[i] = NULL; >>> } >>> - /* tempary disable irq, will remove later */ >>> - local_irq_disable(); >>> __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); >>> - local_irq_enable(); >>> + if (lruvec) >>> + unlock_page_lruvec_irq(lruvec); >> So I am not a fan of this change. You went to all the trouble of >> reducing the lock scope just to bring it back out here again. In >> addition it implies there is a path where you might try to update the >> page state without disabling interrupts. > Right. but any idea to avoid this except a extra local_irq_disable? > The following changes would resolve the problem. Is this ok? @@ -324,7 +322,8 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone) pagevec_add(&pvec_putback, pvec->pages[i]); pvec->pages[i] = NULL; } - __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); + if (delta_munlocked) + __mod_zone_page_state(zone, NR_MLOCK, delta_munlocked); if (lruvec) unlock_page_lruvec_irq(lruvec);
next prev parent reply other threads:[~2020-07-19 9:12 UTC|newest] Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-07-11 0:58 [PATCH v16 00/22] per memcg lru_lock Alex Shi 2020-07-11 0:58 ` [PATCH v16 01/22] mm/vmscan: remove unnecessary lruvec adding Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 02/22] mm/page_idle: no unlikely double check for idle page counting Alex Shi 2020-07-11 0:58 ` [PATCH v16 03/22] mm/compaction: correct the comments of compact_defer_shift Alex Shi 2020-07-11 0:58 ` [PATCH v16 04/22] mm/compaction: rename compact_deferred as compact_should_defer Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 05/22] mm/thp: move lru_add_page_tail func to huge_memory.c Alex Shi 2020-07-16 8:59 ` Alex Shi 2020-07-16 8:59 ` Alex Shi 2020-07-16 13:17 ` Kirill A. Shutemov 2020-07-16 13:17 ` Kirill A. Shutemov 2020-07-17 5:13 ` Alex Shi 2020-07-17 5:13 ` Alex Shi 2020-07-20 8:37 ` Kirill A. Shutemov 2020-07-20 8:37 ` Kirill A. Shutemov 2020-07-11 0:58 ` [PATCH v16 06/22] mm/thp: clean up lru_add_page_tail Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-20 8:43 ` Kirill A. Shutemov 2020-07-20 8:43 ` Kirill A. Shutemov 2020-07-11 0:58 ` [PATCH v16 07/22] mm/thp: remove code path which never got into Alex Shi 2020-07-20 8:43 ` Kirill A. Shutemov 2020-07-20 8:43 ` Kirill A. Shutemov 2020-07-11 0:58 ` [PATCH v16 08/22] mm/thp: narrow lru locking Alex Shi 2020-07-11 0:58 ` [PATCH v16 09/22] mm/memcg: add debug checking in lock_page_memcg Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 10/22] mm/swap: fold vm event PGROTATED into pagevec_move_tail_fn Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 11/22] mm/lru: move lru_lock holding in func lru_note_cost_page Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 12/22] mm/lru: move lock into lru_note_cost Alex Shi 2020-07-11 0:58 ` [PATCH v16 13/22] mm/lru: introduce TestClearPageLRU Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-16 9:06 ` Alex Shi 2020-07-16 9:06 ` Alex Shi 2020-07-16 21:12 ` Alexander Duyck 2020-07-16 21:12 ` Alexander Duyck 2020-07-16 21:12 ` Alexander Duyck 2020-07-17 7:45 ` Alex Shi 2020-07-17 7:45 ` Alex Shi 2020-07-17 18:26 ` Alexander Duyck 2020-07-17 18:26 ` Alexander Duyck 2020-07-19 4:45 ` Alex Shi 2020-07-19 4:45 ` Alex Shi 2020-07-19 11:24 ` Alex Shi 2020-07-19 11:24 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 14/22] mm/thp: add tail pages into lru anyway in split_huge_page() Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-17 9:30 ` Alex Shi 2020-07-17 9:30 ` Alex Shi 2020-07-20 8:49 ` Kirill A. Shutemov 2020-07-20 8:49 ` Kirill A. Shutemov 2020-07-20 9:04 ` Alex Shi 2020-07-20 9:04 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 15/22] mm/compaction: do page isolation first in compaction Alex Shi 2020-07-16 21:32 ` Alexander Duyck 2020-07-16 21:32 ` Alexander Duyck 2020-07-16 21:32 ` Alexander Duyck 2020-07-17 5:09 ` Alex Shi 2020-07-17 5:09 ` Alex Shi 2020-07-17 16:09 ` Alexander Duyck 2020-07-17 16:09 ` Alexander Duyck 2020-07-17 16:09 ` Alexander Duyck 2020-07-19 3:59 ` Alex Shi 2020-07-19 3:59 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 16/22] mm/mlock: reorder isolation sequence during munlock Alex Shi 2020-07-17 20:30 ` Alexander Duyck 2020-07-17 20:30 ` Alexander Duyck 2020-07-17 20:30 ` Alexander Duyck 2020-07-19 3:55 ` Alex Shi 2020-07-19 3:55 ` Alex Shi 2020-07-20 18:51 ` Alexander Duyck 2020-07-20 18:51 ` Alexander Duyck 2020-07-20 18:51 ` Alexander Duyck 2020-07-21 9:26 ` Alex Shi 2020-07-21 9:26 ` Alex Shi 2020-07-21 13:51 ` Alex Shi 2020-07-21 13:51 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 17/22] mm/swap: serialize memcg changes during pagevec_lru_move_fn Alex Shi 2020-07-11 0:58 ` [PATCH v16 18/22] mm/lru: replace pgdat lru_lock with lruvec lock Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-17 21:38 ` Alexander Duyck 2020-07-17 21:38 ` Alexander Duyck 2020-07-17 21:38 ` Alexander Duyck 2020-07-18 14:15 ` Alex Shi 2020-07-19 9:12 ` Alex Shi [this message] 2020-07-19 9:12 ` Alex Shi 2020-07-19 15:14 ` Alexander Duyck 2020-07-19 15:14 ` Alexander Duyck 2020-07-19 15:14 ` Alexander Duyck 2020-07-20 5:47 ` Alex Shi 2020-07-20 5:47 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 19/22] mm/lru: introduce the relock_page_lruvec function Alex Shi 2020-07-11 0:58 ` Alex Shi 2020-07-17 22:03 ` Alexander Duyck 2020-07-17 22:03 ` Alexander Duyck 2020-07-18 14:01 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 20/22] mm/vmscan: use relock for move_pages_to_lru Alex Shi 2020-07-17 21:44 ` Alexander Duyck 2020-07-17 21:44 ` Alexander Duyck 2020-07-17 21:44 ` Alexander Duyck 2020-07-18 14:15 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 21/22] mm/pgdat: remove pgdat lru_lock Alex Shi 2020-07-17 21:09 ` Alexander Duyck 2020-07-17 21:09 ` Alexander Duyck 2020-07-18 14:17 ` Alex Shi 2020-07-18 14:17 ` Alex Shi 2020-07-11 0:58 ` [PATCH v16 22/22] mm/lru: revise the comments of lru_lock Alex Shi 2020-07-11 1:02 ` [PATCH v16 00/22] per memcg lru_lock Alex Shi 2020-07-11 1:02 ` Alex Shi 2020-07-16 8:49 ` Alex Shi 2020-07-16 14:11 ` Alexander Duyck 2020-07-16 14:11 ` Alexander Duyck 2020-07-16 14:11 ` Alexander Duyck 2020-07-17 5:24 ` Alex Shi 2020-07-17 5:24 ` Alex Shi 2020-07-19 15:23 ` Hugh Dickins 2020-07-19 15:23 ` Hugh Dickins 2020-07-20 3:01 ` Alex Shi 2020-07-20 3:01 ` Alex Shi 2020-07-20 4:47 ` Hugh Dickins 2020-07-20 4:47 ` Hugh Dickins 2020-07-20 4:47 ` Hugh Dickins 2020-07-20 7:30 ` Alex Shi 2020-07-20 7:30 ` Alex Shi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=c339f46e-ae04-4e65-2713-a5c8be56051a@linux.alibaba.com \ --to=alex.shi@linux.alibaba.com \ --cc=akpm@linux-foundation.org \ --cc=alexander.duyck@gmail.com \ --cc=cgroups@vger.kernel.org \ --cc=daniel.m.jordan@oracle.com \ --cc=hannes@cmpxchg.org \ --cc=hughd@google.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=khlebnikov@yandex-team.ru \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lkp@intel.com \ --cc=mgorman@techsingularity.net \ --cc=mhocko@kernel.org \ --cc=richard.weiyang@gmail.com \ --cc=rong.a.chen@intel.com \ --cc=shakeelb@google.com \ --cc=tj@kernel.org \ --cc=vdavydov.dev@gmail.com \ --cc=willy@infradead.org \ --cc=yang.shi@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.