From: Alex Shi <alex.shi@linux.alibaba.com> To: akpm@linux-foundation.org, mgorman@techsingularity.net, tj@kernel.org, hughd@google.com, khlebnikov@yandex-team.ru, daniel.m.jordan@oracle.com, yang.shi@linux.alibaba.com, willy@infradead.org, hannes@cmpxchg.org, lkp@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, shakeelb@google.com, iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com, kirill@shutemov.name, alexander.duyck@gmail.com, rong.a.chen@intel.com Subject: [PATCH v17 12/21] mm/lru: move lock into lru_note_cost Date: Sat, 25 Jul 2020 20:59:49 +0800 [thread overview] Message-ID: <1595681998-19193-13-git-send-email-alex.shi@linux.alibaba.com> (raw) In-Reply-To: <1595681998-19193-1-git-send-email-alex.shi@linux.alibaba.com> This patch move lru_lock into lru_note_cost. It's a bit ugly and may cost more locking, but it's necessary for later per pgdat lru_lock to per memcg lru_lock change. Signed-off-by: Alex Shi <alex.shi@linux.alibaba.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/swap.c | 5 +++-- mm/vmscan.c | 4 +--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index b88ca630db70..f645965fde0e 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -269,7 +269,9 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) { do { unsigned long lrusize; + struct pglist_data *pgdat = lruvec_pgdat(lruvec); + spin_lock_irq(&pgdat->lru_lock); /* Record cost event */ if (file) lruvec->file_cost += nr_pages; @@ -293,15 +295,14 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) lruvec->file_cost /= 2; lruvec->anon_cost /= 2; } + spin_unlock_irq(&pgdat->lru_lock); } while ((lruvec = parent_lruvec(lruvec))); } void lru_note_cost_page(struct page *page) { - spin_lock_irq(&page_pgdat(page)->lru_lock); lru_note_cost(mem_cgroup_page_lruvec(page, page_pgdat(page)), page_is_file_lru(page), hpage_nr_pages(page)); - spin_unlock_irq(&page_pgdat(page)->lru_lock); } static void __activate_page(struct page *page, struct lruvec *lruvec) diff --git a/mm/vmscan.c b/mm/vmscan.c index ddb29d813d77..c1c4259b4de5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1976,19 +1976,17 @@ static int current_may_throttle(void) &stat, false); spin_lock_irq(&pgdat->lru_lock); - move_pages_to_lru(lruvec, &page_list); __mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, -nr_taken); - lru_note_cost(lruvec, file, stat.nr_pageout); item = current_is_kswapd() ? PGSTEAL_KSWAPD : PGSTEAL_DIRECT; if (!cgroup_reclaim(sc)) __count_vm_events(item, nr_reclaimed); __count_memcg_events(lruvec_memcg(lruvec), item, nr_reclaimed); __count_vm_events(PGSTEAL_ANON + file, nr_reclaimed); - spin_unlock_irq(&pgdat->lru_lock); + lru_note_cost(lruvec, file, stat.nr_pageout); mem_cgroup_uncharge_list(&page_list); free_unref_page_list(&page_list); -- 1.8.3.1
WARNING: multiple messages have this Message-ID (diff)
From: Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> To: akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, mgorman-3eNAlZScCAx27rWaFMvyedHuzzzSOjJt@public.gmane.org, tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org, daniel.m.jordan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org, yang.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org, willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org, hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org, lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, iamjoonsoo.kim-Hm3cg6mZ9cc@public.gmane.org, richard.weiyang-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org, alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, rong.a.chen-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org Subject: [PATCH v17 12/21] mm/lru: move lock into lru_note_cost Date: Sat, 25 Jul 2020 20:59:49 +0800 [thread overview] Message-ID: <1595681998-19193-13-git-send-email-alex.shi@linux.alibaba.com> (raw) In-Reply-To: <1595681998-19193-1-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> This patch move lru_lock into lru_note_cost. It's a bit ugly and may cost more locking, but it's necessary for later per pgdat lru_lock to per memcg lru_lock change. Signed-off-by: Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> Cc: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org --- mm/swap.c | 5 +++-- mm/vmscan.c | 4 +--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index b88ca630db70..f645965fde0e 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -269,7 +269,9 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) { do { unsigned long lrusize; + struct pglist_data *pgdat = lruvec_pgdat(lruvec); + spin_lock_irq(&pgdat->lru_lock); /* Record cost event */ if (file) lruvec->file_cost += nr_pages; @@ -293,15 +295,14 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages) lruvec->file_cost /= 2; lruvec->anon_cost /= 2; } + spin_unlock_irq(&pgdat->lru_lock); } while ((lruvec = parent_lruvec(lruvec))); } void lru_note_cost_page(struct page *page) { - spin_lock_irq(&page_pgdat(page)->lru_lock); lru_note_cost(mem_cgroup_page_lruvec(page, page_pgdat(page)), page_is_file_lru(page), hpage_nr_pages(page)); - spin_unlock_irq(&page_pgdat(page)->lru_lock); } static void __activate_page(struct page *page, struct lruvec *lruvec) diff --git a/mm/vmscan.c b/mm/vmscan.c index ddb29d813d77..c1c4259b4de5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1976,19 +1976,17 @@ static int current_may_throttle(void) &stat, false); spin_lock_irq(&pgdat->lru_lock); - move_pages_to_lru(lruvec, &page_list); __mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, -nr_taken); - lru_note_cost(lruvec, file, stat.nr_pageout); item = current_is_kswapd() ? PGSTEAL_KSWAPD : PGSTEAL_DIRECT; if (!cgroup_reclaim(sc)) __count_vm_events(item, nr_reclaimed); __count_memcg_events(lruvec_memcg(lruvec), item, nr_reclaimed); __count_vm_events(PGSTEAL_ANON + file, nr_reclaimed); - spin_unlock_irq(&pgdat->lru_lock); + lru_note_cost(lruvec, file, stat.nr_pageout); mem_cgroup_uncharge_list(&page_list); free_unref_page_list(&page_list); -- 1.8.3.1
next prev parent reply other threads:[~2020-07-25 13:01 UTC|newest] Thread overview: 209+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-07-25 12:59 [PATCH v17 00/21] per memcg lru lock Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 01/21] mm/vmscan: remove unnecessary lruvec adding Alex Shi 2020-08-06 3:47 ` Alex Shi 2020-08-06 3:47 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 02/21] mm/page_idle: no unlikely double check for idle page counting Alex Shi 2020-07-25 12:59 ` [PATCH v17 03/21] mm/compaction: correct the comments of compact_defer_shift Alex Shi 2020-07-27 17:29 ` Alexander Duyck 2020-07-27 17:29 ` Alexander Duyck 2020-07-28 11:59 ` Alex Shi 2020-07-28 11:59 ` Alex Shi 2020-07-28 14:17 ` Alexander Duyck 2020-07-28 14:17 ` Alexander Duyck 2020-07-28 14:17 ` Alexander Duyck 2020-07-25 12:59 ` [PATCH v17 04/21] mm/compaction: rename compact_deferred as compact_should_defer Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 05/21] mm/thp: move lru_add_page_tail func to huge_memory.c Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 06/21] mm/thp: clean up lru_add_page_tail Alex Shi 2020-07-25 12:59 ` [PATCH v17 07/21] mm/thp: remove code path which never got into Alex Shi 2020-07-25 12:59 ` [PATCH v17 08/21] mm/thp: narrow lru locking Alex Shi 2020-07-25 12:59 ` [PATCH v17 09/21] mm/memcg: add debug checking in lock_page_memcg Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 10/21] mm/swap: fold vm event PGROTATED into pagevec_move_tail_fn Alex Shi 2020-07-25 12:59 ` [PATCH v17 11/21] mm/lru: move lru_lock holding in func lru_note_cost_page Alex Shi 2020-08-05 21:18 ` Alexander Duyck 2020-08-05 21:18 ` Alexander Duyck 2020-08-05 21:18 ` Alexander Duyck 2020-07-25 12:59 ` Alex Shi [this message] 2020-07-25 12:59 ` [PATCH v17 12/21] mm/lru: move lock into lru_note_cost Alex Shi 2020-07-25 12:59 ` [PATCH v17 13/21] mm/lru: introduce TestClearPageLRU Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-29 3:53 ` Alex Shi 2020-07-29 3:53 ` Alex Shi 2020-08-05 22:43 ` Alexander Duyck 2020-08-05 22:43 ` Alexander Duyck 2020-08-06 1:54 ` Alex Shi 2020-08-06 1:54 ` Alex Shi 2020-08-06 14:41 ` Alexander Duyck 2020-08-06 14:41 ` Alexander Duyck 2020-07-25 12:59 ` [PATCH v17 14/21] mm/compaction: do page isolation first in compaction Alex Shi 2020-08-04 21:35 ` Alexander Duyck 2020-08-04 21:35 ` Alexander Duyck 2020-08-04 21:35 ` Alexander Duyck 2020-08-06 18:38 ` Alexander Duyck 2020-08-06 18:38 ` Alexander Duyck 2020-08-06 18:38 ` Alexander Duyck 2020-08-07 3:24 ` Alex Shi 2020-08-07 3:24 ` Alex Shi 2020-08-07 14:51 ` Alexander Duyck 2020-08-07 14:51 ` Alexander Duyck 2020-08-07 14:51 ` Alexander Duyck 2020-08-10 13:10 ` Alex Shi 2020-08-10 13:10 ` Alex Shi 2020-08-10 14:41 ` Alexander Duyck 2020-08-10 14:41 ` Alexander Duyck 2020-08-10 14:41 ` Alexander Duyck 2020-08-11 8:22 ` Alex Shi 2020-08-11 14:47 ` Alexander Duyck 2020-08-11 14:47 ` Alexander Duyck 2020-08-11 14:47 ` Alexander Duyck 2020-08-12 11:43 ` Alex Shi 2020-08-12 12:16 ` Alex Shi 2020-08-12 12:16 ` Alex Shi 2020-08-12 16:51 ` Alexander Duyck 2020-08-12 16:51 ` Alexander Duyck 2020-08-12 16:51 ` Alexander Duyck 2020-08-13 1:46 ` Alex Shi 2020-08-13 2:17 ` Alexander Duyck 2020-08-13 2:17 ` Alexander Duyck 2020-08-13 2:17 ` Alexander Duyck 2020-08-13 3:52 ` Alex Shi 2020-08-13 3:52 ` Alex Shi 2020-08-13 4:02 ` [RFC PATCH 0/3] " Alexander Duyck 2020-08-13 4:02 ` Alexander Duyck 2020-08-13 4:02 ` [RFC PATCH 1/3] mm: Drop locked from isolate_migratepages_block Alexander Duyck 2020-08-13 4:02 ` Alexander Duyck 2020-08-13 6:56 ` Alex Shi 2020-08-13 6:56 ` Alex Shi 2020-08-13 14:32 ` Alexander Duyck 2020-08-13 14:32 ` Alexander Duyck 2020-08-13 14:32 ` Alexander Duyck 2020-08-14 7:25 ` Alex Shi 2020-08-14 7:25 ` Alex Shi 2020-08-13 7:44 ` Alex Shi 2020-08-13 7:44 ` Alex Shi 2020-08-13 14:26 ` Alexander Duyck 2020-08-13 14:26 ` Alexander Duyck 2020-08-13 4:02 ` [RFC PATCH 2/3] mm: Drop use of test_and_set_skip in favor of just setting skip Alexander Duyck 2020-08-13 4:02 ` Alexander Duyck 2020-08-14 7:19 ` Alex Shi 2020-08-14 7:19 ` Alex Shi 2020-08-14 14:24 ` Alexander Duyck 2020-08-14 14:24 ` Alexander Duyck 2020-08-14 14:24 ` Alexander Duyck 2020-08-14 21:15 ` Alexander Duyck 2020-08-14 21:15 ` Alexander Duyck 2020-08-14 21:15 ` Alexander Duyck 2020-08-15 9:49 ` Alex Shi 2020-08-17 15:38 ` Alexander Duyck 2020-08-17 15:38 ` Alexander Duyck 2020-08-17 15:38 ` Alexander Duyck 2020-08-18 6:50 ` Alex Shi 2020-08-18 6:50 ` Alex Shi 2020-08-13 4:02 ` [RFC PATCH 3/3] mm: Identify compound pages sooner in isolate_migratepages_block Alexander Duyck 2020-08-13 4:02 ` Alexander Duyck 2020-08-14 7:20 ` Alex Shi 2020-08-14 7:20 ` Alex Shi 2020-08-17 22:58 ` [PATCH v17 14/21] mm/compaction: do page isolation first in compaction Alexander Duyck 2020-08-17 22:58 ` Alexander Duyck 2020-08-17 22:58 ` Alexander Duyck 2020-07-25 12:59 ` [PATCH v17 15/21] mm/thp: add tail pages into lru anyway in split_huge_page() Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 16/21] mm/swap: serialize memcg changes in pagevec_lru_move_fn Alex Shi 2020-07-25 12:59 ` [PATCH v17 17/21] mm/lru: replace pgdat lru_lock with lruvec lock Alex Shi 2020-07-25 12:59 ` Alex Shi 2020-07-27 23:34 ` Alexander Duyck 2020-07-27 23:34 ` Alexander Duyck 2020-07-27 23:34 ` Alexander Duyck 2020-07-28 7:15 ` Alex Shi 2020-07-28 7:15 ` Alex Shi 2020-07-28 11:19 ` Alex Shi 2020-07-28 11:19 ` Alex Shi 2020-07-28 14:54 ` Alexander Duyck 2020-07-28 14:54 ` Alexander Duyck 2020-07-28 14:54 ` Alexander Duyck 2020-07-29 1:00 ` Alex Shi 2020-07-29 1:27 ` Alexander Duyck 2020-07-29 1:27 ` Alexander Duyck 2020-07-29 1:27 ` Alexander Duyck 2020-07-29 2:27 ` Alex Shi 2020-07-29 2:27 ` Alex Shi 2020-07-28 15:39 ` Alex Shi 2020-07-28 15:39 ` Alex Shi 2020-07-28 15:55 ` Alexander Duyck 2020-07-28 15:55 ` Alexander Duyck 2020-07-28 15:55 ` Alexander Duyck 2020-07-29 0:48 ` Alex Shi 2020-07-29 3:54 ` Alex Shi 2020-08-06 7:41 ` Alex Shi 2020-08-06 7:41 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 18/21] mm/lru: introduce the relock_page_lruvec function Alex Shi 2020-07-29 17:52 ` Alexander Duyck 2020-07-29 17:52 ` Alexander Duyck 2020-07-29 17:52 ` Alexander Duyck 2020-07-30 6:08 ` Alex Shi 2020-07-30 6:08 ` Alex Shi 2020-07-31 14:20 ` Alexander Duyck 2020-07-31 14:20 ` Alexander Duyck 2020-07-31 14:20 ` Alexander Duyck 2020-07-31 21:14 ` [PATCH RFC] mm: Add function for testing if the current lruvec lock is valid alexander.h.duyck 2020-07-31 21:14 ` alexander.h.duyck-ral2JQCrhuEAvxtiuMwx3w 2020-07-31 23:54 ` Alex Shi 2020-07-31 23:54 ` Alex Shi 2020-08-02 18:20 ` Alexander Duyck 2020-08-02 18:20 ` Alexander Duyck 2020-08-02 18:20 ` Alexander Duyck 2020-08-04 6:13 ` Alex Shi 2020-08-04 6:13 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 19/21] mm/vmscan: use relock for move_pages_to_lru Alex Shi 2020-08-03 22:49 ` Alexander Duyck 2020-08-03 22:49 ` Alexander Duyck 2020-08-04 6:23 ` Alex Shi 2020-08-04 6:23 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 20/21] mm/pgdat: remove pgdat lru_lock Alex Shi 2020-08-03 22:42 ` Alexander Duyck 2020-08-03 22:42 ` Alexander Duyck 2020-08-03 22:42 ` Alexander Duyck 2020-08-03 22:45 ` Alexander Duyck 2020-08-03 22:45 ` Alexander Duyck 2020-08-03 22:45 ` Alexander Duyck 2020-08-04 6:22 ` Alex Shi 2020-08-04 6:22 ` Alex Shi 2020-07-25 12:59 ` [PATCH v17 21/21] mm/lru: revise the comments of lru_lock Alex Shi 2020-08-03 22:37 ` Alexander Duyck 2020-08-03 22:37 ` Alexander Duyck 2020-08-04 10:04 ` Alex Shi 2020-08-04 10:04 ` Alex Shi 2020-08-04 14:29 ` Alexander Duyck 2020-08-04 14:29 ` Alexander Duyck 2020-08-04 14:29 ` Alexander Duyck 2020-08-06 1:39 ` Alex Shi 2020-08-06 1:39 ` Alex Shi 2020-08-06 16:27 ` Alexander Duyck 2020-08-06 16:27 ` Alexander Duyck 2020-08-06 16:27 ` Alexander Duyck 2020-07-27 5:40 ` [PATCH v17 00/21] per memcg lru lock Alex Shi 2020-07-27 5:40 ` Alex Shi 2020-07-29 14:49 ` Alex Shi 2020-07-29 14:49 ` Alex Shi 2020-07-29 18:06 ` Hugh Dickins 2020-07-29 18:06 ` Hugh Dickins 2020-07-30 2:16 ` Alex Shi 2020-07-30 2:16 ` Alex Shi 2020-08-03 15:07 ` Michal Hocko 2020-08-03 15:07 ` Michal Hocko 2020-08-04 6:14 ` Alex Shi 2020-08-04 6:14 ` Alex Shi 2020-07-31 21:31 ` Alexander Duyck 2020-07-31 21:31 ` Alexander Duyck 2020-07-31 21:31 ` Alexander Duyck 2020-08-04 8:36 ` Alex Shi 2020-08-04 8:36 ` Alex Shi 2020-08-04 8:36 ` Alex Shi 2020-08-04 8:36 ` Alex Shi 2020-08-04 8:37 ` Alex Shi 2020-08-04 8:37 ` Alex Shi 2020-08-04 8:37 ` Alex Shi 2020-08-04 8:37 ` Alex Shi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1595681998-19193-13-git-send-email-alex.shi@linux.alibaba.com \ --to=alex.shi@linux.alibaba.com \ --cc=akpm@linux-foundation.org \ --cc=alexander.duyck@gmail.com \ --cc=cgroups@vger.kernel.org \ --cc=daniel.m.jordan@oracle.com \ --cc=hannes@cmpxchg.org \ --cc=hughd@google.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=khlebnikov@yandex-team.ru \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lkp@intel.com \ --cc=mgorman@techsingularity.net \ --cc=richard.weiyang@gmail.com \ --cc=rong.a.chen@intel.com \ --cc=shakeelb@google.com \ --cc=tj@kernel.org \ --cc=willy@infradead.org \ --cc=yang.shi@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.