From: Minchan Kim <minchan@kernel.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
linux-mm <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-api@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
Tim Murray <timmurray@google.com>,
Joel Fernandes <joel@joelfernandes.org>,
Suren Baghdasaryan <surenb@google.com>,
Daniel Colascione <dancol@google.com>,
Shakeel Butt <shakeelb@google.com>,
Sonny Rao <sonnyrao@google.com>,
Brian Geffon <bgeffon@google.com>,
jannh@google.com, oleg@redhat.com, christian@brauner.io,
oleksandr@redhat.com, hdanton@sina.com
Subject: Re: [RFCv2 3/6] mm: introduce MADV_PAGEOUT
Date: Sat, 1 Jun 2019 08:14:38 +0900 [thread overview]
Message-ID: <20190531231438.GA248371@google.com> (raw)
In-Reply-To: <20190531165927.GA20067@cmpxchg.org>
Hey Johannes,
On Fri, May 31, 2019 at 12:59:27PM -0400, Johannes Weiner wrote:
> Hi Michan,
>
> this looks pretty straight-forward to me, only one kink:
>
> On Fri, May 31, 2019 at 03:43:10PM +0900, Minchan Kim wrote:
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -2126,6 +2126,83 @@ static void shrink_active_list(unsigned long nr_to_scan,
> > nr_deactivate, nr_rotated, sc->priority, file);
> > }
> >
> > +unsigned long reclaim_pages(struct list_head *page_list)
> > +{
> > + int nid = -1;
> > + unsigned long nr_isolated[2] = {0, };
> > + unsigned long nr_reclaimed = 0;
> > + LIST_HEAD(node_page_list);
> > + struct reclaim_stat dummy_stat;
> > + struct scan_control sc = {
> > + .gfp_mask = GFP_KERNEL,
> > + .priority = DEF_PRIORITY,
> > + .may_writepage = 1,
> > + .may_unmap = 1,
> > + .may_swap = 1,
> > + };
> > +
> > + while (!list_empty(page_list)) {
> > + struct page *page;
> > +
> > + page = lru_to_page(page_list);
> > + if (nid == -1) {
> > + nid = page_to_nid(page);
> > + INIT_LIST_HEAD(&node_page_list);
> > + nr_isolated[0] = nr_isolated[1] = 0;
> > + }
> > +
> > + if (nid == page_to_nid(page)) {
> > + list_move(&page->lru, &node_page_list);
> > + nr_isolated[!!page_is_file_cache(page)] +=
> > + hpage_nr_pages(page);
> > + continue;
> > + }
> > +
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_ANON,
> > + nr_isolated[0]);
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_FILE,
> > + nr_isolated[1]);
> > + nr_reclaimed += shrink_page_list(&node_page_list,
> > + NODE_DATA(nid), &sc, TTU_IGNORE_ACCESS,
> > + &dummy_stat, true);
> > + while (!list_empty(&node_page_list)) {
> > + struct page *page = lru_to_page(&node_page_list);
> > +
> > + list_del(&page->lru);
> > + putback_lru_page(page);
> > + }
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_ANON,
> > + -nr_isolated[0]);
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_FILE,
> > + -nr_isolated[1]);
> > + nid = -1;
> > + }
> > +
> > + if (!list_empty(&node_page_list)) {
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_ANON,
> > + nr_isolated[0]);
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_FILE,
> > + nr_isolated[1]);
> > + nr_reclaimed += shrink_page_list(&node_page_list,
> > + NODE_DATA(nid), &sc, TTU_IGNORE_ACCESS,
> > + &dummy_stat, true);
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_ANON,
> > + -nr_isolated[0]);
> > + mod_node_page_state(NODE_DATA(nid), NR_ISOLATED_FILE,
> > + -nr_isolated[1]);
> > +
> > + while (!list_empty(&node_page_list)) {
> > + struct page *page = lru_to_page(&node_page_list);
> > +
> > + list_del(&page->lru);
> > + putback_lru_page(page);
> > + }
> > +
> > + }
>
> The NR_ISOLATED accounting, nid parsing etc. is really awkward and
> makes it hard to see what the function actually does.
>
> Can you please make those ISOLATED counters part of the isolation API?
> Your patch really shows this is an overdue cleanup.
Yeah, that was very painful.
>
> These are fast local percpu counters, we don't need the sprawling
> batching we do all over vmscan.c, migrate.c, khugepaged.c,
> compaction.c etc. Isolation can increase the counter page by page, and
> reclaim or putback can likewise decrease them one by one.
>
> It looks like mlock is the only user of the isolation api that does
> not participate in the NR_ISOLATED_* counters protocol, but I don't
> see why it wouldn't, or why doing so would hurt.
>
> There are also seem to be quite a few callsites that use the atomic
> versions of the counter API when they're clearly under the irqsafe
> lru_lock. That would be fixed automatically by this work as well.
I agree all points so will prepare clean up patch.
next prev parent reply other threads:[~2019-05-31 23:14 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-31 6:43 [RFCv2 0/6] introduce memory hinting API for external process Minchan Kim
2019-05-31 6:43 ` [RFCv2 1/6] mm: introduce MADV_COLD Minchan Kim
2019-05-31 8:47 ` Michal Hocko
2019-05-31 13:39 ` Minchan Kim
2019-05-31 14:03 ` Michal Hocko
2019-05-31 14:34 ` Minchan Kim
2019-06-03 7:16 ` Michal Hocko
2019-06-03 15:43 ` Daniel Colascione
2019-06-03 17:27 ` Johannes Weiner
2019-06-03 20:32 ` Michal Hocko
2019-06-03 21:50 ` Johannes Weiner
2019-06-03 23:02 ` Minchan Kim
2019-06-04 6:56 ` Michal Hocko
2019-06-04 12:06 ` Johannes Weiner
2019-06-04 6:55 ` Michal Hocko
2019-06-04 4:26 ` Minchan Kim
2019-06-04 7:02 ` Michal Hocko
2019-05-31 6:43 ` [RFCv2 2/6] mm: change PAGEREF_RECLAIM_CLEAN with PAGE_REFRECLAIM Minchan Kim
2019-05-31 6:43 ` [RFCv2 3/6] mm: introduce MADV_PAGEOUT Minchan Kim
2019-05-31 8:50 ` Michal Hocko
2019-05-31 13:44 ` Minchan Kim
2019-05-31 16:59 ` Johannes Weiner
2019-05-31 23:14 ` Minchan Kim [this message]
2019-05-31 6:43 ` [RFCv2 4/6] mm: factor out madvise's core functionality Minchan Kim
2019-05-31 7:04 ` Oleksandr Natalenko
2019-05-31 13:12 ` Minchan Kim
2019-05-31 14:35 ` Oleksandr Natalenko
2019-05-31 23:29 ` Minchan Kim
2019-06-05 13:27 ` Oleksandr Natalenko
2019-06-10 10:12 ` Minchan Kim
2019-05-31 6:43 ` [RFCv2 5/6] mm: introduce external memory hinting API Minchan Kim
2019-05-31 8:37 ` Michal Hocko
2019-05-31 13:19 ` Minchan Kim
2019-05-31 14:00 ` Michal Hocko
2019-05-31 14:11 ` Minchan Kim
2019-05-31 17:35 ` Daniel Colascione
2019-05-31 6:43 ` [RFCv2 6/6] mm: extend process_madvise syscall to support vector arrary Minchan Kim
2019-05-31 10:06 ` Yann Droneaud
2019-05-31 23:18 ` Minchan Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190531231438.GA248371@google.com \
--to=minchan@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=bgeffon@google.com \
--cc=christian@brauner.io \
--cc=dancol@google.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=jannh@google.com \
--cc=joel@joelfernandes.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=oleg@redhat.com \
--cc=oleksandr@redhat.com \
--cc=shakeelb@google.com \
--cc=sonnyrao@google.com \
--cc=surenb@google.com \
--cc=timmurray@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).