Re: [PATCH v4 3/9] mm/lru: replace pgdat lru_lock with lruvec lock

[Hugh Dickins <hughd@google.com>]

On Fri, 22 Nov 2019, Johannes Weiner wrote:
> 
> But that leaves me with one more worry: compaction. We locked out
> charge moving now, so between that and knowing that the page is alive,
> we have page->mem_cgroup stable. But compaction doesn't know whether
> the page is alive - it comes from a pfn and finds out using PageLRU.
> 
> In the current code, pgdat->lru_lock remains the same before and after
> the page is charged to a cgroup, so once compaction has that locked
> and it observes PageLRU, it can go ahead and isolate the page.
> 
> But lruvec->lru_lock changes during charging, and then compaction may
> hold the wrong lock during isolation:
> 
> compaction:				generic_file_buffered_read:
> 
> 					page_cache_alloc()
> 
> !PageBuddy()
> 
> lock_page_lruvec(page)
>   lruvec = mem_cgroup_page_lruvec()
>   spin_lock(&lruvec->lru_lock)
>   if lruvec != mem_cgroup_page_lruvec()
>     goto again
> 
> 					add_to_page_cache_lru()
> 					  mem_cgroup_commit_charge()
> 					    page->mem_cgroup = foo
> 					  lru_cache_add()
> 					    __pagevec_lru_add()
> 					      SetPageLRU()
> 
> if PageLRU(page):
>   __isolate_lru_page()
> 
> I don't see what prevents the lruvec from changing under compaction,
> neither in your patches nor in Hugh's. Maybe I'm missing something?

Speaking for my patches only: I'm humbled, I think you have caught me,
I cannot find any argument against the race you suggest here.

The race with mem_cgroup_move_account(), which Konstantin pointed out
in 2012's https://lore.kernel.org/lkml/4F433418.3010401@openvz.org/
but I later misunderstood, and came to think I needed no patch against,
until this week coming to perceive the same race in isolate_lru_page():
that one is easily and satisfactorily fixed by holding lruvec lock in
mem_cgroup_move_account() - embarrassing, but not too serious.

Your race here (again, lruvec lock taken then PageLRU observed, but
page->mem_cgroup changed in between) really questions my whole scheme:
I am not going to propose a solution now, I'll have to go back and
recheck my assumptions all over.  Certainly isolate_migratepage_block()
has a harder job than any other, but I need to re-review it all.

Maybe we got it right back in the days of PageCgroupUsed, and then I
paid too little attention when rebasing to your welcome simplifications.
I don't think any of us want to bring back PageCgroupUsed! And maybe we
could get it right by always holding lruvec lock in commit_charge(),
lrucare or not; but that's a much hotter path, and not a change I'd
expect anyone to embrace.

I'll go away and re-examine it all; probably start by verifying that
your race actually happens in practice, though we never observed it.

Heavy-hearted thanks, Hannes!
Hugh