From: Vlastimil Babka <vbabka@suse.cz>
To: Yang Shi <shy828301@gmail.com>
Cc: Roman Gushchin <guro@fb.com>, Kirill Tkhai <ktkhai@virtuozzo.com>,
Shakeel Butt <shakeelb@google.com>,
Dave Chinner <david@fromorbit.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@suse.com>,
Andrew Morton <akpm@linux-foundation.org>,
Linux MM <linux-mm@kvack.org>,
Linux FS-devel Mailing List <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [v7 PATCH 12/12] mm: vmscan: shrink deferred objects proportional to priority
Date: Thu, 11 Feb 2021 19:52:20 +0100 [thread overview]
Message-ID: <a56fa0f1-3ac6-49f1-31c1-8bfec961d04e@suse.cz> (raw)
In-Reply-To: <CAHbLzkpF9+NUp2yUf_yKHHngKXGDya4Mj3ZTc-2rm3yFNw_==A@mail.gmail.com>
On 2/11/21 6:29 PM, Yang Shi wrote:
> On Thu, Feb 11, 2021 at 5:10 AM Vlastimil Babka <vbabka@suse.cz> wrote:
>> > trace_mm_shrink_slab_start(shrinker, shrinkctl, nr,
>> > freeable, delta, total_scan, priority);
>> > @@ -737,10 +708,9 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
>> > cond_resched();
>> > }
>> >
>> > - if (next_deferred >= scanned)
>> > - next_deferred -= scanned;
>> > - else
>> > - next_deferred = 0;
>> > + next_deferred = max_t(long, (nr - scanned), 0) + total_scan;
>>
>> And here's the bias I think. Suppose we scanned 0 due to e.g. GFP_NOFS. We count
>> as newly deferred both the "delta" part of total_scan, which is fine, but also
>> the "nr >> priority" part, where we failed to our share of the "reduce
>> nr_deferred" work, but I don't think it means we should also increase
>> nr_deferred by that amount of failed work.
>
> Here "nr" is the saved deferred work since the last scan, "scanned" is
> the scanned work in this round, total_scan is the *unscanned" work
> which is actually "total_scan - scanned" (total_scan is decreased by
> scanned in each loop). So, the logic is "decrease any scanned work
> from deferred then add newly unscanned work to deferred". IIUC this is
> what "deferred" means even before this patch.
Hm I thought the logic was "increase by any new work (delta) that wasn't done,
decrease by old deferred work that was done now". My examples with scanned = 0
and scanned = total_work (total_work before subtracting scanned from it) should
demonstrate that the logic is different with your patch.
>> OTOH if we succeed and scan exactly the whole goal, we are subtracting from
>> nr_deferred both the "nr >> priority" part, which is correct, but also delta,
>> which was new work, not deferred one, so that's incorrect IMHO as well.
>
> I don't think so. The deferred comes from new work, why not dec new
> work from deferred?
>
> And, the old code did:
>
> if (next_deferred >= scanned)
> next_deferred -= scanned;
> else
> next_deferred = 0;
>
> IIUC, it also decreases the new work (the scanned includes both last
> deferred and new delata).
Yes, but in the old code, next_deferred starts as
nr = count_nr_deferred()...
total_scan = nr;
delta = ... // something based on freeable
total_scan += delta;
next_deferred = total_scan; // in the common case total_scan >= 0
... and that's "total_scan" before "scanned" is subtracted from it, so it
includes the new_work ("delta"), so then it's OK to do "next_deferred -= scanned";
I still think your formula is (unintentionally) changing the logic. You can also
look at it from different angle, it's effectively (without the max_t() part) "nr
- scanned + total_scan" where total_scan is actually "total_scan - scanned" as
you point your yourself. So "scanned" is subtracted twice? That can't be correct...
>> So the calculation should probably be something like this?
>>
>> next_deferred = max_t(long, nr + delta - scanned, 0);
>>
>> Thanks,
>> Vlastimil
>>
>> > + next_deferred = min(next_deferred, (2 * freeable));
>> > +
>> > /*
>> > * move the unused scan count back into the shrinker in a
>> > * manner that handles concurrent updates.
>> >
>>
>
next prev parent reply other threads:[~2021-02-11 18:59 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-09 17:46 [v7 PATCH 0/12] Make shrinker's nr_deferred memcg aware Yang Shi
2021-02-09 17:46 ` [v7 PATCH 01/12] mm: vmscan: use nid from shrink_control for tracepoint Yang Shi
2021-02-09 19:14 ` Shakeel Butt
2021-02-10 16:58 ` Yang Shi
2021-02-09 19:21 ` Roman Gushchin
2021-02-09 17:46 ` [v7 PATCH 02/12] mm: vmscan: consolidate shrinker_maps handling code Yang Shi
2021-02-09 20:27 ` Roman Gushchin
2021-02-10 14:19 ` Shakeel Butt
2021-02-09 17:46 ` [v7 PATCH 03/12] mm: vmscan: use shrinker_rwsem to protect shrinker_maps allocation Yang Shi
2021-02-09 20:33 ` Roman Gushchin
2021-02-09 23:28 ` Yang Shi
2021-02-09 17:46 ` [v7 PATCH 04/12] mm: vmscan: remove memcg_shrinker_map_size Yang Shi
2021-02-09 20:43 ` Roman Gushchin
2021-02-09 23:31 ` Yang Shi
2021-02-10 18:14 ` Vlastimil Babka
2021-02-09 17:46 ` [v7 PATCH 05/12] mm: memcontrol: rename shrinker_map to shrinker_info Yang Shi
2021-02-09 20:50 ` Roman Gushchin
2021-02-09 23:33 ` Yang Shi
2021-02-10 0:16 ` Roman Gushchin
2021-02-11 16:47 ` Kirill Tkhai
2021-02-11 17:29 ` Yang Shi
2021-02-09 17:46 ` [v7 PATCH 06/12] mm: vmscan: add shrinker_info_protected() helper Yang Shi
2021-02-10 0:22 ` Roman Gushchin
2021-02-10 1:07 ` Yang Shi
2021-02-10 1:29 ` Roman Gushchin
2021-02-10 12:12 ` Kirill Tkhai
2021-02-10 18:17 ` Vlastimil Babka
2021-02-12 6:54 ` [mm] bd741fb2ad: WARNING:suspicious_RCU_usage kernel test robot
2021-02-09 17:46 ` [v7 PATCH 07/12] mm: vmscan: use a new flag to indicate shrinker is registered Yang Shi
2021-02-10 0:39 ` Roman Gushchin
2021-02-10 1:12 ` Yang Shi
2021-02-10 1:34 ` Roman Gushchin
2021-02-10 1:55 ` Yang Shi
2021-02-10 18:45 ` Yang Shi
2021-02-10 18:23 ` Vlastimil Babka
2021-02-09 17:46 ` [v7 PATCH 08/12] mm: vmscan: add per memcg shrinker nr_deferred Yang Shi
2021-02-10 1:10 ` Roman Gushchin
2021-02-10 1:25 ` Yang Shi
2021-02-10 1:40 ` Roman Gushchin
2021-02-10 1:57 ` Yang Shi
2021-02-09 17:46 ` [v7 PATCH 09/12] mm: vmscan: use per memcg nr_deferred of shrinker Yang Shi
2021-02-10 1:27 ` Roman Gushchin
2021-02-10 1:52 ` Yang Shi
2021-02-10 14:36 ` Kirill Tkhai
2021-02-10 16:41 ` Yang Shi
2021-02-09 17:46 ` [v7 PATCH 10/12] mm: vmscan: don't need allocate shrinker->nr_deferred for memcg aware shrinkers Yang Shi
2021-02-10 1:23 ` Roman Gushchin
2021-02-09 17:46 ` [v7 PATCH 11/12] mm: memcontrol: reparent nr_deferred when memcg offline Yang Shi
2021-02-10 1:18 ` Roman Gushchin
2021-02-09 17:46 ` [v7 PATCH 12/12] mm: vmscan: shrink deferred objects proportional to priority Yang Shi
2021-02-11 13:10 ` Vlastimil Babka
2021-02-11 17:29 ` Yang Shi
2021-02-11 18:52 ` Vlastimil Babka [this message]
2021-02-11 19:15 ` Yang Shi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a56fa0f1-3ac6-49f1-31c1-8bfec961d04e@suse.cz \
--to=vbabka@suse.cz \
--cc=akpm@linux-foundation.org \
--cc=david@fromorbit.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=ktkhai@virtuozzo.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=shakeelb@google.com \
--cc=shy828301@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).