From: Shakeel Butt <shakeelb@google.com> To: Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@kernel.org>, Roman Gushchin <roman.gushchin@linux.dev>, Muchun Song <songmuchun@bytedance.com> Cc: "Michal Koutný" <mkoutny@suse.com>, "Eric Dumazet" <edumazet@google.com>, "Soheil Hassas Yeganeh" <soheil@google.com>, "Feng Tang" <feng.tang@intel.com>, "Oliver Sang" <oliver.sang@intel.com>, "Andrew Morton" <akpm@linux-foundation.org>, lkp@lists.01.org, cgroups@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Shakeel Butt" <shakeelb@google.com> Subject: [PATCH v2 1/3] mm: page_counter: remove unneeded atomic ops for low/min Date: Thu, 25 Aug 2022 00:05:04 +0000 [thread overview] Message-ID: <20220825000506.239406-2-shakeelb@google.com> (raw) In-Reply-To: <20220825000506.239406-1-shakeelb@google.com> For cgroups using low or min protections, the function propagate_protected_usage() was doing an atomic xchg() operation irrespectively. We can optimize out this atomic operation for one specific scenario where the workload is using the protection (i.e. min > 0) and the usage is above the protection (i.e. usage > min). This scenario is actually very common where the users want a part of their workload to be protected against the external reclaim. Though this optimization does introduce a race when the usage is around the protection and concurrent charges and uncharged trip it over or under the protection. In such cases, we might see lower effective protection but the subsequent charge/uncharge will correct it. To evaluate the impact of this optimization, on a 72 CPUs machine, we ran the following workload in a three level of cgroup hierarchy with top level having min and low setup appropriately to see if this optimization is effective for the mentioned case. $ netserver -6 # 36 instances of netperf with following params $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K Results (average throughput of netperf): Without (6.0-rc1) 10482.7 Mbps With patch 14542.5 Mbps (38.7% improvement) With the patch, the throughput improved by 38.7% Signed-off-by: Shakeel Butt <shakeelb@google.com> Reported-by: kernel test robot <oliver.sang@intel.com> Acked-by: Soheil Hassas Yeganeh <soheil@google.com> Reviewed-by: Feng Tang <feng.tang@intel.com> Acked-by: Roman Gushchin <roman.gushchin@linux.dev> --- Changes since v1: - Commit message update with more detail on which scenario is getting optimized and possible race condition. mm/page_counter.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/page_counter.c b/mm/page_counter.c index eb156ff5d603..47711aa28161 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -17,24 +17,23 @@ static void propagate_protected_usage(struct page_counter *c, unsigned long usage) { unsigned long protected, old_protected; - unsigned long low, min; long delta; if (!c->parent) return; - min = READ_ONCE(c->min); - if (min || atomic_long_read(&c->min_usage)) { - protected = min(usage, min); + protected = min(usage, READ_ONCE(c->min)); + old_protected = atomic_long_read(&c->min_usage); + if (protected != old_protected) { old_protected = atomic_long_xchg(&c->min_usage, protected); delta = protected - old_protected; if (delta) atomic_long_add(delta, &c->parent->children_min_usage); } - low = READ_ONCE(c->low); - if (low || atomic_long_read(&c->low_usage)) { - protected = min(usage, low); + protected = min(usage, READ_ONCE(c->low)); + old_protected = atomic_long_read(&c->low_usage); + if (protected != old_protected) { old_protected = atomic_long_xchg(&c->low_usage, protected); delta = protected - old_protected; if (delta) -- 2.37.1.595.g718a3a8f04-goog
WARNING: multiple messages have this Message-ID (diff)
From: Shakeel Butt <shakeelb@google.com> To: lkp@lists.01.org Subject: [PATCH v2 1/3] mm: page_counter: remove unneeded atomic ops for low/min Date: Thu, 25 Aug 2022 00:05:04 +0000 [thread overview] Message-ID: <20220825000506.239406-2-shakeelb@google.com> (raw) In-Reply-To: <20220825000506.239406-1-shakeelb@google.com> [-- Attachment #1: Type: text/plain, Size: 2993 bytes --] For cgroups using low or min protections, the function propagate_protected_usage() was doing an atomic xchg() operation irrespectively. We can optimize out this atomic operation for one specific scenario where the workload is using the protection (i.e. min > 0) and the usage is above the protection (i.e. usage > min). This scenario is actually very common where the users want a part of their workload to be protected against the external reclaim. Though this optimization does introduce a race when the usage is around the protection and concurrent charges and uncharged trip it over or under the protection. In such cases, we might see lower effective protection but the subsequent charge/uncharge will correct it. To evaluate the impact of this optimization, on a 72 CPUs machine, we ran the following workload in a three level of cgroup hierarchy with top level having min and low setup appropriately to see if this optimization is effective for the mentioned case. $ netserver -6 # 36 instances of netperf with following params $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K Results (average throughput of netperf): Without (6.0-rc1) 10482.7 Mbps With patch 14542.5 Mbps (38.7% improvement) With the patch, the throughput improved by 38.7% Signed-off-by: Shakeel Butt <shakeelb@google.com> Reported-by: kernel test robot <oliver.sang@intel.com> Acked-by: Soheil Hassas Yeganeh <soheil@google.com> Reviewed-by: Feng Tang <feng.tang@intel.com> Acked-by: Roman Gushchin <roman.gushchin@linux.dev> --- Changes since v1: - Commit message update with more detail on which scenario is getting optimized and possible race condition. mm/page_counter.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/mm/page_counter.c b/mm/page_counter.c index eb156ff5d603..47711aa28161 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -17,24 +17,23 @@ static void propagate_protected_usage(struct page_counter *c, unsigned long usage) { unsigned long protected, old_protected; - unsigned long low, min; long delta; if (!c->parent) return; - min = READ_ONCE(c->min); - if (min || atomic_long_read(&c->min_usage)) { - protected = min(usage, min); + protected = min(usage, READ_ONCE(c->min)); + old_protected = atomic_long_read(&c->min_usage); + if (protected != old_protected) { old_protected = atomic_long_xchg(&c->min_usage, protected); delta = protected - old_protected; if (delta) atomic_long_add(delta, &c->parent->children_min_usage); } - low = READ_ONCE(c->low); - if (low || atomic_long_read(&c->low_usage)) { - protected = min(usage, low); + protected = min(usage, READ_ONCE(c->low)); + old_protected = atomic_long_read(&c->low_usage); + if (protected != old_protected) { old_protected = atomic_long_xchg(&c->low_usage, protected); delta = protected - old_protected; if (delta) -- 2.37.1.595.g718a3a8f04-goog
next prev parent reply other threads:[~2022-08-25 0:05 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-08-25 0:05 [PATCH v2 0/3] memcg: optimize charge codepath Shakeel Butt 2022-08-25 0:05 ` Shakeel Butt 2022-08-25 0:05 ` Shakeel Butt 2022-08-25 0:05 ` Shakeel Butt [this message] 2022-08-25 0:05 ` [PATCH v2 1/3] mm: page_counter: remove unneeded atomic ops for low/min Shakeel Butt 2022-08-25 6:43 ` Michal Hocko 2022-08-25 6:43 ` Michal Hocko 2022-08-25 6:43 ` Michal Hocko 2022-08-25 0:05 ` [PATCH v2 2/3] mm: page_counter: rearrange struct page_counter fields Shakeel Butt 2022-08-25 0:05 ` Shakeel Butt 2022-08-25 0:33 ` Andrew Morton 2022-08-25 0:33 ` Andrew Morton 2022-08-25 0:33 ` Andrew Morton 2022-08-25 4:41 ` Shakeel Butt 2022-08-25 4:41 ` Shakeel Butt 2022-08-25 4:41 ` Shakeel Butt 2022-08-25 5:21 ` Andrew Morton 2022-08-25 5:21 ` Andrew Morton 2022-08-25 5:21 ` Andrew Morton 2022-08-25 15:24 ` Shakeel Butt 2022-08-25 15:24 ` Shakeel Butt 2022-08-25 15:24 ` Shakeel Butt 2022-08-25 6:47 ` Michal Hocko 2022-08-25 6:47 ` Michal Hocko 2022-08-25 15:25 ` Shakeel Butt 2022-08-25 15:25 ` Shakeel Butt 2022-08-25 15:25 ` Shakeel Butt 2022-08-25 0:05 ` [PATCH v2 3/3] memcg: increase MEMCG_CHARGE_BATCH to 64 Shakeel Butt 2022-08-25 0:05 ` Shakeel Butt 2022-08-25 6:49 ` Michal Hocko 2022-08-25 6:49 ` Michal Hocko 2022-08-25 8:30 ` Muchun Song 2022-08-25 8:30 ` Muchun Song 2022-08-25 8:30 ` Muchun Song
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220825000506.239406-2-shakeelb@google.com \ --to=shakeelb@google.com \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=edumazet@google.com \ --cc=feng.tang@intel.com \ --cc=hannes@cmpxchg.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lkp@lists.01.org \ --cc=mhocko@kernel.org \ --cc=mkoutny@suse.com \ --cc=netdev@vger.kernel.org \ --cc=oliver.sang@intel.com \ --cc=roman.gushchin@linux.dev \ --cc=soheil@google.com \ --cc=songmuchun@bytedance.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.