From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86208C28D13 for ; Mon, 22 Aug 2022 04:59:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232576AbiHVE7f (ORCPT ); Mon, 22 Aug 2022 00:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231694AbiHVE7d (ORCPT ); Mon, 22 Aug 2022 00:59:33 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BA1124F0E for ; Sun, 21 Aug 2022 21:59:32 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id d71so8393287pgc.13 for ; Sun, 21 Aug 2022 21:59:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=9XhZmihVWYtN8pqGhGwupTf+AWRqtf1YyaQ/pGiog9k=; b=e4g+T2mMYZZcLZKVuyYTBE7KioYD6BNBNG1cLqDFZ5B9t0E6FIRD5EvbyAzLTCr3kJ rA6ZK7oCz9GEATt82qpw+2JFjAHNx6MWzrOEJkCcglpPUJI6hS1usRrjSP0PtvLaklzt 2SWFQW3etXo3F8BRQXjkvsEAD1xWTy3389fS+y8cwwh/ZpGZYzfVQV4mAVDrVneI1RKY LqoHhaNoGRAKG4D6abLWZ4z6qaxhraGCM7tcAmHqKNXwMFf8R6WZS65/KM+mSpunicvW NoZagRqgMyd0mx1A/gtNa8G0tjAlwmRM7GukYQSHxRJh5NrCaUN7Ht0Ligb/wX1I4ajV MSHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=9XhZmihVWYtN8pqGhGwupTf+AWRqtf1YyaQ/pGiog9k=; b=XGhtMBa40RzZ0BQUXPFPmha3bk1ULpICOBMu98uj3irAxw2ZnzUbcfYbmc6QX3XVc9 4jYe2hToojY6DAs4KgFU5HmldyYTINIoKl9enHLTaNv+hG3XyM/ufzQnjGM3JNbWU0qY HfeuD+twyR/XPxn7eJAryW4eKoQdfQIVKa+dpGalV2UNYfTNDt6YXtrwpAhPNg/Ql/zt /5rK7jI73XUTpb3niEe22I1q89w0Og5YfU6Bb/17EeGLAPIfq/ZUmktFy0z3OQMJOqxm XwdNoDEoHe6PJbu0b6mJwV4Qpx8ZpefJIYoYam9AqLG+HpadFP6Xbi4PPeiDCNGdEoG4 FCfA== X-Gm-Message-State: ACgBeo0N3Z+fP9fp+eQsszvF1EI91Pr0V0pUgvd3BjAPVMvmp09kufmg CAlH4RpFsZGyGESw7znk6YLzRHzLSGGMnoJ/Nun5cg== X-Google-Smtp-Source: AA6agR4/1s7y++qNsA2/cfFgEIhAK4Q20RaQ1xUDhpnZDmYg93tgbnj5pntFlvFsEA9PvT9gkP9gZgjLoqCg32dK4EA= X-Received: by 2002:a62:6497:0:b0:52e:e0cd:1963 with SMTP id y145-20020a626497000000b0052ee0cd1963mr19195169pfb.58.1661144371422; Sun, 21 Aug 2022 21:59:31 -0700 (PDT) MIME-Version: 1.0 References: <20220822001737.4120417-1-shakeelb@google.com> <20220822001737.4120417-3-shakeelb@google.com> In-Reply-To: From: Shakeel Butt Date: Sun, 21 Aug 2022 21:59:20 -0700 Message-ID: Subject: Re: [PATCH 2/3] mm: page_counter: rearrange struct page_counter fields To: Feng Tang Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "Michal Koutn??" , Eric Dumazet , Soheil Hassas Yeganeh , "Sang, Oliver" , Andrew Morton , "lkp@lists.01.org" , "cgroups@vger.kernel.org" , "linux-mm@kvack.org" , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Aug 21, 2022 at 7:12 PM Feng Tang wrote: > > On Mon, Aug 22, 2022 at 08:17:36AM +0800, Shakeel Butt wrote: > > With memcg v2 enabled, memcg->memory.usage is a very hot member for > > the workloads doing memcg charging on multiple CPUs concurrently. > > Particularly the network intensive workloads. In addition, there is a > > false cache sharing between memory.usage and memory.high on the charge > > path. This patch moves the usage into a separate cacheline and move all > > the read most fields into separate cacheline. > > > > To evaluate the impact of this optimization, on a 72 CPUs machine, we > > ran the following workload in a three level of cgroup hierarchy with top > > level having min and low setup appropriately. More specifically > > memory.min equal to size of netperf binary and memory.low double of > > that. > > > > $ netserver -6 > > # 36 instances of netperf with following params > > $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K > > > > Results (average throughput of netperf): > > Without (6.0-rc1) 10482.7 Mbps > > With patch 12413.7 Mbps (18.4% improvement) > > > > With the patch, the throughput improved by 18.4%. > > > > One side-effect of this patch is the increase in the size of struct > > mem_cgroup. However for the performance improvement, this additional > > size is worth it. In addition there are opportunities to reduce the size > > of struct mem_cgroup like deprecation of kmem and tcpmem page counters > > and better packing. > > > > Signed-off-by: Shakeel Butt > > Reported-by: kernel test robot > > Looks good to me, with one nit below. > > Reviewed-by: Feng Tang Thanks. > > > --- > > include/linux/page_counter.h | 34 +++++++++++++++++++++++----------- > > 1 file changed, 23 insertions(+), 11 deletions(-) > > > > diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h > > index 679591301994..8ce99bde645f 100644 > > --- a/include/linux/page_counter.h > > +++ b/include/linux/page_counter.h > > @@ -3,15 +3,27 @@ > > #define _LINUX_PAGE_COUNTER_H > > > > #include > > +#include > > #include > > #include > > > > +#if defined(CONFIG_SMP) > > +struct pc_padding { > > + char x[0]; > > +} ____cacheline_internodealigned_in_smp; > > +#define PC_PADDING(name) struct pc_padding name > > +#else > > +#define PC_PADDING(name) > > +#endif > > There are 2 similar padding definitions in mmzone.h and memcontrol.h: > > struct memcg_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define MEMCG_PADDING(name) struct memcg_padding name > > struct zone_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define ZONE_PADDING(name) struct zone_padding name; > > Maybe we can generalize them, and lift it into include/cache.h? so > that more places can reuse it in future. > This makes sense but let me do that in a separate patch. From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============2320881671132643540==" MIME-Version: 1.0 From: Shakeel Butt To: lkp@lists.01.org Subject: Re: [PATCH 2/3] mm: page_counter: rearrange struct page_counter fields Date: Sun, 21 Aug 2022 21:59:20 -0700 Message-ID: In-Reply-To: List-Id: --===============2320881671132643540== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Sun, Aug 21, 2022 at 7:12 PM Feng Tang wrote: > > On Mon, Aug 22, 2022 at 08:17:36AM +0800, Shakeel Butt wrote: > > With memcg v2 enabled, memcg->memory.usage is a very hot member for > > the workloads doing memcg charging on multiple CPUs concurrently. > > Particularly the network intensive workloads. In addition, there is a > > false cache sharing between memory.usage and memory.high on the charge > > path. This patch moves the usage into a separate cacheline and move all > > the read most fields into separate cacheline. > > > > To evaluate the impact of this optimization, on a 72 CPUs machine, we > > ran the following workload in a three level of cgroup hierarchy with top > > level having min and low setup appropriately. More specifically > > memory.min equal to size of netperf binary and memory.low double of > > that. > > > > $ netserver -6 > > # 36 instances of netperf with following params > > $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K > > > > Results (average throughput of netperf): > > Without (6.0-rc1) 10482.7 Mbps > > With patch 12413.7 Mbps (18.4% improvement) > > > > With the patch, the throughput improved by 18.4%. > > > > One side-effect of this patch is the increase in the size of struct > > mem_cgroup. However for the performance improvement, this additional > > size is worth it. In addition there are opportunities to reduce the size > > of struct mem_cgroup like deprecation of kmem and tcpmem page counters > > and better packing. > > > > Signed-off-by: Shakeel Butt > > Reported-by: kernel test robot > > Looks good to me, with one nit below. > > Reviewed-by: Feng Tang Thanks. > > > --- > > include/linux/page_counter.h | 34 +++++++++++++++++++++++----------- > > 1 file changed, 23 insertions(+), 11 deletions(-) > > > > diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h > > index 679591301994..8ce99bde645f 100644 > > --- a/include/linux/page_counter.h > > +++ b/include/linux/page_counter.h > > @@ -3,15 +3,27 @@ > > #define _LINUX_PAGE_COUNTER_H > > > > #include > > +#include > > #include > > #include > > > > +#if defined(CONFIG_SMP) > > +struct pc_padding { > > + char x[0]; > > +} ____cacheline_internodealigned_in_smp; > > +#define PC_PADDING(name) struct pc_padding name > > +#else > > +#define PC_PADDING(name) > > +#endif > > There are 2 similar padding definitions in mmzone.h and memcontrol.h: > > struct memcg_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define MEMCG_PADDING(name) struct memcg_padding name > > struct zone_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define ZONE_PADDING(name) struct zone_padding name; > > Maybe we can generalize them, and lift it into include/cache.h? so > that more places can reuse it in future. > This makes sense but let me do that in a separate patch. --===============2320881671132643540==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shakeel Butt Subject: Re: [PATCH 2/3] mm: page_counter: rearrange struct page_counter fields Date: Sun, 21 Aug 2022 21:59:20 -0700 Message-ID: References: <20220822001737.4120417-1-shakeelb@google.com> <20220822001737.4120417-3-shakeelb@google.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=9XhZmihVWYtN8pqGhGwupTf+AWRqtf1YyaQ/pGiog9k=; b=e4g+T2mMYZZcLZKVuyYTBE7KioYD6BNBNG1cLqDFZ5B9t0E6FIRD5EvbyAzLTCr3kJ rA6ZK7oCz9GEATt82qpw+2JFjAHNx6MWzrOEJkCcglpPUJI6hS1usRrjSP0PtvLaklzt 2SWFQW3etXo3F8BRQXjkvsEAD1xWTy3389fS+y8cwwh/ZpGZYzfVQV4mAVDrVneI1RKY LqoHhaNoGRAKG4D6abLWZ4z6qaxhraGCM7tcAmHqKNXwMFf8R6WZS65/KM+mSpunicvW NoZagRqgMyd0mx1A/gtNa8G0tjAlwmRM7GukYQSHxRJh5NrCaUN7Ht0Ligb/wX1I4ajV MSHA== In-Reply-To: List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Feng Tang Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Michal Koutn?? , Eric Dumazet , Soheil Hassas Yeganeh , "Sang, Oliver" , Andrew Morton , "lkp-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org" , "cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org" , "netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" On Sun, Aug 21, 2022 at 7:12 PM Feng Tang wrote: > > On Mon, Aug 22, 2022 at 08:17:36AM +0800, Shakeel Butt wrote: > > With memcg v2 enabled, memcg->memory.usage is a very hot member for > > the workloads doing memcg charging on multiple CPUs concurrently. > > Particularly the network intensive workloads. In addition, there is a > > false cache sharing between memory.usage and memory.high on the charge > > path. This patch moves the usage into a separate cacheline and move all > > the read most fields into separate cacheline. > > > > To evaluate the impact of this optimization, on a 72 CPUs machine, we > > ran the following workload in a three level of cgroup hierarchy with top > > level having min and low setup appropriately. More specifically > > memory.min equal to size of netperf binary and memory.low double of > > that. > > > > $ netserver -6 > > # 36 instances of netperf with following params > > $ netperf -6 -H ::1 -l 60 -t TCP_SENDFILE -- -m 10K > > > > Results (average throughput of netperf): > > Without (6.0-rc1) 10482.7 Mbps > > With patch 12413.7 Mbps (18.4% improvement) > > > > With the patch, the throughput improved by 18.4%. > > > > One side-effect of this patch is the increase in the size of struct > > mem_cgroup. However for the performance improvement, this additional > > size is worth it. In addition there are opportunities to reduce the size > > of struct mem_cgroup like deprecation of kmem and tcpmem page counters > > and better packing. > > > > Signed-off-by: Shakeel Butt > > Reported-by: kernel test robot > > Looks good to me, with one nit below. > > Reviewed-by: Feng Tang Thanks. > > > --- > > include/linux/page_counter.h | 34 +++++++++++++++++++++++----------- > > 1 file changed, 23 insertions(+), 11 deletions(-) > > > > diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h > > index 679591301994..8ce99bde645f 100644 > > --- a/include/linux/page_counter.h > > +++ b/include/linux/page_counter.h > > @@ -3,15 +3,27 @@ > > #define _LINUX_PAGE_COUNTER_H > > > > #include > > +#include > > #include > > #include > > > > +#if defined(CONFIG_SMP) > > +struct pc_padding { > > + char x[0]; > > +} ____cacheline_internodealigned_in_smp; > > +#define PC_PADDING(name) struct pc_padding name > > +#else > > +#define PC_PADDING(name) > > +#endif > > There are 2 similar padding definitions in mmzone.h and memcontrol.h: > > struct memcg_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define MEMCG_PADDING(name) struct memcg_padding name > > struct zone_padding { > char x[0]; > } ____cacheline_internodealigned_in_smp; > #define ZONE_PADDING(name) struct zone_padding name; > > Maybe we can generalize them, and lift it into include/cache.h? so > that more places can reuse it in future. > This makes sense but let me do that in a separate patch.