From: Feng Tang <feng.tang@intel.com> To: Shakeel Butt <shakeelb@google.com> Cc: Eric Dumazet <edumazet@google.com>, Linux MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Roman Gushchin <roman.gushchin@linux.dev>, Michal Hocko <mhocko@kernel.org>, Johannes Weiner <hannes@cmpxchg.org>, Muchun Song <songmuchun@bytedance.com>, Jakub Kicinski <kuba@kernel.org>, Xin Long <lucien.xin@gmail.com>, Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>, kernel test robot <oliver.sang@intel.com>, Soheil Hassas Yeganeh <soheil@google.com>, LKML <linux-kernel@vger.kernel.org>, network dev <netdev@vger.kernel.org>, linux-s390@vger.kernel.org, MPTCP Upstream <mptcp@lists.linux.dev>, "linux-sctp @ vger . kernel . org" <linux-sctp@vger.kernel.org>, lkp@lists.01.org, kbuild test robot <lkp@intel.com>, Huang Ying <ying.huang@intel.com>, Xing Zhengjun <zhengjun.xing@linux.intel.com>, Yin Fengwei <fengwei.yin@intel.com>, Ying Xu <yinxu@redhat.com> Subject: Re: [net] 4890b686f4: netperf.Throughput_Mbps -69.4% regression Date: Mon, 27 Jun 2022 23:12:58 +0800 [thread overview] Message-ID: <20220627151258.GB20878@shbuild999.sh.intel.com> (raw) In-Reply-To: <CALvZod7i_=7bNZR-LAXBPXJFxj-1KBuYs+rmG0iABAE1T90BPg@mail.gmail.com> On Mon, Jun 27, 2022 at 07:52:55AM -0700, Shakeel Butt wrote: > On Mon, Jun 27, 2022 at 5:34 AM Feng Tang <feng.tang@intel.com> wrote: > > Yes, 1% is just around noise level for a microbenchmark. > > > > I went check the original test data of Oliver's report, the tests was > > run 6 rounds and the performance data is pretty stable (0Day's report > > will show any std deviation bigger than 2%) > > > > The test platform is a 4 sockets 72C/144T machine, and I run the > > same job (nr_tasks = 25% * nr_cpus) on one CascadeLake AP (4 nodes) > > and one Icelake 2 sockets platform, and saw 75% and 53% regresson on > > them. > > > > In the first email, there is a file named 'reproduce', it shows the > > basic test process: > > > > " > > use 'performane' cpufre governor for all CPUs > > > > netserver -4 -D > > modprobe sctp > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > (repeat 36 times in total) > > ... > > > > " > > > > Which starts 36 (25% of nr_cpus) netperf clients. And the clients number > > also matters, I tried to increase the client number from 36 to 72(50%), > > and the regression is changed from 69.4% to 73.7% > > > > Am I understanding correctly that this 69.4% (or 73.7%) regression is > with cgroup v2? Yes. > Eric did the experiments on v2 but on real hardware where the > performance impact was negligible. > > BTW do you see similar regression for tcp as well or just sctp? Yes, I run TCP_SENDFILE case with 'send_size'==10K, it hits a 70%+ regressioin. Thanks, Feng
WARNING: multiple messages have this Message-ID (diff)
From: Feng Tang <feng.tang@intel.com> To: lkp@lists.01.org Subject: Re: [net] 4890b686f4: netperf.Throughput_Mbps -69.4% regression Date: Mon, 27 Jun 2022 23:12:58 +0800 [thread overview] Message-ID: <20220627151258.GB20878@shbuild999.sh.intel.com> (raw) In-Reply-To: <CALvZod7i_=7bNZR-LAXBPXJFxj-1KBuYs+rmG0iABAE1T90BPg@mail.gmail.com> [-- Attachment #1: Type: text/plain, Size: 1742 bytes --] On Mon, Jun 27, 2022 at 07:52:55AM -0700, Shakeel Butt wrote: > On Mon, Jun 27, 2022 at 5:34 AM Feng Tang <feng.tang@intel.com> wrote: > > Yes, 1% is just around noise level for a microbenchmark. > > > > I went check the original test data of Oliver's report, the tests was > > run 6 rounds and the performance data is pretty stable (0Day's report > > will show any std deviation bigger than 2%) > > > > The test platform is a 4 sockets 72C/144T machine, and I run the > > same job (nr_tasks = 25% * nr_cpus) on one CascadeLake AP (4 nodes) > > and one Icelake 2 sockets platform, and saw 75% and 53% regresson on > > them. > > > > In the first email, there is a file named 'reproduce', it shows the > > basic test process: > > > > " > > use 'performane' cpufre governor for all CPUs > > > > netserver -4 -D > > modprobe sctp > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > netperf -4 -H 127.0.0.1 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K & > > (repeat 36 times in total) > > ... > > > > " > > > > Which starts 36 (25% of nr_cpus) netperf clients. And the clients number > > also matters, I tried to increase the client number from 36 to 72(50%), > > and the regression is changed from 69.4% to 73.7% > > > > Am I understanding correctly that this 69.4% (or 73.7%) regression is > with cgroup v2? Yes. > Eric did the experiments on v2 but on real hardware where the > performance impact was negligible. > > BTW do you see similar regression for tcp as well or just sctp? Yes, I run TCP_SENDFILE case with 'send_size'==10K, it hits a 70%+ regressioin. Thanks, Feng
next prev parent reply other threads:[~2022-06-27 15:13 UTC|newest] Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-06-19 15:04 [net] 4890b686f4: netperf.Throughput_Mbps -69.4% regression kernel test robot 2022-06-19 15:04 ` kernel test robot 2022-06-23 0:28 ` Jakub Kicinski 2022-06-23 0:28 ` Jakub Kicinski 2022-06-23 3:08 ` Xin Long 2022-06-23 3:08 ` Xin Long 2022-06-23 22:50 ` Xin Long 2022-06-23 22:50 ` Xin Long 2022-06-24 1:57 ` Jakub Kicinski 2022-06-24 1:57 ` Jakub Kicinski 2022-06-24 4:13 ` Eric Dumazet 2022-06-24 4:13 ` Eric Dumazet 2022-06-24 4:22 ` Eric Dumazet 2022-06-24 4:22 ` Eric Dumazet 2022-06-24 5:13 ` Feng Tang 2022-06-24 5:13 ` Feng Tang 2022-06-24 5:45 ` Eric Dumazet 2022-06-24 5:45 ` Eric Dumazet 2022-06-24 6:00 ` Feng Tang 2022-06-24 6:00 ` Feng Tang 2022-06-24 6:07 ` Eric Dumazet 2022-06-24 6:07 ` Eric Dumazet 2022-06-24 6:34 ` Shakeel Butt 2022-06-24 6:34 ` Shakeel Butt 2022-06-24 7:06 ` Feng Tang 2022-06-24 7:06 ` Feng Tang 2022-06-24 14:43 ` Shakeel Butt 2022-06-24 14:43 ` Shakeel Butt 2022-06-25 2:36 ` Feng Tang 2022-06-25 2:36 ` Feng Tang 2022-06-27 2:38 ` Feng Tang 2022-06-27 2:38 ` Feng Tang 2022-06-27 8:46 ` Eric Dumazet 2022-06-27 8:46 ` Eric Dumazet 2022-06-27 12:34 ` Feng Tang 2022-06-27 12:34 ` Feng Tang 2022-06-27 14:07 ` Eric Dumazet 2022-06-27 14:07 ` Eric Dumazet 2022-06-27 14:48 ` Feng Tang 2022-06-27 14:48 ` Feng Tang 2022-06-27 16:25 ` Eric Dumazet 2022-06-27 16:25 ` Eric Dumazet 2022-06-27 16:48 ` Shakeel Butt 2022-06-27 16:48 ` Shakeel Butt 2022-06-27 17:05 ` Eric Dumazet 2022-06-27 17:05 ` Eric Dumazet 2022-06-28 1:46 ` Roman Gushchin 2022-06-28 1:46 ` Roman Gushchin 2022-06-28 3:49 ` Feng Tang 2022-06-28 3:49 ` Feng Tang 2022-07-01 15:47 ` Shakeel Butt 2022-07-01 15:47 ` Shakeel Butt 2022-07-03 10:43 ` Feng Tang 2022-07-03 10:43 ` Feng Tang 2022-07-03 22:55 ` Roman Gushchin 2022-07-03 22:55 ` Roman Gushchin 2022-07-05 5:03 ` Feng Tang 2022-07-05 5:03 ` Feng Tang 2022-08-16 5:52 ` Oliver Sang 2022-08-16 5:52 ` Oliver Sang 2022-08-16 15:55 ` Shakeel Butt 2022-08-16 15:55 ` Shakeel Butt 2022-06-27 14:52 ` Shakeel Butt 2022-06-27 14:52 ` Shakeel Butt 2022-06-27 14:56 ` Eric Dumazet 2022-06-27 14:56 ` Eric Dumazet 2022-06-27 15:12 ` Feng Tang [this message] 2022-06-27 15:12 ` Feng Tang 2022-06-27 16:25 ` Shakeel Butt 2022-06-27 16:25 ` Shakeel Butt
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220627151258.GB20878@shbuild999.sh.intel.com \ --to=feng.tang@intel.com \ --cc=akpm@linux-foundation.org \ --cc=edumazet@google.com \ --cc=fengwei.yin@intel.com \ --cc=hannes@cmpxchg.org \ --cc=kuba@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-s390@vger.kernel.org \ --cc=linux-sctp@vger.kernel.org \ --cc=lkp@intel.com \ --cc=lkp@lists.01.org \ --cc=lucien.xin@gmail.com \ --cc=marcelo.leitner@gmail.com \ --cc=mhocko@kernel.org \ --cc=mptcp@lists.linux.dev \ --cc=netdev@vger.kernel.org \ --cc=oliver.sang@intel.com \ --cc=roman.gushchin@linux.dev \ --cc=shakeelb@google.com \ --cc=soheil@google.com \ --cc=songmuchun@bytedance.com \ --cc=ying.huang@intel.com \ --cc=yinxu@redhat.com \ --cc=zhengjun.xing@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.