From: "Mickaël Salaün" <mic@digikod.net>
To: Kees Cook <keescook@chromium.org>,
kernel test robot <oliver.sang@intel.com>
Cc: lkp@lists.01.org, lkp@intel.com, ying.huang@intel.com,
feng.tang@intel.com, zhengjun.xing@linux.intel.com,
fengwei.yin@intel.com, "Al Viro" <viro@zeniv.linux.org.uk>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Aleksa Sarai" <cyphar@cyphar.com>,
"Andy Lutomirski" <luto@kernel.org>,
"Arnd Bergmann" <arnd@arndb.de>,
"Casey Schaufler" <casey@schaufler-ca.com>,
"Christian Brauner" <christian.brauner@ubuntu.com>,
"Christian Heimes" <christian@python.org>,
"Deven Bowers" <deven.desai@linux.microsoft.com>,
"Dmitry Vyukov" <dvyukov@google.com>,
"Eric Biggers" <ebiggers@kernel.org>,
"Eric Chiang" <ericchiang@google.com>,
"Florian Weimer" <fweimer@redhat.com>,
"Geert Uytterhoeven" <geert@linux-m68k.org>,
"James Morris" <jmorris@namei.org>, "Jan Kara" <jack@suse.cz>,
"Jann Horn" <jannh@google.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Lakshmi Ramasubramanian" <nramas@linux.microsoft.com>,
"Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
"Matthew Garrett" <mjg59@google.com>,
"Matthew Wilcox" <willy@infradead.org>,
"Miklos Szeredi" <mszeredi@redhat.com>,
"Mimi Zohar" <zohar@linux.ibm.com>,
"Paul Moore" <paul@paul-moore.com>,
"Philippe Trébuchet" <philippe.trebuchet@ssi.gouv.fr>,
"Scott Shell" <scottsh@microsoft.com>,
"Shuah Khan" <shuah@kernel.org>,
"Steve Dower" <steve.dower@python.org>,
"Steve Grubb" <sgrubb@redhat.com>,
"Thibaut Sautereau" <thibaut.sautereau@ssi.gouv.fr>,
"Vincent Strubel" <vincent.strubel@ssi.gouv.fr>,
kernel-hardening@lists.openwall.com, linux-api@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-integrity@vger.kernel.org,
linux-kernel@vger.kernel.org,
linux-security-module@vger.kernel.org,
"Mickaël Salaün" <mic@linux.microsoft.com>
Subject: Re: [fs] a0918006f9: netperf.Throughput_tps -11.6% regression
Date: Wed, 10 Nov 2021 09:52:51 +0100 [thread overview]
Message-ID: <95966337-b36e-f45e-6b16-f433bcb90c4d@digikod.net> (raw)
In-Reply-To: <202111090920.4958E610D1@keescook>
On 09/11/2021 18:21, Kees Cook wrote:
> On Fri, Nov 05, 2021 at 02:41:59PM +0800, kernel test robot wrote:
>>
>>
>> Greeting,
>>
>> FYI, we noticed a -11.6% regression of netperf.Throughput_tps due to commit:
>>
>>
>> commit: a0918006f9284b77397ae4f163f055c3e0f987b2 ("[PATCH v15 1/3] fs: Add trusted_for(2) syscall implementation and related sysctl")
>> url: https://github.com/0day-ci/linux/commits/Micka-l-Sala-n/Add-trusted_for-2-was-O_MAYEXEC/20211013-032533
>> patch link: https://lore.kernel.org/kernel-hardening/20211012192410.2356090-2-mic@digikod.net
>>
>> in testcase: netperf
>> on test machine: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
>> with following parameters:
>>
>> ip: ipv4
>> runtime: 300s
>> nr_threads: 16
>> cluster: cs-localhost
>> test: TCP_CRR
>> cpufreq_governor: performance
>> ucode: 0x5003006
>>
>> test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
>> test-url: http://www.netperf.org/netperf/
>>
>>
>> please be noted we made out some further analysis/tests, as Fengwei mentioned:
>> ==============================================================================
>> Here is my investigation result of this regression:
>>
>> If I add patch to make sure the kernel function address and data address is
>> almost same even with this patch, there is almost no performance delta(0.1%)
>> w/o the patch.
>>
>> And if I only make sure function address same w/o the patch, the performance
>> delta is about 5.1%.
>>
>> So suppose this regression is triggered by different function and data address.
>> We don't know why the different address could bring such kind of regression yet
>> ===============================================================================
>>
>>
>> we also tested on other platforms.
>> on a Cooper Lake (Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory),
>> we also observed regression but the gap is smaller:
>> =========================================================================================
>> cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
>> cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/16/debian-10.4-x86_64-20200603.cgz/300s/lkp-cpl-4sp1/TCP_CRR/netperf/0x700001e
>>
>> commit:
>> v5.15-rc4
>> a0918006f9284b77397ae4f163f055c3e0f987b2
>>
>> v5.15-rc4 a0918006f9284b77397ae4f163f
>> ---------------- ---------------------------
>> %stddev %change %stddev
>> \ | \
>> 333492 -5.7% 314346 ± 2% netperf.Throughput_total_tps
>> 20843 -4.5% 19896 netperf.Throughput_tps
>>
>>
>> but no regression on a 96 threads 2 sockets Ice Lake with 256G memory:
>> =========================================================================================
>> cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode:
>> cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/16/debian-10.4-x86_64-20200603.cgz/300s/lkp-icl-2sp1/TCP_CRR/netperf/0xb000280
>>
>> commit:
>> v5.15-rc4
>> a0918006f9284b77397ae4f163f055c3e0f987b2
>>
>> v5.15-rc4 a0918006f9284b77397ae4f163f
>> ---------------- ---------------------------
>> %stddev %change %stddev
>> \ | \
>> 555600 -0.1% 555305 netperf.Throughput_total_tps
>> 34725 -0.1% 34706 netperf.Throughput_tps
>>
>>
>> Fengwei also helped review these results and commented:
>> I suppose these three CPUs have different cache policy. It also could be
>> related with netperf throughput testing.
>
> Does moving the syscall implementation somewhere else change things?
> That's a _huge_ performance change for something that isn't even called.
> What's going on here?
This regression doesn't make sense. I guess this is the result of a
flaky netperf test, maybe because the test machine was overloaded at
that time.
next prev parent reply other threads:[~2021-11-10 8:52 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-12 19:24 [PATCH v15 0/3] Add trusted_for(2) (was O_MAYEXEC) Mickaël Salaün
2021-10-12 19:24 ` [PATCH v15 1/3] fs: Add trusted_for(2) syscall implementation and related sysctl Mickaël Salaün
2021-11-05 6:41 ` [fs] a0918006f9: netperf.Throughput_tps -11.6% regression kernel test robot
2021-11-09 17:21 ` Kees Cook
2021-11-10 1:54 ` Yin Fengwei
2021-11-10 8:52 ` Mickaël Salaün [this message]
2021-11-11 3:29 ` Yin Fengwei
2021-11-12 12:25 ` Yin Fengwei
2021-10-12 19:24 ` [PATCH v15 2/3] arch: Wire up trusted_for(2) Mickaël Salaün
2021-10-12 19:24 ` [PATCH v15 3/3] selftest/interpreter: Add tests for trusted_for(2) policies Mickaël Salaün
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=95966337-b36e-f45e-6b16-f433bcb90c4d@digikod.net \
--to=mic@digikod.net \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=casey@schaufler-ca.com \
--cc=christian.brauner@ubuntu.com \
--cc=christian@python.org \
--cc=corbet@lwn.net \
--cc=cyphar@cyphar.com \
--cc=deven.desai@linux.microsoft.com \
--cc=dvyukov@google.com \
--cc=ebiggers@kernel.org \
--cc=ericchiang@google.com \
--cc=feng.tang@intel.com \
--cc=fengwei.yin@intel.com \
--cc=fweimer@redhat.com \
--cc=geert@linux-m68k.org \
--cc=jack@suse.cz \
--cc=jannh@google.com \
--cc=jmorris@namei.org \
--cc=keescook@chromium.org \
--cc=kernel-hardening@lists.openwall.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-integrity@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-security-module@vger.kernel.org \
--cc=lkp@intel.com \
--cc=lkp@lists.01.org \
--cc=luto@kernel.org \
--cc=madvenka@linux.microsoft.com \
--cc=mic@linux.microsoft.com \
--cc=mjg59@google.com \
--cc=mszeredi@redhat.com \
--cc=nramas@linux.microsoft.com \
--cc=oliver.sang@intel.com \
--cc=paul@paul-moore.com \
--cc=philippe.trebuchet@ssi.gouv.fr \
--cc=scottsh@microsoft.com \
--cc=sgrubb@redhat.com \
--cc=shuah@kernel.org \
--cc=steve.dower@python.org \
--cc=thibaut.sautereau@ssi.gouv.fr \
--cc=vincent.strubel@ssi.gouv.fr \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
--cc=ying.huang@intel.com \
--cc=zhengjun.xing@linux.intel.com \
--cc=zohar@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).