From: Dave Hansen <dave.hansen@intel.com> To: Nadav Amit <namit@vmware.com> Cc: kernel test robot <oliver.sang@intel.com>, Ingo Molnar <mingo@kernel.org>, Dave Hansen <dave.hansen@linux.intel.com>, LKML <linux-kernel@vger.kernel.org>, "lkp@lists.01.org" <lkp@lists.01.org>, "lkp@intel.com" <lkp@intel.com>, "ying.huang@intel.com" <ying.huang@intel.com>, "feng.tang@intel.com" <feng.tang@intel.com>, "zhengjun.xing@linux.intel.com" <zhengjun.xing@linux.intel.com>, "fengwei.yin@intel.com" <fengwei.yin@intel.com> Subject: Re: [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression Date: Thu, 17 Mar 2022 12:11:08 -0700 [thread overview] Message-ID: <96f9b880-876f-bf4d-8eb0-9ae8bbc8df6d@intel.com> (raw) In-Reply-To: <3B958B13-75F0-4B81-B8CF-99CD140436EB@vmware.com> On 3/17/22 12:02, Nadav Amit wrote: >> This new "early lazy check" behavior could theoretically work both ways. >> If threads tended to be waking up from idle when TLB flushes were being >> sent, this would tend to reduce the number of IPIs. But, since they >> tend to be going to sleep it increases the number of IPIs. >> >> Anybody have a better theory? I think we should probably revert the commit. > > Let’s get back to the motivation behind this patch. > > Originally we had an indirect branch that on system which are > vulnerable to Spectre v2 translates into a retpoline. > > So I would not paraphrase this patch purpose as “early lazy check” > but instead “more efficient lazy check”. There is very little code > that was executed between the call to on_each_cpu_cond_mask() and > the actual check of tlb_is_not_lazy(). So what it seems to happen > in this test-case - according to what you say - is that *slower* > checks of is-lazy allows to send fewer IPIs since some cores go > into idle-state. > > Was this test run with retpolines? If there is a difference in > performance without retpoline - I am probably wrong. Nope, no retpolines: > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling which is the same situation as the "Xeon Platinum 8358" which found this in 0day. Maybe the increased IPIs with this approach end up being a wash with the reduced retpoline overhead. Did you have any specific performance numbers that show the benefit on retpoline systems?
WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave.hansen@intel.com> To: lkp@lists.01.org Subject: Re: [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression Date: Thu, 17 Mar 2022 12:11:08 -0700 [thread overview] Message-ID: <96f9b880-876f-bf4d-8eb0-9ae8bbc8df6d@intel.com> (raw) In-Reply-To: <3B958B13-75F0-4B81-B8CF-99CD140436EB@vmware.com> [-- Attachment #1: Type: text/plain, Size: 1590 bytes --] On 3/17/22 12:02, Nadav Amit wrote: >> This new "early lazy check" behavior could theoretically work both ways. >> If threads tended to be waking up from idle when TLB flushes were being >> sent, this would tend to reduce the number of IPIs. But, since they >> tend to be going to sleep it increases the number of IPIs. >> >> Anybody have a better theory? I think we should probably revert the commit. > > Let’s get back to the motivation behind this patch. > > Originally we had an indirect branch that on system which are > vulnerable to Spectre v2 translates into a retpoline. > > So I would not paraphrase this patch purpose as “early lazy check” > but instead “more efficient lazy check”. There is very little code > that was executed between the call to on_each_cpu_cond_mask() and > the actual check of tlb_is_not_lazy(). So what it seems to happen > in this test-case - according to what you say - is that *slower* > checks of is-lazy allows to send fewer IPIs since some cores go > into idle-state. > > Was this test run with retpolines? If there is a difference in > performance without retpoline - I am probably wrong. Nope, no retpolines: > /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling which is the same situation as the "Xeon Platinum 8358" which found this in 0day. Maybe the increased IPIs with this approach end up being a wash with the reduced retpoline overhead. Did you have any specific performance numbers that show the benefit on retpoline systems?
next prev parent reply other threads:[~2022-03-17 19:11 UTC|newest] Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-17 9:04 [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression kernel test robot 2022-03-17 9:04 ` kernel test robot 2022-03-17 18:38 ` Dave Hansen 2022-03-17 18:38 ` Dave Hansen 2022-03-17 19:02 ` Nadav Amit 2022-03-17 19:02 ` Nadav Amit 2022-03-17 19:11 ` Dave Hansen [this message] 2022-03-17 19:11 ` Dave Hansen 2022-03-17 20:32 ` Nadav Amit 2022-03-17 20:32 ` Nadav Amit 2022-03-17 20:49 ` Dave Hansen 2022-03-17 20:49 ` Dave Hansen 2022-03-18 2:56 ` Oliver Sang 2022-03-18 2:56 ` Oliver Sang 2022-03-18 0:16 ` Dave Hansen 2022-03-18 0:16 ` Dave Hansen 2022-03-18 0:20 ` Nadav Amit 2022-03-18 0:20 ` Nadav Amit 2022-03-18 0:45 ` Dave Hansen 2022-03-18 0:45 ` Dave Hansen 2022-03-18 3:02 ` Nadav Amit 2022-03-18 3:02 ` Nadav Amit
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=96f9b880-876f-bf4d-8eb0-9ae8bbc8df6d@intel.com \ --to=dave.hansen@intel.com \ --cc=dave.hansen@linux.intel.com \ --cc=feng.tang@intel.com \ --cc=fengwei.yin@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=lkp@intel.com \ --cc=lkp@lists.01.org \ --cc=mingo@kernel.org \ --cc=namit@vmware.com \ --cc=oliver.sang@intel.com \ --cc=ying.huang@intel.com \ --cc=zhengjun.xing@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.