All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dave Hansen <dave.hansen@intel.com>
To: Nadav Amit <namit@vmware.com>
Cc: kernel test robot <oliver.sang@intel.com>,
	Ingo Molnar <mingo@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	LKML <linux-kernel@vger.kernel.org>,
	"lkp@lists.01.org" <lkp@lists.01.org>,
	"lkp@intel.com" <lkp@intel.com>,
	"ying.huang@intel.com" <ying.huang@intel.com>,
	"feng.tang@intel.com" <feng.tang@intel.com>,
	"zhengjun.xing@linux.intel.com" <zhengjun.xing@linux.intel.com>,
	"fengwei.yin@intel.com" <fengwei.yin@intel.com>,
	Andy Lutomirski <luto@kernel.org>
Subject: Re: [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression
Date: Thu, 17 Mar 2022 17:16:18 -0700	[thread overview]
Message-ID: <dd8be93c-ded6-b962-50d4-96b1c3afb2b7@intel.com> (raw)
In-Reply-To: <DC37F01B-A80F-4839-B4FB-C21F64943E64@vmware.com>

On 3/17/22 13:32, Nadav Amit wrote:
> I’m not married to this patch, but before a revert it would be good
> to know why it even matters. I wonder whether you can confirm that
> reverting the patch (without the rest of the series) even helps. If
> it does, I’ll try to run some tests to understand what the heck is
> going on.

I went back and tested on a "Intel(R) Core(TM) i7-8086K CPU @ 4.00GHz"
which is evidently a 6-core "Coffee Lake".  It needs retpolines:

> /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full
generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB
filling

I ran the will-it-scale test:

	./malloc1_threads -s 30 -t 12

and took the 30-second average "ops/sec" at the two commits:

	4c1ba3923e:197876
	6035152d8e:199367 +0.75%

Where bigger is better.  So, a small win, but probably mostly in the
noise.  The number of IPIs definitely went up, probably 3-4% to get that
win.

IPI costs go up the more threads you throw at it.  The retpolines do
too, though because you do *more* of them.  Systems with no retpolines
get hit harder by the IPI costs and have no upsides from removing the
retpoline.

So, we've got a small (<1%, possibly zero) win on the bulk of systems
(which have retpolines).  Newer, retpoline-free systems see a
double-digit regression.  The bigger the system, the bigger the
regression (probably).

I tend to think the bigger regression wins and we should probably revert
the patch, or at least back out its behavior.

Nadav, do you have some different data or a different take?

WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave.hansen@intel.com>
To: lkp@lists.01.org
Subject: Re: [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression
Date: Thu, 17 Mar 2022 17:16:18 -0700	[thread overview]
Message-ID: <dd8be93c-ded6-b962-50d4-96b1c3afb2b7@intel.com> (raw)
In-Reply-To: <DC37F01B-A80F-4839-B4FB-C21F64943E64@vmware.com>

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

On 3/17/22 13:32, Nadav Amit wrote:
> I’m not married to this patch, but before a revert it would be good
> to know why it even matters. I wonder whether you can confirm that
> reverting the patch (without the rest of the series) even helps. If
> it does, I’ll try to run some tests to understand what the heck is
> going on.

I went back and tested on a "Intel(R) Core(TM) i7-8086K CPU @ 4.00GHz"
which is evidently a 6-core "Coffee Lake".  It needs retpolines:

> /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full
generic retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB
filling

I ran the will-it-scale test:

	./malloc1_threads -s 30 -t 12

and took the 30-second average "ops/sec" at the two commits:

	4c1ba3923e:197876
	6035152d8e:199367 +0.75%

Where bigger is better.  So, a small win, but probably mostly in the
noise.  The number of IPIs definitely went up, probably 3-4% to get that
win.

IPI costs go up the more threads you throw at it.  The retpolines do
too, though because you do *more* of them.  Systems with no retpolines
get hit harder by the IPI costs and have no upsides from removing the
retpoline.

So, we've got a small (<1%, possibly zero) win on the bulk of systems
(which have retpolines).  Newer, retpoline-free systems see a
double-digit regression.  The bigger the system, the bigger the
regression (probably).

I tend to think the bigger regression wins and we should probably revert
the patch, or@least back out its behavior.

Nadav, do you have some different data or a different take?

  parent reply	other threads:[~2022-03-18  0:16 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-17  9:04 [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression kernel test robot
2022-03-17  9:04 ` kernel test robot
2022-03-17 18:38 ` Dave Hansen
2022-03-17 18:38   ` Dave Hansen
2022-03-17 19:02   ` Nadav Amit
2022-03-17 19:02     ` Nadav Amit
2022-03-17 19:11     ` Dave Hansen
2022-03-17 19:11       ` Dave Hansen
2022-03-17 20:32       ` Nadav Amit
2022-03-17 20:32         ` Nadav Amit
2022-03-17 20:49         ` Dave Hansen
2022-03-17 20:49           ` Dave Hansen
2022-03-18  2:56           ` Oliver Sang
2022-03-18  2:56             ` Oliver Sang
2022-03-18  0:16         ` Dave Hansen [this message]
2022-03-18  0:16           ` Dave Hansen
2022-03-18  0:20           ` Nadav Amit
2022-03-18  0:20             ` Nadav Amit
2022-03-18  0:45             ` Dave Hansen
2022-03-18  0:45               ` Dave Hansen
2022-03-18  3:02               ` Nadav Amit
2022-03-18  3:02                 ` Nadav Amit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=dd8be93c-ded6-b962-50d4-96b1c3afb2b7@intel.com \
    --to=dave.hansen@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=feng.tang@intel.com \
    --cc=fengwei.yin@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lkp@intel.com \
    --cc=lkp@lists.01.org \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=namit@vmware.com \
    --cc=oliver.sang@intel.com \
    --cc=ying.huang@intel.com \
    --cc=zhengjun.xing@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.