On Wed, Dec 2, 2020 at 10:31 PM kernel test robot <oliver.sang@intel.com> wrote:
>
> FYI, we noticed a -4.8% regression of will-it-scale.per_process_ops due to commit:

Ok, I guess that's bigger than expected, but the profile data does
show how bad the indirect branches are.

There's both a "direct" cost of them:

>       0.55 ą 14%      +0.3        0.87 ą 15%  perf-profile.children.cycles-pp.__x86_retpoline_rax
>       0.12 ą 14%      +0.1        0.19 ą 14%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
>       0.43 ą 14%      +0.3        0.68 ą 15%  perf-profile.self.cycles-pp.__x86_retpoline_rax

The actual retpoline profile costs themselves do not add up to 4%, but
I think that's because the indirect costs are higher, because the
branch mis-predicts will basically make everything run slower for a
while as the OoO engine needs to restart.

So the global cost then shows up in CPU and branch miss stats, where
the IPC goes down (which is the same thing as saying that CPI goes
up):

>  1.741e+08           +42.3%  2.476e+08        perf-stat.i.branch-misses
>       0.74            -3.9%       0.71        perf-stat.overall.ipc
>       1.35            +4.1%       1.41        perf-stat.overall.cpi

which is why it ends up being so costly even if the retpoline overhead
itself is "only" just under 1%.

           Linus