From: Nadav Amit <namit@vmware.com>
To: Andy Lutomirski <luto@kernel.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>,
Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
"H. Peter Anvin" <hpa@zytor.com>,
Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
Borislav Petkov <bp@alien8.de>,
"Woodhouse, David" <dwmw@amazon.co.uk>
Subject: Re: [RFC PATCH 0/5] x86: dynamic indirect call promotion
Date: Thu, 29 Nov 2018 02:06:05 +0000 [thread overview]
Message-ID: <0E75C656-18BF-4967-98A3-35E0BD83D603@vmware.com> (raw)
In-Reply-To: <CALCETrUfC37TxuYLgqbN3_pZ1RzfpV03byKMpSt87o+JHtk6FQ@mail.gmail.com>
> On Nov 28, 2018, at 5:40 PM, Andy Lutomirski <luto@kernel.org> wrote:
>
> On Wed, Nov 28, 2018 at 4:38 PM Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>> On Wed, Nov 28, 2018 at 07:34:52PM +0000, Nadav Amit wrote:
>>>> On Nov 28, 2018, at 8:08 AM, Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>>>>
>>>> On Wed, Oct 17, 2018 at 05:54:15PM -0700, Nadav Amit wrote:
>>>>> This RFC introduces indirect call promotion in runtime, which for the
>>>>> matter of simplification (and branding) will be called here "relpolines"
>>>>> (relative call + trampoline). Relpolines are mainly intended as a way
>>>>> of reducing retpoline overheads due to Spectre v2.
>>>>>
>>>>> Unlike indirect call promotion through profile guided optimization, the
>>>>> proposed approach does not require a profiling stage, works well with
>>>>> modules whose address is unknown and can adapt to changing workloads.
>>>>>
>>>>> The main idea is simple: for every indirect call, we inject a piece of
>>>>> code with fast- and slow-path calls. The fast path is used if the target
>>>>> matches the expected (hot) target. The slow-path uses a retpoline.
>>>>> During training, the slow-path is set to call a function that saves the
>>>>> call source and target in a hash-table and keep count for call
>>>>> frequency. The most common target is then patched into the hot path.
>>>>>
>>>>> The patching is done on-the-fly by patching the conditional branch
>>>>> (opcode and offset) that is used to compare the target to the hot
>>>>> target. This allows to direct all cores to the fast-path, while patching
>>>>> the slow-path and vice-versa. Patching follows 2 more rules: (1) Only
>>>>> patch a single byte when the code might be executed by any core. (2)
>>>>> When patching more than one byte, ensure that all cores do not run the
>>>>> to-be-patched-code by preventing this code from being preempted, and
>>>>> using synchronize_sched() after patching the branch that jumps over this
>>>>> code.
>>>>>
>>>>> Changing all the indirect calls to use relpolines is done using assembly
>>>>> macro magic. There are alternative solutions, but this one is
>>>>> relatively simple and transparent. There is also logic to retrain the
>>>>> software predictor, but the policy it uses may need to be refined.
>>>>>
>>>>> Eventually the results are not bad (2 VCPU VM, throughput reported):
>>>>>
>>>>> base relpoline
>>>>> ---- ---------
>>>>> nginx 22898 25178 (+10%)
>>>>> redis-ycsb 24523 25486 (+4%)
>>>>> dbench 2144 2103 (+2%)
>>>>>
>>>>> When retpolines are disabled, and if retraining is off, performance
>>>>> benefits are up to 2% (nginx), but are much less impressive.
>>>>
>>>> Hi Nadav,
>>>>
>>>> Peter pointed me to these patches during a discussion about retpoline
>>>> profiling. Personally, I think this is brilliant. This could help
>>>> networking and filesystem intensive workloads a lot.
>>>
>>> Thanks! I was a bit held-back by the relatively limited number of responses.
>>
>> It is a rather, erm, ambitious idea, maybe they were speechless :-)
>>
>>> I finished another version two weeks ago, and every day I think: "should it
>>> be RFCv2 or v1”, ending up not sending it…
>>>
>>> There is one issue that I realized while working on the new version: I’m not
>>> sure it is well-defined what an outline retpoline is allowed to do. The
>>> indirect branch promotion code can change rflags, which might cause
>>> correction issues. In practice, using gcc, it is not a problem.
>>
>> Callees can clobber flags, so it seems fine to me.
>
> Just to check I understand your approach right: you made a macro
> called "call", and you're therefore causing all instances of "call" to
> become magic? This is... terrifying. It's even plausibly worse than
> "#define if" :) The scariest bit is that it will impact inline asm as
> well. Maybe a gcc plugin would be less alarming?
It is likely to look less alarming. When I looked at the inline retpoline
implementation of gcc, it didn’t look much better than what I did - it
basically just emits assembly instructions.
Anyhow, I look (again) into using gcc-plugins.
>>> 1. An indirect branch inside the BP handler might be the one we patch
>>
>> I _think_ nested INT3s should be doable, because they don't use IST.
>> Maybe Andy can clarify.
>
> int3 should survive recursion these days. Although I admit I'm
> currently wondering what happens if one thread puts a kprobe on an
> address that another thread tries to text_poke.
The issue I regarded is having an indirect call *inside* the the handler.
For example, you try to patch the call to bp_int3_handler and then get an
int3. They can be annotated to prevent them from being patched. Then again,
I need to see how gcc plugins can get these annotations.
>
> Also, this relpoline magic is likely to start patching text at runtime
> on a semi-regular basis. This type of patching is *slow*. Is it a
> problem?
It didn’t appear so. Although there are >10000 indirect branches in the
kernel, you don’t patch too many of them even you are doing relearning.
>
>>> 2. An indirect branch inside an interrupt or NMI handler might be the
>>> one we patch
>>
>> But INT3s just use the existing stack, and NMIs support nesting, so I'm
>> thinking that should also be doable. Andy?
>
> In principle, as long as the code isn't NOKPROBE_SYMBOL-ified, we
> should be fine, right? I'd be a little nervous if we get an int3 in
> the C code that handles the early part of an NMI from user mode. It's
> *probably* okay, but one of the alarming issues is that the int3
> return path will implicitly unmask NMI, which isn't fantastic. Maybe
> we finally need to dust off my old "return using RET" code to get rid
> of that problem.
So it may be possible. It would require having a new text_poke_bp() variant
for multiple instructions. text_poke_bp() might be slower though.
next prev parent reply other threads:[~2018-11-29 2:06 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-18 0:54 [RFC PATCH 0/5] x86: dynamic indirect call promotion Nadav Amit
2018-10-18 0:54 ` [RFC PATCH 1/5] x86: introduce preemption disable prefix Nadav Amit
2018-10-18 1:22 ` Andy Lutomirski
2018-10-18 3:12 ` Nadav Amit
2018-10-18 3:26 ` Nadav Amit
2018-10-18 3:51 ` Andy Lutomirski
2018-10-18 16:47 ` Nadav Amit
2018-10-18 17:00 ` Andy Lutomirski
2018-10-18 17:25 ` Nadav Amit
2018-10-18 17:29 ` Andy Lutomirski
2018-10-18 17:42 ` Nadav Amit
2018-10-19 1:08 ` Nadav Amit
2018-10-19 4:29 ` Andy Lutomirski
2018-10-19 4:44 ` Nadav Amit
2018-10-20 1:22 ` Masami Hiramatsu
2018-10-19 5:00 ` Alexei Starovoitov
2018-10-19 8:22 ` Peter Zijlstra
2018-10-19 14:47 ` Alexei Starovoitov
2018-10-19 8:19 ` Peter Zijlstra
2018-10-19 10:38 ` Oleg Nesterov
2018-10-19 8:33 ` Peter Zijlstra
2018-10-19 14:29 ` Andy Lutomirski
2018-11-29 9:46 ` Peter Zijlstra
2018-10-18 7:54 ` Peter Zijlstra
2018-10-18 18:14 ` Nadav Amit
2018-10-18 0:54 ` [RFC PATCH 2/5] x86: patch indirect branch promotion Nadav Amit
2018-10-18 0:54 ` [RFC PATCH 3/5] x86: interface for accessing indirect branch locations Nadav Amit
2018-10-18 0:54 ` [RFC PATCH 4/5] x86: learning and patching indirect branch targets Nadav Amit
2018-10-18 0:54 ` [RFC PATCH 5/5] x86: relpoline: disabling interface Nadav Amit
2018-10-23 18:36 ` [RFC PATCH 0/5] x86: dynamic indirect call promotion Dave Hansen
2018-10-23 20:32 ` Nadav Amit
2018-10-23 20:37 ` Dave Hansen
2018-11-28 16:08 ` Josh Poimboeuf
2018-11-28 19:34 ` Nadav Amit
2018-11-29 0:38 ` Josh Poimboeuf
2018-11-29 1:40 ` Andy Lutomirski
2018-11-29 2:06 ` Nadav Amit [this message]
2018-11-29 3:24 ` Andy Lutomirski
2018-11-29 4:36 ` Josh Poimboeuf
2018-11-29 6:06 ` Andy Lutomirski
2018-11-29 15:19 ` Josh Poimboeuf
2018-12-01 6:52 ` Nadav Amit
2018-12-01 14:25 ` Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0E75C656-18BF-4967-98A3-35E0BD83D603@vmware.com \
--to=namit@vmware.com \
--cc=bp@alien8.de \
--cc=dwmw@amazon.co.uk \
--cc=hpa@zytor.com \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).