From: Peter Zijlstra <peterz@infradead.org>
To: Ard Biesheuvel <ardb@kernel.org>
Cc: mark.rutland@arm.com, catalin.marinas@arm.com, will@kernel.org,
james.morse@arm.com, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2] arm64: implement support for static call trampolines
Date: Thu, 29 Oct 2020 11:40:07 +0100 [thread overview]
Message-ID: <20201029104007.GK2628@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20201028184114.6834-1-ardb@kernel.org>
On Wed, Oct 28, 2020 at 07:41:14PM +0100, Ard Biesheuvel wrote:
> +/*
> + * The static call trampoline consists of one of the following sequences:
> + *
> + * (A) (B) (C) (D) (E)
> + * 00: BTI C BTI C BTI C BTI C BTI C
> + * 04: B fn NOP NOP NOP NOP
> + * 08: RET RET ADRP X16, fn ADRP X16, fn ADRP X16, fn
> + * 0c: NOP NOP ADD X16, fn ADD X16, fn ADD X16, fn
> + * 10: BR X16 RET NOP
> + * 14: ADRP X16, &fn
> + * 18: LDR X16, [X16, &fn]
> + * 1c: BR X16
> + *
> + * The architecture permits us to patch B instructions into NOPs or vice versa
> + * directly, but patching any other instruction sequence requires careful
> + * synchronization. Since branch targets may be out of range for ordinary
> + * immediate branch instructions, we may have to fall back to ADRP/ADD/BR
> + * sequences in some cases, which complicates things considerably; since any
> + * sleeping tasks may have been preempted right in the middle of any of these
> + * sequences, we have to carefully transform one into the other, and ensure
> + * that it is safe to resume execution at any point in the sequence for tasks
> + * that have already executed part of it.
> + *
> + * So the rules are:
> + * - we start out with (A) or (B)
> + * - a branch within immediate range can always be patched in at offset 0x4;
> + * - sequence (A) can be turned into (B) for NULL branch targets;
> + * - a branch outside of immediate range can be patched using (C), but only if
> + * . the sequence being updated is (A) or (B), or
> + * . the branch target address modulo 4k results in the same ADD opcode
> + * (which could occur when patching the same far target a second time)
> + * - once we have patched in (C) we cannot go back to (A) or (B), so patching
> + * in a NULL target now requires sequence (D);
> + * - if we cannot patch a far target using (C), we fall back to sequence (E),
> + * which loads the function pointer from memory.
> + *
> + * If we abide by these rules, then the following must hold for tasks that were
> + * interrupted halfway through execution of the trampoline:
> + * - when resuming at offset 0x8, we can only encounter a RET if (B) or (D)
> + * was patched in at any point, and therefore a NULL target is valid;
> + * - when resuming at offset 0xc, we are executing the ADD opcode that is only
> + * reachable via the preceding ADRP, and which is patched in only a single
> + * time, and is therefore guaranteed to be consistent with the ADRP target;
> + * - when resuming at offset 0x10, X16 must refer to a valid target, since it
> + * is only reachable via a ADRP/ADD pair that is guaranteed to be consistent.
> + *
> + * Note that sequence (E) is only used when switching between multiple far
> + * targets, and that it is not a terminal degraded state.
> + */
Would it make things easier if your trampoline consisted of two complete
slots, between which you can flip?
Something like:
0x00 B 0x24 / NOP
0x04 < slot 1 >
....
0x20
0x24 < slot 2 >
....
0x40
Then each (20 byte) slot can contain any of the variants above and you
can write the unused slot without stop-machine. Then, when the unused
slot is populated, flip the initial instruction (like a static-branch),
issue synchronize_rcu_tasks() and flip to using the other slot for next
time.
Alternatively, you can patch the call-sites to point to the alternative
trampoline slot, but that might be pushing things a bit.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-10-29 10:40 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-28 18:41 [PATCH v2] arm64: implement support for static call trampolines Ard Biesheuvel
2020-10-29 10:28 ` Peter Zijlstra
2020-10-29 10:40 ` Peter Zijlstra [this message]
2020-10-29 10:58 ` Ard Biesheuvel
2020-10-29 11:46 ` Peter Zijlstra
2020-10-29 11:49 ` Ard Biesheuvel
2020-10-29 11:54 ` Peter Zijlstra
2020-10-29 12:14 ` Ard Biesheuvel
2020-10-29 11:27 ` Quentin Perret
2020-10-29 11:32 ` Ard Biesheuvel
2020-10-29 11:44 ` Peter Zijlstra
2020-10-29 14:10 ` Steven Rostedt
2020-10-29 11:54 ` Quentin Perret
2020-10-29 13:22 ` Ard Biesheuvel
2020-11-16 10:18 ` Quentin Perret
2020-11-16 10:31 ` Ard Biesheuvel
2020-11-16 12:05 ` Quentin Perret
2020-10-29 11:50 ` Mark Rutland
2020-10-29 11:58 ` Peter Zijlstra
2020-10-29 13:30 ` Mark Rutland
2020-10-29 11:59 ` Ard Biesheuvel
2020-10-29 13:21 ` Mark Rutland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201029104007.GK2628@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=ardb@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=james.morse@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=mark.rutland@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).