From: Ard Biesheuvel <ard.biesheuvel@linaro.org> To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Arnd Bergmann <arnd@arndb.de>, Heiko Carstens <heiko.carstens@de.ibm.com>, Kees Cook <keescook@chromium.org>, Will Deacon <will.deacon@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Catalin Marinas <catalin.marinas@arm.com>, Ingo Molnar <mingo@redhat.com>, Steven Rostedt <rostedt@goodmis.org>, Martin Schwidefsky <schwidefsky@de.ibm.com>, Jessica Yu <jeyu@kernel.org>, Peter Zijlstra <peterz@infradead.org> Subject: [PATCH v3 5/9] x86: jump_label: switch to jump_entry accessors Date: Tue, 18 Sep 2018 23:51:40 -0700 [thread overview] Message-ID: <20180919065144.25010-6-ard.biesheuvel@linaro.org> (raw) In-Reply-To: <20180919065144.25010-1-ard.biesheuvel@linaro.org> In preparation of switching x86 to use place-relative references for the code, target and key members of struct jump_entry, replace direct references to the struct members with invocations of the new accessors. This will allow us to make the switch by modifying the accessors only. This incorporates a cleanup of __jump_label_transform() proposed by Peter. Cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/x86/kernel/jump_label.c | 62 ++++++++------------ 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index eeea935e9bb5..aac0c1f7e354 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -42,55 +42,40 @@ static void __ref __jump_label_transform(struct jump_entry *entry, void *(*poker)(void *, const void *, size_t), int init) { - union jump_code_union code; + union jump_code_union jmp; const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; + const void *expect, *code; + int line; + + jmp.jump = 0xe9; + jmp.offset = jump_entry_target(entry) - + (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); if (early_boot_irqs_disabled) poker = text_poke_early; if (type == JUMP_LABEL_JMP) { if (init) { - /* - * Jump label is enabled for the first time. - * So we expect a default_nop... - */ - if (unlikely(memcmp((void *)entry->code, default_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - /* - * ...otherwise expect an ideal_nop. Otherwise - * something went horribly wrong. - */ - if (unlikely(memcmp((void *)entry->code, ideal_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = ideal_nop; line = __LINE__; } - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); + code = &jmp.code; } else { - /* - * We are disabling this jump label. If it is not what - * we think it is, then something must have gone wrong. - * If this is the first initialization call, then we - * are converting the default nop to the ideal nop. - */ if (init) { - if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); - if (unlikely(memcmp((void *)entry->code, &code, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = &jmp.code; line = __LINE__; } - memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE); + + code = ideal_nop; } + if (memcmp((void *)jump_entry_code(entry), expect, JUMP_LABEL_NOP_SIZE)) + bug_at((void *)jump_entry_code(entry), line); + /* * Make text_poke_bp() a default fallback poker. * @@ -99,11 +84,14 @@ static void __ref __jump_label_transform(struct jump_entry *entry, * always nop being the 'currently valid' instruction * */ - if (poker) - (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE); - else - text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE, - (void *)entry->code + JUMP_LABEL_NOP_SIZE); + if (poker) { + (*poker)((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); + return; + } + + text_poke_bp((void *)jump_entry_code(entry), code, JUMP_LABEL_NOP_SIZE, + (void *)jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); } void arch_jump_label_transform(struct jump_entry *entry, -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: ard.biesheuvel@linaro.org (Ard Biesheuvel) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 5/9] x86: jump_label: switch to jump_entry accessors Date: Tue, 18 Sep 2018 23:51:40 -0700 [thread overview] Message-ID: <20180919065144.25010-6-ard.biesheuvel@linaro.org> (raw) In-Reply-To: <20180919065144.25010-1-ard.biesheuvel@linaro.org> In preparation of switching x86 to use place-relative references for the code, target and key members of struct jump_entry, replace direct references to the struct members with invocations of the new accessors. This will allow us to make the switch by modifying the accessors only. This incorporates a cleanup of __jump_label_transform() proposed by Peter. Cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/x86/kernel/jump_label.c | 62 ++++++++------------ 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index eeea935e9bb5..aac0c1f7e354 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -42,55 +42,40 @@ static void __ref __jump_label_transform(struct jump_entry *entry, void *(*poker)(void *, const void *, size_t), int init) { - union jump_code_union code; + union jump_code_union jmp; const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; + const void *expect, *code; + int line; + + jmp.jump = 0xe9; + jmp.offset = jump_entry_target(entry) - + (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); if (early_boot_irqs_disabled) poker = text_poke_early; if (type == JUMP_LABEL_JMP) { if (init) { - /* - * Jump label is enabled for the first time. - * So we expect a default_nop... - */ - if (unlikely(memcmp((void *)entry->code, default_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - /* - * ...otherwise expect an ideal_nop. Otherwise - * something went horribly wrong. - */ - if (unlikely(memcmp((void *)entry->code, ideal_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = ideal_nop; line = __LINE__; } - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); + code = &jmp.code; } else { - /* - * We are disabling this jump label. If it is not what - * we think it is, then something must have gone wrong. - * If this is the first initialization call, then we - * are converting the default nop to the ideal nop. - */ if (init) { - if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); - if (unlikely(memcmp((void *)entry->code, &code, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = &jmp.code; line = __LINE__; } - memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE); + + code = ideal_nop; } + if (memcmp((void *)jump_entry_code(entry), expect, JUMP_LABEL_NOP_SIZE)) + bug_at((void *)jump_entry_code(entry), line); + /* * Make text_poke_bp() a default fallback poker. * @@ -99,11 +84,14 @@ static void __ref __jump_label_transform(struct jump_entry *entry, * always nop being the 'currently valid' instruction * */ - if (poker) - (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE); - else - text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE, - (void *)entry->code + JUMP_LABEL_NOP_SIZE); + if (poker) { + (*poker)((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); + return; + } + + text_poke_bp((void *)jump_entry_code(entry), code, JUMP_LABEL_NOP_SIZE, + (void *)jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); } void arch_jump_label_transform(struct jump_entry *entry, -- 2.17.1
next prev parent reply other threads:[~2018-09-19 6:52 UTC|newest] Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-09-19 6:51 [PATCH v3 0/9] add support for relative references in jump tables Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 1/9] kernel/jump_label: abstract jump_entry member accessors Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:00 ` [tip:core/core] jump_label: Abstract " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 2/9] kernel/jump_label: implement generic support for relative references Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:01 ` [tip:core/core] jump_label: Implement " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 3/9] arm64/kernel: jump_label: switch to " Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:02 ` [tip:core/core] arm64/kernel: jump_label: Switch " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 4/9] x86: add support for 64-bit place relative relocations Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:02 ` [tip:core/core] x86: Add " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel [this message] 2018-09-19 6:51 ` [PATCH v3 5/9] x86: jump_label: switch to jump_entry accessors Ard Biesheuvel 2018-09-27 16:03 ` [tip:core/core] x86/jump_label: Switch " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 6/9] x86/kernel: jump_table: use relative references Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:03 ` [tip:core/core] x86/jump_table: Use " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 7/9] jump_label: annotate entries that operate on __init code earlier Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:04 ` [tip:core/core] jump_label: Annotate " tip-bot for Ard Biesheuvel 2018-09-19 6:51 ` [PATCH v3 8/9] jump_table: move entries into ro_after_init region Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:04 ` [tip:core/core] jump_table: Move " tip-bot for Ard Biesheuvel 2018-09-30 15:42 ` [PATCH v3 8/9] jump_table: move " Guenter Roeck 2018-09-30 15:42 ` Guenter Roeck 2018-09-19 6:51 ` [PATCH v3 9/9] s390/jump_label: switch to relative references Ard Biesheuvel 2018-09-19 6:51 ` Ard Biesheuvel 2018-09-27 16:05 ` [tip:core/core] s390/jump_label: Switch " tip-bot for Heiko Carstens 2018-09-19 13:08 ` [PATCH v3 0/9] add support for relative references in jump tables Peter Zijlstra 2018-09-19 13:08 ` Peter Zijlstra 2018-09-27 15:22 ` Ard Biesheuvel 2018-09-27 15:22 ` Ard Biesheuvel 2018-09-27 15:46 ` Kees Cook 2018-09-27 15:46 ` Kees Cook
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180919065144.25010-6-ard.biesheuvel@linaro.org \ --to=ard.biesheuvel@linaro.org \ --cc=arnd@arndb.de \ --cc=catalin.marinas@arm.com \ --cc=heiko.carstens@de.ibm.com \ --cc=jeyu@kernel.org \ --cc=keescook@chromium.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-s390@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=schwidefsky@de.ibm.com \ --cc=tglx@linutronix.de \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.