From: Christophe Leroy <christophe.leroy@csgroup.eu> To: Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, Michael Ellerman <mpe@ellerman.id.au>, peterz@infradead.org, aik@ozlabs.ru, sv@linux.ibm.com, rostedt@goodmis.org, jpoimboe@redhat.com, naveen.n.rao@linux.vnet.ibm.com, mbenes@suse.cz Cc: Christophe Leroy <christophe.leroy@csgroup.eu>, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH v1 4/4] powerpc/static_call: Implement inline static calls Date: Wed, 25 May 2022 17:58:17 +0200 [thread overview] Message-ID: <2e74b10072ca594c394dd1c445d827505725f27a.1653494186.git.christophe.leroy@csgroup.eu> (raw) In-Reply-To: <cover.1653494186.git.christophe.leroy@csgroup.eu> Implement inline static calls: - Put a 'bl' to the destination function - Put a 'nop' when the destination function is NULL - Put a 'li r3,0' when the destination is the RET0 function For the time being it only works if the destination is within 32Mb from the caller. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> --- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/static_call.h | 2 + arch/powerpc/kernel/static_call.c | 41 ++++++++++++------- tools/objtool/arch/powerpc/include/arch/elf.h | 1 + 4 files changed, 31 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 5ef8bf8eb202..3257a1c258d8 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -246,6 +246,7 @@ config PPC select HAVE_STACKPROTECTOR if PPC32 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r2) select HAVE_STACKPROTECTOR if PPC64 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r13) select HAVE_STATIC_CALL if PPC32 + select HAVE_STATIC_CALL_INLINE if PPC32 select HAVE_SYSCALL_TRACEPOINTS select HAVE_VIRT_CPU_ACCOUNTING select HUGETLB_PAGE_SIZE_VARIABLE if PPC_BOOK3S_64 && HUGETLB_PAGE diff --git a/arch/powerpc/include/asm/static_call.h b/arch/powerpc/include/asm/static_call.h index de1018cc522b..e3d5d3823dac 100644 --- a/arch/powerpc/include/asm/static_call.h +++ b/arch/powerpc/include/asm/static_call.h @@ -26,4 +26,6 @@ #define ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) __PPC_SCT(name, "blr") #define ARCH_DEFINE_STATIC_CALL_RET0_TRAMP(name) __PPC_SCT(name, "b .+20") +#define CALL_INSN_SIZE 4 + #endif /* _ASM_POWERPC_STATIC_CALL_H */ diff --git a/arch/powerpc/kernel/static_call.c b/arch/powerpc/kernel/static_call.c index 863a7aa24650..fd25954cfd24 100644 --- a/arch/powerpc/kernel/static_call.c +++ b/arch/powerpc/kernel/static_call.c @@ -9,25 +9,38 @@ void arch_static_call_transform(void *site, void *tramp, void *func, bool tail) int err; bool is_ret0 = (func == __static_call_return0); unsigned long target = (unsigned long)(is_ret0 ? tramp + PPC_SCT_RET0 : func); - bool is_short = is_offset_in_branch_range((long)target - (long)tramp); - - if (!tramp) - return; mutex_lock(&text_mutex); - if (func && !is_short) { - err = patch_instruction(tramp + PPC_SCT_DATA, ppc_inst(target)); - if (err) - goto out; + if (tramp) { + bool is_short = is_offset_in_branch_range((long)target - (long)tramp); + + if (func && !is_short) { + err = patch_instruction(tramp + PPC_SCT_DATA, ppc_inst(target)); + if (err) + goto out; + } + + if (!func) + err = patch_instruction(tramp, ppc_inst(PPC_RAW_BLR())); + else if (is_short) + err = patch_branch(tramp, target, 0); + else + err = patch_instruction(tramp, ppc_inst(PPC_RAW_NOP())); } - if (!func) - err = patch_instruction(tramp, ppc_inst(PPC_RAW_BLR())); - else if (is_short) - err = patch_branch(tramp, target, 0); - else - err = patch_instruction(tramp, ppc_inst(PPC_RAW_NOP())); + if (site) { + bool is_short = is_offset_in_branch_range((long)func - (long)site); + + if (!func) + err = patch_instruction(site, ppc_inst(PPC_RAW_NOP())); + else if (is_ret0) + err = patch_instruction(site, ppc_inst(PPC_RAW_LI(_R3, 0))); + else if (is_short) + err = patch_branch(site, target, BRANCH_SET_LINK); + else + panic("%s: function %pS is out of reach of %pS\n", __func__, func, site); + } out: mutex_unlock(&text_mutex); diff --git a/tools/objtool/arch/powerpc/include/arch/elf.h b/tools/objtool/arch/powerpc/include/arch/elf.h index 3c8ebb7d2a6b..18784c764c14 100644 --- a/tools/objtool/arch/powerpc/include/arch/elf.h +++ b/tools/objtool/arch/powerpc/include/arch/elf.h @@ -4,5 +4,6 @@ #define _OBJTOOL_ARCH_ELF #define R_NONE R_PPC_NONE +#define R_REL32 R_PPC_REL32 #endif /* _OBJTOOL_ARCH_ELF */ -- 2.35.3
WARNING: multiple messages have this Message-ID (diff)
From: Christophe Leroy <christophe.leroy@csgroup.eu> To: Benjamin Herrenschmidt <benh@kernel.crashing.org>, Paul Mackerras <paulus@samba.org>, Michael Ellerman <mpe@ellerman.id.au>, peterz@infradead.org, aik@ozlabs.ru, sv@linux.ibm.com, rostedt@goodmis.org, jpoimboe@redhat.com, naveen.n.rao@linux.vnet.ibm.com, mbenes@suse.cz Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH v1 4/4] powerpc/static_call: Implement inline static calls Date: Wed, 25 May 2022 17:58:17 +0200 [thread overview] Message-ID: <2e74b10072ca594c394dd1c445d827505725f27a.1653494186.git.christophe.leroy@csgroup.eu> (raw) In-Reply-To: <cover.1653494186.git.christophe.leroy@csgroup.eu> Implement inline static calls: - Put a 'bl' to the destination function - Put a 'nop' when the destination function is NULL - Put a 'li r3,0' when the destination is the RET0 function For the time being it only works if the destination is within 32Mb from the caller. Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu> --- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/static_call.h | 2 + arch/powerpc/kernel/static_call.c | 41 ++++++++++++------- tools/objtool/arch/powerpc/include/arch/elf.h | 1 + 4 files changed, 31 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 5ef8bf8eb202..3257a1c258d8 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -246,6 +246,7 @@ config PPC select HAVE_STACKPROTECTOR if PPC32 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r2) select HAVE_STACKPROTECTOR if PPC64 && $(cc-option,-mstack-protector-guard=tls -mstack-protector-guard-reg=r13) select HAVE_STATIC_CALL if PPC32 + select HAVE_STATIC_CALL_INLINE if PPC32 select HAVE_SYSCALL_TRACEPOINTS select HAVE_VIRT_CPU_ACCOUNTING select HUGETLB_PAGE_SIZE_VARIABLE if PPC_BOOK3S_64 && HUGETLB_PAGE diff --git a/arch/powerpc/include/asm/static_call.h b/arch/powerpc/include/asm/static_call.h index de1018cc522b..e3d5d3823dac 100644 --- a/arch/powerpc/include/asm/static_call.h +++ b/arch/powerpc/include/asm/static_call.h @@ -26,4 +26,6 @@ #define ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) __PPC_SCT(name, "blr") #define ARCH_DEFINE_STATIC_CALL_RET0_TRAMP(name) __PPC_SCT(name, "b .+20") +#define CALL_INSN_SIZE 4 + #endif /* _ASM_POWERPC_STATIC_CALL_H */ diff --git a/arch/powerpc/kernel/static_call.c b/arch/powerpc/kernel/static_call.c index 863a7aa24650..fd25954cfd24 100644 --- a/arch/powerpc/kernel/static_call.c +++ b/arch/powerpc/kernel/static_call.c @@ -9,25 +9,38 @@ void arch_static_call_transform(void *site, void *tramp, void *func, bool tail) int err; bool is_ret0 = (func == __static_call_return0); unsigned long target = (unsigned long)(is_ret0 ? tramp + PPC_SCT_RET0 : func); - bool is_short = is_offset_in_branch_range((long)target - (long)tramp); - - if (!tramp) - return; mutex_lock(&text_mutex); - if (func && !is_short) { - err = patch_instruction(tramp + PPC_SCT_DATA, ppc_inst(target)); - if (err) - goto out; + if (tramp) { + bool is_short = is_offset_in_branch_range((long)target - (long)tramp); + + if (func && !is_short) { + err = patch_instruction(tramp + PPC_SCT_DATA, ppc_inst(target)); + if (err) + goto out; + } + + if (!func) + err = patch_instruction(tramp, ppc_inst(PPC_RAW_BLR())); + else if (is_short) + err = patch_branch(tramp, target, 0); + else + err = patch_instruction(tramp, ppc_inst(PPC_RAW_NOP())); } - if (!func) - err = patch_instruction(tramp, ppc_inst(PPC_RAW_BLR())); - else if (is_short) - err = patch_branch(tramp, target, 0); - else - err = patch_instruction(tramp, ppc_inst(PPC_RAW_NOP())); + if (site) { + bool is_short = is_offset_in_branch_range((long)func - (long)site); + + if (!func) + err = patch_instruction(site, ppc_inst(PPC_RAW_NOP())); + else if (is_ret0) + err = patch_instruction(site, ppc_inst(PPC_RAW_LI(_R3, 0))); + else if (is_short) + err = patch_branch(site, target, BRANCH_SET_LINK); + else + panic("%s: function %pS is out of reach of %pS\n", __func__, func, site); + } out: mutex_unlock(&text_mutex); diff --git a/tools/objtool/arch/powerpc/include/arch/elf.h b/tools/objtool/arch/powerpc/include/arch/elf.h index 3c8ebb7d2a6b..18784c764c14 100644 --- a/tools/objtool/arch/powerpc/include/arch/elf.h +++ b/tools/objtool/arch/powerpc/include/arch/elf.h @@ -4,5 +4,6 @@ #define _OBJTOOL_ARCH_ELF #define R_NONE R_PPC_NONE +#define R_REL32 R_PPC_REL32 #endif /* _OBJTOOL_ARCH_ELF */ -- 2.35.3
next prev parent reply other threads:[~2022-05-25 15:59 UTC|newest] Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-05-25 15:58 [RFC PATCH v1 0/4] Implement inline static calls on PPC32 Christophe Leroy 2022-05-25 15:58 ` Christophe Leroy 2022-05-25 15:58 ` [RFC PATCH v1 1/4] Revert "objtool: Enable objtool to run only on files with ftrace enabled" Christophe Leroy 2022-05-25 15:58 ` Christophe Leroy 2022-05-25 16:34 ` Peter Zijlstra 2022-05-25 16:34 ` Peter Zijlstra 2022-05-25 17:03 ` Christophe Leroy 2022-05-25 17:03 ` Christophe Leroy 2022-05-25 15:58 ` [RFC PATCH v1 2/4] objtool: Add R_REL32 macro Christophe Leroy 2022-05-25 15:58 ` Christophe Leroy 2022-05-25 16:40 ` Segher Boessenkool 2022-05-25 16:40 ` Segher Boessenkool 2022-05-25 15:58 ` [RFC PATCH v1 3/4] static_call: Call static_call_init() from start_kernel() Christophe Leroy 2022-05-25 15:58 ` Christophe Leroy 2022-05-25 15:58 ` Christophe Leroy [this message] 2022-05-25 15:58 ` [RFC PATCH v1 4/4] powerpc/static_call: Implement inline static calls Christophe Leroy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=2e74b10072ca594c394dd1c445d827505725f27a.1653494186.git.christophe.leroy@csgroup.eu \ --to=christophe.leroy@csgroup.eu \ --cc=aik@ozlabs.ru \ --cc=benh@kernel.crashing.org \ --cc=jpoimboe@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mbenes@suse.cz \ --cc=mpe@ellerman.id.au \ --cc=naveen.n.rao@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=sv@linux.ibm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.