From: Florent Revest <revest@chromium.org> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, lihuafei1@huawei.com, Florent Revest <revest@chromium.org> Subject: [PATCH v2 04/10] ftrace: Store direct called addresses in their ops Date: Tue, 7 Feb 2023 19:21:29 +0100 [thread overview] Message-ID: <20230207182135.2671106-5-revest@chromium.org> (raw) In-Reply-To: <20230207182135.2671106-1-revest@chromium.org> All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest <revest@chromium.org> --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a7dbd307c3a4..84f717f8959e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index cb77a0a208c7..dfa5f34ec320 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2577,9 +2577,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = ops->direct_call; - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5375,6 +5374,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + WRITE_ONCE(ops->direct_call, addr); mutex_unlock(&ftrace_lock); -- 2.39.1.519.gcb327c4b5f-goog
WARNING: multiple messages have this Message-ID (diff)
From: Florent Revest <revest@chromium.org> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, lihuafei1@huawei.com, Florent Revest <revest@chromium.org> Subject: [PATCH v2 04/10] ftrace: Store direct called addresses in their ops Date: Tue, 7 Feb 2023 19:21:29 +0100 [thread overview] Message-ID: <20230207182135.2671106-5-revest@chromium.org> (raw) In-Reply-To: <20230207182135.2671106-1-revest@chromium.org> All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest <revest@chromium.org> --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a7dbd307c3a4..84f717f8959e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index cb77a0a208c7..dfa5f34ec320 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2577,9 +2577,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = ops->direct_call; - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5375,6 +5374,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + WRITE_ONCE(ops->direct_call, addr); mutex_unlock(&ftrace_lock); -- 2.39.1.519.gcb327c4b5f-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-02-07 18:23 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-02-07 18:21 [PATCH v2 00/10] Add ftrace direct call for arm64 Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 01/10] ftrace: Replace uses of _ftrace_direct APIs with _ftrace_direct_multi Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-03-15 23:33 ` Steven Rostedt 2023-03-15 23:33 ` Steven Rostedt 2023-03-16 15:40 ` Florent Revest 2023-03-16 15:40 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 02/10] ftrace: Remove the legacy _ftrace_direct API Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 03/10] ftrace: Rename _ftrace_direct_multi APIs to _ftrace_direct APIs Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` Florent Revest [this message] 2023-02-07 18:21 ` [PATCH v2 04/10] ftrace: Store direct called addresses in their ops Florent Revest 2023-03-15 23:43 ` Steven Rostedt 2023-03-15 23:43 ` Steven Rostedt 2023-03-16 15:40 ` Florent Revest 2023-03-16 15:40 ` Florent Revest 2023-03-16 15:45 ` Steven Rostedt 2023-03-16 15:45 ` Steven Rostedt 2023-03-16 16:15 ` Florent Revest 2023-03-16 16:15 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 05/10] ftrace: Make DIRECT_CALLS work WITH_ARGS and !WITH_REGS Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 06/10] ftrace: selftest: remove broken trace_direct_tramp Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-03-15 23:51 ` Steven Rostedt 2023-03-15 23:51 ` Steven Rostedt 2023-03-16 15:41 ` Florent Revest 2023-03-16 15:41 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 07/10] arm64: ftrace: Add direct call support Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 08/10] arm64: ftrace: Simplify get_ftrace_plt Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 09/10] arm64: ftrace: Add direct call trampoline samples support Florent Revest 2023-02-07 18:21 ` Florent Revest 2023-02-07 18:21 ` [PATCH v2 10/10] selftests/bpf: Update the tests deny list on aarch64 Florent Revest 2023-02-07 18:21 ` Florent Revest
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230207182135.2671106-5-revest@chromium.org \ --to=revest@chromium.org \ --cc=andrii@kernel.org \ --cc=ast@kernel.org \ --cc=bpf@vger.kernel.org \ --cc=catalin.marinas@arm.com \ --cc=daniel@iogearbox.net \ --cc=jolsa@kernel.org \ --cc=kpsingh@kernel.org \ --cc=lihuafei1@huawei.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-trace-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=mhiramat@kernel.org \ --cc=rostedt@goodmis.org \ --cc=will@kernel.org \ --cc=xukuohai@huaweicloud.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.