From: Mark Rutland <mark.rutland@arm.com>
To: Donglin Peng <pengdonglin@sangfor.com.cn>
Cc: mhiramat@kernel.org, rostedt@goodmis.org, linux@armlinux.org.uk,
will@kernel.org, catalin.marinas@arm.com, palmer@dabbelt.com,
paul.walmsley@sifive.com, tglx@linutronix.de,
dave.hansen@linux.intel.com, x86@kernel.org, mingo@redhat.com,
xiehuan09@gmail.com, dinghui@sangfor.com.cn,
huangcun@sangfor.com.cn, dolinux.peng@gmail.com,
linux-trace-kernel@vger.kernel.org,
linux-riscv@lists.infradead.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 1/2] function_graph: Support recording and printing the return value of function
Date: Tue, 21 Mar 2023 14:24:28 +0000 [thread overview]
Message-ID: <ZBm+HKK9laIXAdsc@FVFF77S0Q05N> (raw)
In-Reply-To: <20230320131650.482594-2-pengdonglin@sangfor.com.cn>
On Mon, Mar 20, 2023 at 06:16:49AM -0700, Donglin Peng wrote:
> When using the function_graph tracer to analyze system call failures,
> it can be time-consuming to analyze the trace logs and locate the kernel
> function that first returns an error. This change aims to simplify the
> process by recording the function return value to the 'retval' member of
> 'ftrace_graph_ent' and printing it when outputing the trace log.
>
> New trace options are introduced: funcgraph-retval and graph_retval_hex.
> The former is used to control whether to display the return value, while
> the latter is used to control the display format of the reutrn value.
>
> Note that even if a function's return type is void, a return value will
> still be printed, so just ignore it.
>
> Currently, this modification supports the following commonly used
> processor architectures: x86_64, x86, arm64, arm, riscv.
>
> Here is an example:
>
> I want to attach the demo process to a cpu cgroup, but it failed:
>
> echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
> -bash: echo: write error: Invalid argument
>
> The strace logs tells that the write system call returned -EINVAL(-22):
> ...
> write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
> ...
>
> Use the following commands to capture trace logs when calling the write
> system call:
>
> cd /sys/kernel/debug/tracing/
> echo 0 > tracing_on
> echo > trace
> echo *sys_write > set_graph_function
> echo *spin* > set_graph_notrace
> echo *rcu* >> set_graph_notrace
> echo *alloc* >> set_graph_notrace
> echo preempt* >> set_graph_notrace
> echo kfree* >> set_graph_notrace
> echo $$ > set_ftrace_pid
> echo function_graph > current_tracer
> echo 1 > tracing_on
> echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
> echo 0 > tracing_on
> echo 1 > options/funcgraph-retval
> cat trace > ~/trace.log
>
> Search the error code -22 directly in the file trace.log and find the
> first function that return -22, then read the function code to get the
> root cause.
>
> ...
>
> 1) | cgroup_migrate() {
> 1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
> 1) | cgroup_migrate_execute() {
> 1) | cpu_cgroup_can_attach() {
> 1) | cgroup_taskset_first() {
> 1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
> 1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
> 1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
> 1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
> 1) 4.369 us | } /* cgroup_migrate_execute = -22 */
> 1) 7.143 us | } /* cgroup_migrate = -22 */
>
> ...
>
> Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
> diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S
> index 350ed81324ac..d1a5d76e6d72 100644
> --- a/arch/arm64/kernel/entry-ftrace.S
> +++ b/arch/arm64/kernel/entry-ftrace.S
> @@ -276,7 +276,15 @@ SYM_CODE_START(return_to_handler)
> stp x4, x5, [sp, #32]
> stp x6, x7, [sp, #48]
>
> +#ifdef CONFIG_FUNCTION_GRAPH_RETVAL
> + /*
> + * Pass both the function return values in the register x0 and x1
> + * to ftrace_return_to_handler.
> + */
> + mov x2, x29 // parent's fp
> +#else
> mov x0, x29 // parent's fp
> +#endif
> bl ftrace_return_to_handler// addr = ftrace_return_to_hander(fp);
> mov x30, x0 // restore the original return address
Please don't make the calling convention of the asm change depending on a
selectable config option.
We already store the regs here; I'd be happy to make that a struct ftrace_regs
and pass a pointer to that to C code. Then it's be easy to acquire the value
you want in the exact same way as upon entry, and it'll work even if we decide
to return a structure by value somewhere (as that can use registers x2-x7 too).
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-03-21 14:25 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-20 13:16 [PATCH v5 0/2] function_graph: Support recording and printing the return value of function Donglin Peng
2023-03-20 13:16 ` [PATCH v5 1/2] " Donglin Peng
2023-03-21 14:09 ` Florian Kauer
2023-03-21 14:44 ` Steven Rostedt
2023-03-21 15:11 ` Florian Kauer
2023-03-22 5:01 ` Donglin Peng
2023-03-21 14:24 ` Mark Rutland [this message]
2023-03-22 8:47 ` Donglin Peng
2023-03-21 17:31 ` Russell King (Oracle)
2023-03-22 9:01 ` Donglin Peng
2023-03-20 13:16 ` [PATCH v5 2/2] tracing: Add documentation for funcgraph-retval and graph_retval_hex Donglin Peng
2023-03-21 2:31 ` Ding Hui
2023-03-21 3:24 ` Donglin Peng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZBm+HKK9laIXAdsc@FVFF77S0Q05N \
--to=mark.rutland@arm.com \
--cc=catalin.marinas@arm.com \
--cc=dave.hansen@linux.intel.com \
--cc=dinghui@sangfor.com.cn \
--cc=dolinux.peng@gmail.com \
--cc=huangcun@sangfor.com.cn \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=pengdonglin@sangfor.com.cn \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=xiehuan09@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).