All of lore.kernel.org
 help / color / mirror / Atom feed
From: AKASHI Takahiro <takahiro.akashi@linaro.org>
To: Jungseok Lee <jungseoklee85@gmail.com>
Cc: catalin.marinas@arm.com, will.deacon@arm.com,
	rostedt@goodmis.org, broonie@kernel.org, david.griego@linaro.org,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 0/6] arm64: ftrace: fix incorrect output from stack tracer
Date: Tue, 10 Nov 2015 11:58:24 +0900	[thread overview]
Message-ID: <56415D50.40500@linaro.org> (raw)
In-Reply-To: <252DE845-D0C2-460C-8161-090ED08396A4@gmail.com>

On 11/09/2015 11:24 PM, Jungseok Lee wrote:
> On Nov 6, 2015, at 3:44 PM, AKASHI Takahiro wrote:
>
> Hi Akashi,
>
>> This is the fifth patch series for fixing stack tracer on arm64.
>> The original issue was reported by Jungseok[1], and then I found more
>> issues[2].
>>
>> We don't have to care about the original issue because the root cause
>> (patch "ARM64: unwind: Fix PC calculation") has been reverted in v4.3.
>>
>> I address here all the issues and implement fixes described in [2] except
>> for interrupt-triggered problems(II-3) and leaf function(II-5).  Recent
>> discussions[3] about introducing a dedicated interrupt stack suggests that
>> we may avoid walking through from an interrupt stack to a process stack.
>> (So interrupt-stack patch is a prerequisite.)
>>
>> Basically,
>> patch1 is a proactive improvement of function_graph tracer.
>> patch2 corresponds to II-4(functions under function_graph tracer).
>> patch3, 4 and 5 correspond to II-1(slurping stack) and II-2(differences
>> between x86 and arm64).
>> patch6 is a function prologue analyzer test. This won't attest
>> the correctness of the functionality, but it can suggest that all
>> the traced functions are treated properly by this function.
>> (Please note that patch3 has already been queued in Steven's for-next.)
>>
>> I tested the code with v4.3 + Jungseok's patch v5[4].
>
> I've played this series with IRQ stack patch and it works well at least
> on my system! In addition to this condition, I've run these changes without
> IRQ stack since it is in progress. I could observe a single strange behavior,
> minus stack size around elX_irq. Am I missing something?

You saw the result like:
  ...
  13)     4336      64   gic_handle_irq+0x5c/0xa4
  14)     4272     576   el1_irq+0x68/0xd8
  15)     3696    -160   smc_hardware_send_pkt+0x278/0x42c

This is the most difficult problem that I mentioned in II-3 of [1] and tried to fix.
For example, smc_hardware_send_pkt is NOT the function interrupted, but
_raw_spin_unlock_irqstore which is called at '+0x278/0x42c' is.
Giving a *perfect* solution against it is quite tough (and complicated).
Since you have introduced interrupt stack and even on x86 an interrupt stack is
not supported, I removed related patches.

-Takahiro AKASHI

> My reproduction scenario is simple.
>
>    $ sudo echo 1 > /proc/sys/kernel/stack_trace_enabled
>    $ sudo echo function_graph > /sys/kernel/debug/tracing/current_tracer
>    $ [ Run any workload ]
>    $ sudo cat /sys/kernel/debug/tracing/stack_trace
>
> Best Regards
> Jungseok Lee
>

WARNING: multiple messages have this Message-ID (diff)
From: takahiro.akashi@linaro.org (AKASHI Takahiro)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v5 0/6] arm64: ftrace: fix incorrect output from stack tracer
Date: Tue, 10 Nov 2015 11:58:24 +0900	[thread overview]
Message-ID: <56415D50.40500@linaro.org> (raw)
In-Reply-To: <252DE845-D0C2-460C-8161-090ED08396A4@gmail.com>

On 11/09/2015 11:24 PM, Jungseok Lee wrote:
> On Nov 6, 2015, at 3:44 PM, AKASHI Takahiro wrote:
>
> Hi Akashi,
>
>> This is the fifth patch series for fixing stack tracer on arm64.
>> The original issue was reported by Jungseok[1], and then I found more
>> issues[2].
>>
>> We don't have to care about the original issue because the root cause
>> (patch "ARM64: unwind: Fix PC calculation") has been reverted in v4.3.
>>
>> I address here all the issues and implement fixes described in [2] except
>> for interrupt-triggered problems(II-3) and leaf function(II-5).  Recent
>> discussions[3] about introducing a dedicated interrupt stack suggests that
>> we may avoid walking through from an interrupt stack to a process stack.
>> (So interrupt-stack patch is a prerequisite.)
>>
>> Basically,
>> patch1 is a proactive improvement of function_graph tracer.
>> patch2 corresponds to II-4(functions under function_graph tracer).
>> patch3, 4 and 5 correspond to II-1(slurping stack) and II-2(differences
>> between x86 and arm64).
>> patch6 is a function prologue analyzer test. This won't attest
>> the correctness of the functionality, but it can suggest that all
>> the traced functions are treated properly by this function.
>> (Please note that patch3 has already been queued in Steven's for-next.)
>>
>> I tested the code with v4.3 + Jungseok's patch v5[4].
>
> I've played this series with IRQ stack patch and it works well at least
> on my system! In addition to this condition, I've run these changes without
> IRQ stack since it is in progress. I could observe a single strange behavior,
> minus stack size around elX_irq. Am I missing something?

You saw the result like:
  ...
  13)     4336      64   gic_handle_irq+0x5c/0xa4
  14)     4272     576   el1_irq+0x68/0xd8
  15)     3696    -160   smc_hardware_send_pkt+0x278/0x42c

This is the most difficult problem that I mentioned in II-3 of [1] and tried to fix.
For example, smc_hardware_send_pkt is NOT the function interrupted, but
_raw_spin_unlock_irqstore which is called at '+0x278/0x42c' is.
Giving a *perfect* solution against it is quite tough (and complicated).
Since you have introduced interrupt stack and even on x86 an interrupt stack is
not supported, I removed related patches.

-Takahiro AKASHI

> My reproduction scenario is simple.
>
>    $ sudo echo 1 > /proc/sys/kernel/stack_trace_enabled
>    $ sudo echo function_graph > /sys/kernel/debug/tracing/current_tracer
>    $ [ Run any workload ]
>    $ sudo cat /sys/kernel/debug/tracing/stack_trace
>
> Best Regards
> Jungseok Lee
>

  reply	other threads:[~2015-11-10  2:58 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-06  6:44 [PATCH v5 0/6] arm64: ftrace: fix incorrect output from stack tracer AKASHI Takahiro
2015-11-06  6:44 ` AKASHI Takahiro
2015-11-06  6:44 ` [PATCH v5 1/6] arm64: ftrace: modify a stack frame in a safe way AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-06  6:44 ` [PATCH v5 2/6] arm64: ftrace: fix a stack tracer's output under function graph tracer AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-09 14:04   ` Jungseok Lee
2015-11-09 14:04     ` Jungseok Lee
2015-11-10  2:42     ` AKASHI Takahiro
2015-11-10  2:42       ` AKASHI Takahiro
2015-11-13 15:01       ` Jungseok Lee
2015-11-13 15:01         ` Jungseok Lee
2015-11-16  9:23         ` AKASHI Takahiro
2015-11-16  9:23           ` AKASHI Takahiro
2015-11-06  6:44 ` [PATCH v5 3/6] ftrace: allow arch-specific stack tracer AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-06 13:39   ` Steven Rostedt
2015-11-06 13:39     ` Steven Rostedt
2015-11-06  6:44 ` [PATCH v5 4/6] arm64: insn: add instruction decoders for ldp/stp and add/sub AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-10 13:40   ` Jungseok Lee
2015-11-10 13:40     ` Jungseok Lee
2015-11-11  4:54     ` AKASHI Takahiro
2015-11-11  4:54       ` AKASHI Takahiro
2015-11-06  6:44 ` [PATCH v5 5/6] arm64: ftrace: add arch-specific stack tracer AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-10 14:05   ` Jungseok Lee
2015-11-10 14:05     ` Jungseok Lee
2015-11-11  5:03     ` AKASHI Takahiro
2015-11-11  5:03       ` AKASHI Takahiro
2015-11-11 22:56       ` Jungseok Lee
2015-11-11 22:56         ` Jungseok Lee
2015-11-06  6:44 ` [PATCH v5 6/6] arm64: ftrace: add a test of function prologue analyzer AKASHI Takahiro
2015-11-06  6:44   ` AKASHI Takahiro
2015-11-09 14:24 ` [PATCH v5 0/6] arm64: ftrace: fix incorrect output from stack tracer Jungseok Lee
2015-11-09 14:24   ` Jungseok Lee
2015-11-10  2:58   ` AKASHI Takahiro [this message]
2015-11-10  2:58     ` AKASHI Takahiro
2015-11-10 13:32     ` Jungseok Lee
2015-11-10 13:32       ` Jungseok Lee

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=56415D50.40500@linaro.org \
    --to=takahiro.akashi@linaro.org \
    --cc=broonie@kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=david.griego@linaro.org \
    --cc=jungseoklee85@gmail.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.