From: tengfeif@codeaurora.org
To: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: mark.rutland@arm.com, tengfei@codeaurora.org,
marc.zyngier@arm.com, catalin.marinas@arm.com,
will.deacon@arm.com, linux-kernel@vger.kernel.org,
andreyknvl@google.com, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] arm64: break while loop if task had been rescheduled
Date: Thu, 30 May 2019 09:38:47 +0800 [thread overview]
Message-ID: <253affe7493cb9689e224d74aa182765@codeaurora.org> (raw)
In-Reply-To: <f2d62227-4694-d973-cacc-8225e2b2baf4@arm.com>
On 2019-05-22 17:04, Anshuman Khandual wrote:
> On 05/21/2019 02:50 PM, Tengfei Fan wrote:
>> While printing a task's backtrace and this task isn't
>> current task, it is possible that task's fp and fp+8
>> have the same value, so cannot break the while loop.
>> This can break while loop if this task had been
>> rescheduled during print this task's backtrace.
>
> This is very confusing. IIUC it suggests that while printing
> the backtrace for non-current tasks the do/while loop does not
> exit because fp and fp+8 might have the same value ? When would
> this happen ? Even in that case the commit message here does not
> properly match the change in this patch.
In our issue, we got fp=pc=0xFFFFFF8025A13BA0, so cannot exit while
loop in dump_basktrace().
After analyze our issue's dump, we found one task(such as: task A)
is exiting via invoke do_exit() during another task is showing task
A's dumptask. In kernel code, do_exit() and exit_notify are defined
as follows:
void noreturn do_exit(long code)
{
......
exit_notify(tsk, group_dead);
......
}
static void exit_notify(struct task_struct *tsk, int group_dead)
{
......
}
Because of exit_notify() is a static function, so it is inlined to
do_exit() when compile kernel, so we can get partial assembly code
of do_exit() as follows:
……
{
bool autoreap;
struct task_struct *p, *n;
LIST_HEAD(dead);
write_lock_irq(&tasklist_lock);
c10: 90000000 adrp x0, 0 <tasklist_lock>
c14: 910003e8 mov x8, sp
c18: 91000000 add x0, x0, #0x0
*/
static void exit_notify(struct task_struct *tsk, int group_dead)
{
bool autoreap;
struct task_struct *p, *n;
LIST_HEAD(dead);
c1c: a90023e8 stp x8, x8, [sp]
write_lock_irq(&tasklist_lock);
c20: 94000000 bl 0 <_raw_write_lock_irq>
c24: f9435268 ldr x8, [x19,#1696]
……
From the code "c14:" and "c1c:", we will find sp's addr value is stored
in sp and sp+8, so sp's vaule equal (sp+8)'s value.
In our issue, there is a chance of fp point sp, so there will be
fp=pc=fp's
addr value,so code cannot break from while loop in dump_backtrace().
>
> This patch tries to stop printing the stack for non-current tasks
> if their state change while there is one dump_backtrace() trying
> to print back trace. Dont we have any lock preventing a task in
> this situation (while dumping it's backtrace) from running again
> or changing state.
I haven't found any lock preventing a task in this situation, and I
think we shouldn't
prevent task running if this task is scheduled.
>
>>
>> Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
>> ---
>> arch/arm64/kernel/traps.c | 23 +++++++++++++++++++++++
>> 1 file changed, 23 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
>> index 2975598..9df6e02 100644
>> --- a/arch/arm64/kernel/traps.c
>> +++ b/arch/arm64/kernel/traps.c
>> @@ -103,6 +103,9 @@ void dump_backtrace(struct pt_regs *regs, struct
>> task_struct *tsk)
>> {
>> struct stackframe frame;
>> int skip = 0;
>> + long cur_state = 0;
>> + unsigned long cur_sp = 0;
>> + unsigned long cur_fp = 0;
>>
>> pr_debug("%s(regs = %p tsk = %p)\n", __func__, regs, tsk);
>>
>> @@ -127,6 +130,9 @@ void dump_backtrace(struct pt_regs *regs, struct
>> task_struct *tsk)
>> */
>> frame.fp = thread_saved_fp(tsk);
>> frame.pc = thread_saved_pc(tsk);
>> + cur_state = tsk->state;
>> + cur_sp = thread_saved_sp(tsk);
>> + cur_fp = frame.fp;
>
> Should 'saved_state|sp|fp' instead as its applicable to non-current
> tasks only.
'saved_state|sp|fp' only applies to non-current tasks.
>
>> }
>> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
>> frame.graph = 0;
>> @@ -134,6 +140,23 @@ void dump_backtrace(struct pt_regs *regs, struct
>> task_struct *tsk)
>>
>> printk("Call trace:\n");
>> do {
>> + if (tsk != current && (cur_state != tsk->state
>> + /*
>> + * We would not be printing backtrace for the task
>> + * that has changed state from "saved" state to other
>> + * state before hitting the do-while loop but after
>> + * saving the current state. If task's current state
>
> This does not check any explicit task states like 'un-interruptible' or
> 'running' but instead tracks change from any previously 'saved' state.
have updated comments.
>
>
>> + * not equal the "saved" state, then we may print
>> + * wrong call trace or end up in infinite while loop
>> + * if *(fp) and *(fp+8) are same. While the situation
>
> Then dump_backtrace() must detect it, should not save it and just
> abort.
have updatd commentes.
>
>
>> + * should be stoped once we found the task's state
+ * is changed, so we detect the task's current state,
+ * sp and fp in each while.
>
> Thats not a reliable solution. AFICS we should not proceed further if
> there is a chance of an wrong trace or an infinite loop. Hoping that
> the printing will stop when task gets scheduled out does not seem
> right.
In this patch, it will break while loop and stop to print backtrace if
we
find the task's state change or there is a chance of an infinite loop.
>
>> + */
>> + || cur_sp != thread_saved_sp(tsk)
>> + || cur_fp != thread_saved_fp(tsk))) {
>
> Why does any of these three mismatches detect the problematic
> transition
> not just the state ?
1. we can use "cur_state != tsk->state" prevent printing backtrace if
the task's
state is changed after "saved" task's state.
2. we can use "cur_sp != thread_saved_sp(tsk)" and "cur_fp !=
thread_saved_fp(tsk)"
prevent printing backtrace if the task's state is changed before
"saved" task's
state. Because the value of "thread_saved_sp(tsk)" and
"thread_saved_fp(tsk)"
will not equal "saved" sp(cur_sp) and fp(cur_fp).
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-05-30 1:39 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-21 9:20 [PATCH] arm64: break while loop if task had been rescheduled Tengfei Fan
2019-05-22 9:04 ` Anshuman Khandual
2019-05-30 1:38 ` tengfeif [this message]
2019-05-24 10:41 ` Mark Rutland
2019-05-30 2:41 ` tengfeif
2019-05-24 3:16 tengfeif
2019-05-24 10:38 ` Mark Rutland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=253affe7493cb9689e224d74aa182765@codeaurora.org \
--to=tengfeif@codeaurora.org \
--cc=andreyknvl@google.com \
--cc=anshuman.khandual@arm.com \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=marc.zyngier@arm.com \
--cc=mark.rutland@arm.com \
--cc=tengfei@codeaurora.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).