From: William Cohen <wcohen@redhat.com>
To: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>,
Steve Capper <steve.capper@linaro.org>,
David Long <dave.long@linaro.org>
Cc: "Jon Medhurst (Tixy)" <tixy@linaro.org>,
Russell King <linux@arm.linux.org.uk>,
Ananth N Mavinakayanahalli <ananth@in.ibm.com>,
Sandeepa Prabhu <sandeepa.prabhu@linaro.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
David Miller <davem@davemloft.net>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH v3 0/5] ARM64: Add kernel probes(Kprobes) support
Date: Wed, 03 Dec 2014 09:54:37 -0500 [thread overview]
Message-ID: <547F242D.1060705@redhat.com> (raw)
In-Reply-To: <547C36DB.7060903@hitachi.com>
On 12/01/2014 04:37 AM, Masami Hiramatsu wrote:
> (2014/11/29 1:01), Steve Capper wrote:
>> On 27 November 2014 at 06:07, Masami Hiramatsu
>> <masami.hiramatsu.pt@hitachi.com> wrote:
>>> (2014/11/27 3:59), Steve Capper wrote:
>>>> The crash is extremely easy to reproduce.
>>>>
>>>> I've not observed any missed events on a kprobe on an arm64 system
>>>> that's still alive.
>>>> My (limited!) understanding is that this suggests there could be a
>>>> problem with how missed events from a recursive call to memcpy are
>>>> being handled.
>>>
>>> I think so too. BTW, could you bisect that? :)
>>>
>>
>> I can't bisect, but the following functions look suspicious to me
>> (again I'm new to kprobes...):
>> kprobes_save_local_irqflag
>> kprobes_restore_local_irqflag
>>
>> I think these are breaking somehow when nested (i.e. from a recursive probe).
>
> Agreed. On x86, prev_kprobe has old_flags and saved_flags, this
> at least must have saved_irqflag and save/restore it in
> save/restore_previous_kprobe().
>
> What about adding this?
>
> struct prev_kprobe {
> struct kprobe *kp;
> unsigned int status;
> + unsigned long saved_irqflag;
> };
>
> and
>
> static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)
> {
> kcb->prev_kprobe.kp = kprobe_running();
> kcb->prev_kprobe.status = kcb->kprobe_status;
> + kcb->prev_kprobe.saved_irqflag = kcb->saved_irqflag;
> }
>
> static void __kprobes restore_previous_kprobe(struct kprobe_ctlblk *kcb)
> {
> __this_cpu_write(current_kprobe, kcb->prev_kprobe.kp);
> kcb->kprobe_status = kcb->prev_kprobe.status;
> + kcb->saved_irqflag = kcb->prev_kprobe.saved_irqflag;
> }
>
>
I have noticed with the aarch64 kprobe patches and recent kernel I can get the machine to end up getting stuck and printing out endless strings of
[187694.855843] Unexpected kernel single-step exception at EL1
[187694.861385] Unexpected kernel single-step exception at EL1
[187694.866926] Unexpected kernel single-step exception at EL1
[187694.872467] Unexpected kernel single-step exception at EL1
[187694.878009] Unexpected kernel single-step exception at EL1
[187694.883550] Unexpected kernel single-step exception at EL1
I can reproduce this pretty easily on my machine with functioncallcount.stp from https://sourceware.org/systemtap/examples/profiling/functioncallcount.stp and the following steps:
# stap -p4 -k -m mm_probes -w functioncallcount.stp "*@mm/*.c" -c "sleep 1"
# staprun mm_probes.ko -c "sleep 1"
-Will
>
>
>> That would explain why the state of play of the interrupts is in an
>> unexpected state in the crash I reported:
>> "The point of failure in the panic was:
>> fs/buffer.c:1257
>>
>> static inline void check_irqs_on(void)
>> {
>> #ifdef irqs_disabled
>> BUG_ON(irqs_disabled());
>> #endif
>> }
>> "
>>
>> This is all new to me so I'm still at the head-scratching stage.
>
> Ah, I see.
>
> Thank you,
>
>>
>> David,
>> Does the above make sense to you? Have you managed to reproduce the crash I get?
>>
>> Cheers,
>> --
>> Steve
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
>>
>
>
next prev parent reply other threads:[~2014-12-03 15:20 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-18 6:32 [PATCH v3 0/5] ARM64: Add kernel probes(Kprobes) support David Long
2014-11-18 6:32 ` [PATCH v3 1/5] arm64: Kprobes with single stepping support David Long
2014-11-18 13:28 ` Jon Medhurst (Tixy)
2014-11-21 4:28 ` David Long
2014-11-18 14:38 ` William Cohen
2014-11-18 14:39 ` William Cohen
2014-11-18 14:56 ` Will Deacon
2014-11-19 11:21 ` Sandeepa Prabhu
2014-11-19 11:25 ` Will Deacon
2014-11-19 14:55 ` David Long
2014-11-20 5:10 ` Sandeepa Prabhu
2014-11-26 6:46 ` David Long
2014-11-26 10:09 ` Will Deacon
2014-12-22 10:10 ` Pratyush Anand
2014-11-18 6:32 ` [PATCH v3 2/5] arm64: Kprobes instruction simulation support David Long
2014-11-18 14:43 ` William Cohen
2014-11-18 6:32 ` [PATCH v3 3/5] arm64: Add kernel return probes support(kretprobes) David Long
2014-11-18 14:50 ` William Cohen
2014-11-18 6:32 ` [PATCH v3 4/5] kprobes: Add arm64 case in kprobe example module David Long
2014-11-18 6:32 ` [PATCH v3 5/5] arm64: Add HAVE_REGS_AND_STACK_ACCESS_API feature David Long
2014-11-18 14:52 ` Will Deacon
2014-11-20 7:20 ` Masami Hiramatsu
2014-11-21 6:16 ` David Long
2014-11-20 15:02 ` [PATCH v3 0/5] ARM64: Add kernel probes(Kprobes) support Steve Capper
2014-11-26 8:33 ` Masami Hiramatsu
2014-11-26 10:03 ` Steve Capper
2014-11-26 17:46 ` David Long
2014-11-26 18:59 ` Steve Capper
2014-11-27 6:07 ` Masami Hiramatsu
2014-11-28 16:01 ` Steve Capper
2014-12-01 9:37 ` Masami Hiramatsu
2014-12-02 19:27 ` William Cohen
2014-12-02 20:00 ` William Cohen
2014-12-03 3:36 ` Masami Hiramatsu
2014-12-03 14:54 ` William Cohen [this message]
2014-12-03 22:54 ` David Long
2014-12-04 0:02 ` David Long
2014-12-04 1:16 ` William Cohen
2014-12-04 2:48 ` David Long
2014-12-04 10:21 ` Steve Capper
2014-12-04 10:43 ` Masami Hiramatsu
2014-12-04 11:29 ` Steve Capper
2014-12-04 11:53 ` Masami Hiramatsu
2014-12-09 13:33 ` Steve Capper
2014-12-09 14:27 ` David Long
2014-12-10 16:38 ` Steve Capper
2014-12-12 22:42 ` David Long
2014-12-12 23:10 ` Steve Capper
2014-12-15 5:58 ` Masami Hiramatsu
2014-12-15 6:29 ` David Long
2014-12-05 5:08 ` William Cohen
2014-11-27 5:13 ` Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=547F242D.1060705@redhat.com \
--to=wcohen@redhat.com \
--cc=ananth@in.ibm.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=catalin.marinas@arm.com \
--cc=dave.long@linaro.org \
--cc=davem@davemloft.net \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@arm.linux.org.uk \
--cc=masami.hiramatsu.pt@hitachi.com \
--cc=sandeepa.prabhu@linaro.org \
--cc=steve.capper@linaro.org \
--cc=tixy@linaro.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).