From: David Long <dave.long@linaro.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com>, William Cohen <wcohen@redhat.com>, Pratyush Anand <panand@redhat.com>, Steve Capper <steve.capper@linaro.org>, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Marc Zyngier <marc.zyngier@arm.com> Cc: "Dave P Martin" <Dave.Martin@arm.com>, "Mark Rutland" <mark.rutland@arm.com>, "Robin Murphy" <Robin.Murphy@arm.com>, "Ard Biesheuvel" <ard.biesheuvel@linaro.org>, "Jens Wiklander" <jens.wiklander@linaro.org>, "Christoffer Dall" <christoffer.dall@linaro.org>, "Alex Bennée" <alex.bennee@linaro.org>, "Yang Shi" <yang.shi@linaro.org>, "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>, "Viresh Kumar" <viresh.kumar@linaro.org>, "Suzuki K. Poulose" <suzuki.poulose@arm.com>, "Kees Cook" <keescook@chromium.org>, "Zi Shen Lim" <zlim.lnx@gmail.com>, "John Blackwood" <john.blackwood@ccur.com>, "Feng Kan" <fkan@apm.com>, "Balamurugan Shanmugam" <bshanmugam@apm.com>, "James Morse" <james.morse@arm.com>, "Vladimir Murzin" <Vladimir.Murzin@arm.com>, "Mark Salyzyn" <salyzyn@android.com>, "Petr Mladek" <pmladek@suse.com>, "Andrew Morton" <akpm@linux-foundation.org>, "Mark Brown" <broonie@kernel.org> Subject: [PATCH v11 8/9] arm64: Add kernel return probes support (kretprobes) Date: Wed, 9 Mar 2016 00:32:22 -0500 [thread overview] Message-ID: <1457501543-24197-9-git-send-email-dave.long@linaro.org> (raw) In-Reply-To: <1457501543-24197-1-git-send-email-dave.long@linaro.org> From: Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com> The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com> Signed-off-by: David A. Long <dave.long@linaro.org> --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/kprobes.c | 75 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c395386..72412de 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -82,6 +82,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index bd3f233..13d3333 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -534,7 +534,80 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return NULL; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + /* return 1 so that post handlers not called */ + return (void *) orig_ret_addr; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void) -- 2.5.0
WARNING: multiple messages have this Message-ID (diff)
From: dave.long@linaro.org (David Long) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v11 8/9] arm64: Add kernel return probes support (kretprobes) Date: Wed, 9 Mar 2016 00:32:22 -0500 [thread overview] Message-ID: <1457501543-24197-9-git-send-email-dave.long@linaro.org> (raw) In-Reply-To: <1457501543-24197-1-git-send-email-dave.long@linaro.org> From: Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com> The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu <sandeepa.s.prabhu@gmail.com> Signed-off-by: David A. Long <dave.long@linaro.org> --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/kprobes.c | 75 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c395386..72412de 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -82,6 +82,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index bd3f233..13d3333 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -534,7 +534,80 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return NULL; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + /* return 1 so that post handlers not called */ + return (void *) orig_ret_addr; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void) -- 2.5.0
next prev parent reply other threads:[~2016-03-09 5:34 UTC|newest] Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-03-09 5:32 [PATCH v11 0/9] arm64: Add kernel probes (kprobes) support David Long 2016-03-09 5:32 ` David Long 2016-03-09 5:32 ` [PATCH v11 1/9] arm64: Add HAVE_REGS_AND_STACK_ACCESS_API feature David Long 2016-03-09 5:32 ` David Long 2016-03-11 18:07 ` James Morse 2016-03-11 18:07 ` James Morse 2016-03-18 13:06 ` David Long 2016-03-18 13:06 ` David Long 2016-03-15 11:04 ` Marc Zyngier 2016-03-15 11:04 ` Marc Zyngier 2016-03-21 7:08 ` David Long 2016-03-21 7:08 ` David Long 2016-03-09 5:32 ` [PATCH v11 2/9] arm64: Add more test functions to insn.c David Long 2016-03-09 5:32 ` David Long 2016-03-09 5:32 ` [PATCH v11 3/9] arm64: add copy_to/from_user to kprobes blacklist David Long 2016-03-09 5:32 ` David Long 2016-03-15 18:47 ` James Morse 2016-03-15 18:47 ` James Morse 2016-03-16 5:43 ` Pratyush Anand 2016-03-16 5:43 ` Pratyush Anand 2016-03-16 10:27 ` James Morse 2016-03-16 10:27 ` James Morse 2016-03-17 7:57 ` Pratyush Anand 2016-03-17 7:57 ` Pratyush Anand 2016-03-18 13:29 ` Pratyush Anand 2016-03-18 13:29 ` Pratyush Anand 2016-03-18 14:02 ` James Morse 2016-03-18 14:02 ` James Morse 2016-03-18 14:43 ` Pratyush Anand 2016-03-18 14:43 ` Pratyush Anand 2016-03-18 18:12 ` James Morse 2016-03-18 18:12 ` James Morse 2016-03-21 5:17 ` Pratyush Anand 2016-03-21 5:17 ` Pratyush Anand 2016-03-21 14:52 ` Will Deacon 2016-03-21 14:52 ` Will Deacon 2016-03-22 16:51 ` Pratyush Anand 2016-03-22 16:51 ` Pratyush Anand 2016-03-17 12:04 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-17 12:04 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-09 5:32 ` [PATCH v11 4/9] arm64: add conditional instruction simulation support David Long 2016-03-09 5:32 ` David Long 2016-03-13 12:09 ` Marc Zyngier 2016-03-13 12:09 ` Marc Zyngier 2016-03-14 4:04 ` Pratyush Anand 2016-03-14 4:04 ` Pratyush Anand 2016-03-14 7:38 ` Marc Zyngier 2016-03-14 7:38 ` Marc Zyngier 2016-03-21 8:35 ` David Long 2016-03-21 8:35 ` David Long 2016-03-09 5:32 ` [PATCH v11 5/9] arm64: Kprobes with single stepping support David Long 2016-03-09 5:32 ` David Long 2016-04-20 1:29 ` Li Bin 2016-04-20 1:29 ` Li Bin 2016-03-09 5:32 ` [PATCH v11 6/9] arm64: kprobes instruction simulation support David Long 2016-03-09 5:32 ` David Long 2016-03-12 3:56 ` Marc Zyngier 2016-03-12 3:56 ` Marc Zyngier 2016-03-21 9:39 ` David Long 2016-03-21 9:39 ` David Long 2016-03-09 5:32 ` [PATCH v11 7/9] arm64: Add trampoline code for kretprobes David Long 2016-03-09 5:32 ` David Long 2016-03-13 13:52 ` Marc Zyngier 2016-03-13 13:52 ` Marc Zyngier 2016-03-21 13:30 ` David Long 2016-03-21 13:30 ` David Long 2016-03-09 5:32 ` David Long [this message] 2016-03-09 5:32 ` [PATCH v11 8/9] arm64: Add kernel return probes support (kretprobes) David Long 2016-03-17 12:22 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-17 12:22 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-17 12:58 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-17 12:58 ` 平松雅巳 / HIRAMATU,MASAMI 2016-03-21 13:33 ` David Long 2016-03-21 13:33 ` David Long 2016-03-09 5:32 ` [PATCH v11 9/9] kprobes: Add arm64 case in kprobe example module David Long 2016-03-09 5:32 ` David Long
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1457501543-24197-9-git-send-email-dave.long@linaro.org \ --to=dave.long@linaro.org \ --cc=Dave.Martin@arm.com \ --cc=Robin.Murphy@arm.com \ --cc=Vladimir.Murzin@arm.com \ --cc=akpm@linux-foundation.org \ --cc=alex.bennee@linaro.org \ --cc=ard.biesheuvel@linaro.org \ --cc=broonie@kernel.org \ --cc=bshanmugam@apm.com \ --cc=catalin.marinas@arm.com \ --cc=christoffer.dall@linaro.org \ --cc=fkan@apm.com \ --cc=gregkh@linuxfoundation.org \ --cc=james.morse@arm.com \ --cc=jens.wiklander@linaro.org \ --cc=john.blackwood@ccur.com \ --cc=keescook@chromium.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=marc.zyngier@arm.com \ --cc=mark.rutland@arm.com \ --cc=panand@redhat.com \ --cc=pmladek@suse.com \ --cc=salyzyn@android.com \ --cc=sandeepa.s.prabhu@gmail.com \ --cc=steve.capper@linaro.org \ --cc=suzuki.poulose@arm.com \ --cc=viresh.kumar@linaro.org \ --cc=wcohen@redhat.com \ --cc=will.deacon@arm.com \ --cc=yang.shi@linaro.org \ --cc=zlim.lnx@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.