From: Kan Liang <kan.liang@intel.com>
To: a.p.zijlstra@chello.nl, eranian@google.com
Cc: linux-kernel@vger.kernel.org, mingo@redhat.com, paulus@samba.org,
acme@kernel.org, jolsa@redhat.com, ak@linux.intel.com,
Kan Liang <kan.liang@intel.com>,
"Yan, Zheng" <zheng.z.yan@intel.com>
Subject: [PATCH V6 09/17] perf, x86: Save/resotre LBR stack during context switch
Date: Sun, 19 Oct 2014 17:55:04 -0400 [thread overview]
Message-ID: <1413755712-8259-10-git-send-email-kan.liang@intel.com> (raw)
In-Reply-To: <1413755712-8259-1-git-send-email-kan.liang@intel.com>
When the LBR call stack is enabled, it is necessary to save/restore
the LBR stack on context switch. The solution is saving/restoring
the LBR stack to/from task's perf event context.
The LBR stack is saved/restored only when there are events that use
the LBR call stack. If no event uses LBR call stack, the LBR stack
is reset when task is scheduled in.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Kan Liang <kan.liang@intel.com>
---
arch/x86/kernel/cpu/perf_event_intel_lbr.c | 89 ++++++++++++++++++++++++++----
1 file changed, 77 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/cpu/perf_event_intel_lbr.c b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
index f9b9f2b..1908875 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
@@ -180,13 +180,90 @@ void intel_pmu_lbr_reset(void)
intel_pmu_lbr_reset_64();
}
+/*
+ * TOS = most recently recorded branch
+ */
+static inline u64 intel_pmu_lbr_tos(void)
+{
+ u64 tos;
+
+ rdmsrl(x86_pmu.lbr_tos, tos);
+ return tos;
+}
+
+enum {
+ LBR_NONE,
+ LBR_VALID,
+};
+
+static void __intel_pmu_lbr_restore(struct x86_perf_task_context *task_ctx)
+{
+ int i;
+ unsigned lbr_idx, mask;
+ u64 tos;
+
+ if (task_ctx->lbr_callstack_users == 0 ||
+ task_ctx->lbr_stack_state == LBR_NONE) {
+ intel_pmu_lbr_reset();
+ return;
+ }
+
+ mask = x86_pmu.lbr_nr - 1;
+ tos = intel_pmu_lbr_tos();
+ for (i = 0; i < x86_pmu.lbr_nr; i++) {
+ lbr_idx = (tos - i) & mask;
+ wrmsrl(x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);
+ wrmsrl(x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]);
+ }
+ task_ctx->lbr_stack_state = LBR_NONE;
+}
+
+static void __intel_pmu_lbr_save(struct x86_perf_task_context *task_ctx)
+{
+ int i;
+ unsigned lbr_idx, mask;
+ u64 tos;
+
+ if (task_ctx->lbr_callstack_users == 0) {
+ task_ctx->lbr_stack_state = LBR_NONE;
+ return;
+ }
+
+ mask = x86_pmu.lbr_nr - 1;
+ tos = intel_pmu_lbr_tos();
+ for (i = 0; i < x86_pmu.lbr_nr; i++) {
+ lbr_idx = (tos - i) & mask;
+ rdmsrl(x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);
+ rdmsrl(x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]);
+ }
+ task_ctx->lbr_stack_state = LBR_VALID;
+}
+
+
void intel_pmu_lbr_sched_task(struct perf_event_context *ctx, bool sched_in)
{
struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events);
+ struct x86_perf_task_context *task_ctx;
if (!x86_pmu.lbr_nr)
return;
/*
+ * If LBR callstack feature is enabled and the stack was saved when
+ * the task was scheduled out, restore the stack. Otherwise flush
+ * the LBR stack.
+ */
+ task_ctx = ctx ? ctx->task_ctx_data : NULL;
+ if (task_ctx) {
+ if (sched_in) {
+ __intel_pmu_lbr_restore(task_ctx);
+ cpuc->lbr_context = ctx;
+ } else {
+ __intel_pmu_lbr_save(task_ctx);
+ }
+ return;
+ }
+
+ /*
* When sampling the branck stack in system-wide, it may be
* necessary to flush the stack on context switch. This happens
* when the branch stack does not tag its entries with the pid
@@ -277,18 +354,6 @@ void intel_pmu_lbr_disable_all(void)
__intel_pmu_lbr_disable();
}
-/*
- * TOS = most recently recorded branch
- */
-static inline u64 intel_pmu_lbr_tos(void)
-{
- u64 tos;
-
- rdmsrl(x86_pmu.lbr_tos, tos);
-
- return tos;
-}
-
static void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)
{
unsigned long mask = x86_pmu.lbr_nr - 1;
--
1.8.3.2
next prev parent reply other threads:[~2014-10-19 22:07 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-10-19 21:54 [PATCH V6 00/17] perf, x86: Haswell LBR call stack support Kan Liang
2014-10-19 21:54 ` [PATCH V6 01/17] perf, x86: Reduce lbr_sel_map size Kan Liang
2014-10-24 9:39 ` Peter Zijlstra
2014-11-04 1:07 ` Liang, Kan
2014-11-04 7:14 ` Peter Zijlstra
2014-11-04 7:16 ` Peter Zijlstra
2014-10-19 21:54 ` [PATCH V6 02/17] perf, core: introduce pmu context switch callback Kan Liang
2014-10-19 21:54 ` [PATCH V6 03/17] perf, x86: use context switch callback to flush LBR stack Kan Liang
2014-10-19 21:54 ` [PATCH V6 04/17] perf, x86: Basic Haswell LBR call stack support Kan Liang
2014-10-19 21:55 ` [PATCH V6 05/17] perf, core: pmu specific data for perf task context Kan Liang
2014-10-19 21:55 ` [PATCH V6 06/17] perf, core: always switch pmu specific data during context switch Kan Liang
2014-10-19 21:55 ` [PATCH V6 07/17] perf, x86: allocate space for storing LBR stack Kan Liang
2014-10-19 21:55 ` [PATCH V6 08/17] perf, x86: track number of events that use LBR callstack Kan Liang
2014-10-19 21:55 ` Kan Liang [this message]
2014-10-19 21:55 ` [PATCH V6 10/17] perf, core: simplify need branch stack check Kan Liang
2014-10-19 21:55 ` [PATCH V6 11/17] perf, core: expose LBR call stack to user perf tool Kan Liang
2014-10-19 21:55 ` [PATCH V6 12/17] perf, x86: re-organize code that implicitly enables LBR/PEBS Kan Liang
2014-10-19 21:55 ` [PATCH V6 13/17] perf, x86: enable LBR callstack when recording callchain Kan Liang
2014-10-24 13:39 ` Jiri Olsa
2014-10-24 14:49 ` Liang, Kan
2014-10-19 21:55 ` [PATCH V6 14/17] perf, x86: disable FREEZE_LBRS_ON_PMI when LBR operates in callstack mode Kan Liang
2014-10-19 21:55 ` [PATCH V6 15/17] perf, x86: Discard zero length call entries in LBR call stack Kan Liang
2014-10-19 21:55 ` [PATCH V6 16/17] perf tools: handle LBR call stack data Kan Liang
2014-10-19 21:55 ` [PATCH V6 17/17] perf tools: choose to dump callchain from LBR and FP Kan Liang
2014-10-24 13:36 ` Jiri Olsa
2014-10-24 13:55 ` Jiri Olsa
2014-10-24 15:20 ` Liang, Kan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1413755712-8259-10-git-send-email-kan.liang@intel.com \
--to=kan.liang@intel.com \
--cc=a.p.zijlstra@chello.nl \
--cc=acme@kernel.org \
--cc=ak@linux.intel.com \
--cc=eranian@google.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=paulus@samba.org \
--cc=zheng.z.yan@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).