From: Ravi Bangoria <ravi.bangoria@linux.ibm.com> To: Paul Clarke <pc@us.ibm.com> Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, mark.rutland@arm.com, ak@linux.intel.com, maddy@linux.ibm.com, peterz@infradead.org, alexey.budankov@linux.intel.com, adrian.hunter@intel.com, acme@kernel.org, alexander.shishkin@linux.intel.com, yao.jin@linux.intel.com, mingo@redhat.com, paulus@samba.org, eranian@google.com, robert.richter@amd.com, namhyung@kernel.org, kim.phillips@amd.com, jolsa@redhat.com, kan.liang@linux.intel.com, Ravi Bangoria <ravi.bangoria@linux.ibm.com> Subject: Re: [RFC 00/11] perf: Enhancing perf to export processor hazard information Date: Thu, 5 Mar 2020 10:36:15 +0530 [thread overview] Message-ID: <b0518fdd-4993-90d4-e8b1-2330f7488d72@linux.ibm.com> (raw) In-Reply-To: <91026159-1e83-6efd-c624-464b12b18b5c@us.ibm.com> Hi Paul, Sorry for bit late reply. On 3/3/20 2:38 AM, Paul Clarke wrote: > On 3/1/20 11:23 PM, Ravi Bangoria wrote: >> Most modern microprocessors employ complex instruction execution >> pipelines such that many instructions can be 'in flight' at any >> given point in time. Various factors affect this pipeline and >> hazards are the primary among them. Different types of hazards >> exist - Data hazards, Structural hazards and Control hazards. >> Data hazard is the case where data dependencies exist between >> instructions in different stages in the pipeline. Structural >> hazard is when the same processor hardware is needed by more >> than one instruction in flight at the same time. Control hazards >> are more the branch misprediction kinds. >> >> Information about these hazards are critical towards analyzing >> performance issues and also to tune software to overcome such >> issues. Modern processors export such hazard data in Performance >> Monitoring Unit (PMU) registers. Ex, 'Sampled Instruction Event >> Register' on IBM PowerPC[1][2] and 'Instruction-Based Sampling' on >> AMD[3] provides similar information. >> >> Implementation detail: >> >> A new sample_type called PERF_SAMPLE_PIPELINE_HAZ is introduced. >> If it's set, kernel converts arch specific hazard information >> into generic format: >> >> struct perf_pipeline_haz_data { >> /* Instruction/Opcode type: Load, Store, Branch .... */ >> __u8 itype; > > At the risk of bike-shedding (in an RFC, no less), "itype" doesn't convey enough meaning to me. "inst_type"? I see in 03/11, you use "perf_inst_type". I was thinking to rename itype with operation_type or op_type. Because AMD IBS and ARM SPE observes micro ops and also op_type is more aligned to pipeline word. > >> /* Instruction Cache source */ >> __u8 icache; > > Possibly same here, and you use "perf_inst_cache" in 03/11. Sure. > >> /* Instruction suffered hazard in pipeline stage */ >> __u8 hazard_stage; >> /* Hazard reason */ >> __u8 hazard_reason; >> /* Instruction suffered stall in pipeline stage */ >> __u8 stall_stage; >> /* Stall reason */ >> __u8 stall_reason; >> __u16 pad; >> }; >> >> ... which can be read by user from mmap() ring buffer. With this >> approach, sample perf report in hazard mode looks like (On IBM >> PowerPC): >> >> # ./perf record --hazard ./ebizzy >> # ./perf report --hazard >> Overhead Symbol Shared Instruction Type Hazard Stage Hazard Reason Stall Stage Stall Reason ICache access >> 36.58% [.] thread_run ebizzy Load LSU Mispredict LSU Load fin L1 hit >> 9.46% [.] thread_run ebizzy Load LSU Mispredict LSU Dcache_miss L1 hit >> 1.76% [.] thread_run ebizzy Fixed point - - - - L1 hit >> 1.31% [.] thread_run ebizzy Load LSU ERAT Miss LSU Load fin L1 hit >> 1.27% [.] thread_run ebizzy Load LSU Mispredict - - L1 hit >> 1.16% [.] thread_run ebizzy Fixed point - - FXU Fixed cycle L1 hit >> 0.50% [.] thread_run ebizzy Fixed point ISU Source Unavailable FXU Fixed cycle L1 hit >> 0.30% [.] thread_run ebizzy Load LSU LMQ Full, DERAT Miss LSU Load fin L1 hit >> 0.24% [.] thread_run ebizzy Load LSU ERAT Miss - - L1 hit >> 0.08% [.] thread_run ebizzy - - - BRU Fixed cycle L1 hit >> 0.05% [.] thread_run ebizzy Branch - - BRU Fixed cycle L1 hit >> 0.04% [.] thread_run ebizzy Fixed point ISU Source Unavailable - - L1 hit > > How are these to be interpreted? This is great information, but is it possible to make it more readable for non-experts? For the RFC proposal we just pulled the details from the spec. But yes, will look into this. > If each of these map 1:1 with hardware events, should you emit the name of the event here, so that can be used to look up further information? For example, does the first line map to PM_CMPLU_STALL_LSU_FIN? I'm using PM_MRK_INST_CMPL event in perf record an SIER provides all these information. > What was "Mispredict[ed]"? (Is it different from a branch misprediction?) And how does this relate to "L1 hit"? I'm not 100% sure. I'll check with the hw folks about it. > Can we emit "Load finish" instead of "Load fin" for easier reading? 03/11 also has "Marked fin before NTC". > Nit: why does "Dcache_miss" have an underscore and none of the others? Sure. Will change it. > >> Also perf annotate with hazard data: > >> │ static int >> │ compare(const void *p1, const void *p2) >> │ { >> 33.23 │ std r31,-8(r1) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: Load Hit Store, stall_stage: LSU, stall_reason: -, icache: L3 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: -, stall_reason: -, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> 0.84 │ stdu r1,-64(r1) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: -, stall_reason: -, icache: L1 hit} >> 0.24 │ mr r31,r1 >> │ {haz_stage: -, haz_reason: -, stall_stage: -, stall_reason: -, icache: L1 hit} >> 21.18 │ std r3,32(r31) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> >> >> Patches: >> - Patch #1 is a simple cleanup patch >> - Patch #2, #3, #4 implements generic and arch specific kernel >> infrastructure >> - Patch #5 enables perf record and script with hazard mode >> - Patch #6, #7, #8 enables perf report with hazard mode >> - Patch #9, #10, #11 enables perf annotate with hazard mode >> >> Note: >> - This series is based on the talk by Madhavan in LPC 2018[4]. This is >> just an early RFC to get comments about the approach and not intended >> to be merged yet. >> - I've prepared the series base on v5.6-rc3. But it depends on generic >> perf annotate fixes [5][6] which are already merged by Arnaldo in >> perf/urgent and perf/core. >> >> [1]: Book III, Section 9.4.10: >> https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0 >> [2]: https://wiki.raptorcs.com/w/images/6/6b/POWER9_PMU_UG_v12_28NOV2018_pub.pdf#G9.1106986 > > This document is also available from the "IBM Portal for OpenPOWER" under the "All IBM Material for OpenPOWER" https://www-355.ibm.com/systems/power/openpower/tgcmDocumentRepository.xhtml?aliasId=OpenPOWER, under each of the individual modules. (Well hidden, it might be said, and not a simple link like you have here.) Thanks for pointing it :) Ravi
WARNING: multiple messages have this Message-ID (diff)
From: Ravi Bangoria <ravi.bangoria@linux.ibm.com> To: Paul Clarke <pc@us.ibm.com> Cc: mark.rutland@arm.com, Ravi Bangoria <ravi.bangoria@linux.ibm.com>, ak@linux.intel.com, peterz@infradead.org, jolsa@redhat.com, alexey.budankov@linux.intel.com, linux-kernel@vger.kernel.org, acme@kernel.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, yao.jin@linux.intel.com, maddy@linux.ibm.com, paulus@samba.org, eranian@google.com, robert.richter@amd.com, namhyung@kernel.org, kim.phillips@amd.com, linuxppc-dev@lists.ozlabs.org, mingo@redhat.com, kan.liang@linux.intel.com Subject: Re: [RFC 00/11] perf: Enhancing perf to export processor hazard information Date: Thu, 5 Mar 2020 10:36:15 +0530 [thread overview] Message-ID: <b0518fdd-4993-90d4-e8b1-2330f7488d72@linux.ibm.com> (raw) In-Reply-To: <91026159-1e83-6efd-c624-464b12b18b5c@us.ibm.com> Hi Paul, Sorry for bit late reply. On 3/3/20 2:38 AM, Paul Clarke wrote: > On 3/1/20 11:23 PM, Ravi Bangoria wrote: >> Most modern microprocessors employ complex instruction execution >> pipelines such that many instructions can be 'in flight' at any >> given point in time. Various factors affect this pipeline and >> hazards are the primary among them. Different types of hazards >> exist - Data hazards, Structural hazards and Control hazards. >> Data hazard is the case where data dependencies exist between >> instructions in different stages in the pipeline. Structural >> hazard is when the same processor hardware is needed by more >> than one instruction in flight at the same time. Control hazards >> are more the branch misprediction kinds. >> >> Information about these hazards are critical towards analyzing >> performance issues and also to tune software to overcome such >> issues. Modern processors export such hazard data in Performance >> Monitoring Unit (PMU) registers. Ex, 'Sampled Instruction Event >> Register' on IBM PowerPC[1][2] and 'Instruction-Based Sampling' on >> AMD[3] provides similar information. >> >> Implementation detail: >> >> A new sample_type called PERF_SAMPLE_PIPELINE_HAZ is introduced. >> If it's set, kernel converts arch specific hazard information >> into generic format: >> >> struct perf_pipeline_haz_data { >> /* Instruction/Opcode type: Load, Store, Branch .... */ >> __u8 itype; > > At the risk of bike-shedding (in an RFC, no less), "itype" doesn't convey enough meaning to me. "inst_type"? I see in 03/11, you use "perf_inst_type". I was thinking to rename itype with operation_type or op_type. Because AMD IBS and ARM SPE observes micro ops and also op_type is more aligned to pipeline word. > >> /* Instruction Cache source */ >> __u8 icache; > > Possibly same here, and you use "perf_inst_cache" in 03/11. Sure. > >> /* Instruction suffered hazard in pipeline stage */ >> __u8 hazard_stage; >> /* Hazard reason */ >> __u8 hazard_reason; >> /* Instruction suffered stall in pipeline stage */ >> __u8 stall_stage; >> /* Stall reason */ >> __u8 stall_reason; >> __u16 pad; >> }; >> >> ... which can be read by user from mmap() ring buffer. With this >> approach, sample perf report in hazard mode looks like (On IBM >> PowerPC): >> >> # ./perf record --hazard ./ebizzy >> # ./perf report --hazard >> Overhead Symbol Shared Instruction Type Hazard Stage Hazard Reason Stall Stage Stall Reason ICache access >> 36.58% [.] thread_run ebizzy Load LSU Mispredict LSU Load fin L1 hit >> 9.46% [.] thread_run ebizzy Load LSU Mispredict LSU Dcache_miss L1 hit >> 1.76% [.] thread_run ebizzy Fixed point - - - - L1 hit >> 1.31% [.] thread_run ebizzy Load LSU ERAT Miss LSU Load fin L1 hit >> 1.27% [.] thread_run ebizzy Load LSU Mispredict - - L1 hit >> 1.16% [.] thread_run ebizzy Fixed point - - FXU Fixed cycle L1 hit >> 0.50% [.] thread_run ebizzy Fixed point ISU Source Unavailable FXU Fixed cycle L1 hit >> 0.30% [.] thread_run ebizzy Load LSU LMQ Full, DERAT Miss LSU Load fin L1 hit >> 0.24% [.] thread_run ebizzy Load LSU ERAT Miss - - L1 hit >> 0.08% [.] thread_run ebizzy - - - BRU Fixed cycle L1 hit >> 0.05% [.] thread_run ebizzy Branch - - BRU Fixed cycle L1 hit >> 0.04% [.] thread_run ebizzy Fixed point ISU Source Unavailable - - L1 hit > > How are these to be interpreted? This is great information, but is it possible to make it more readable for non-experts? For the RFC proposal we just pulled the details from the spec. But yes, will look into this. > If each of these map 1:1 with hardware events, should you emit the name of the event here, so that can be used to look up further information? For example, does the first line map to PM_CMPLU_STALL_LSU_FIN? I'm using PM_MRK_INST_CMPL event in perf record an SIER provides all these information. > What was "Mispredict[ed]"? (Is it different from a branch misprediction?) And how does this relate to "L1 hit"? I'm not 100% sure. I'll check with the hw folks about it. > Can we emit "Load finish" instead of "Load fin" for easier reading? 03/11 also has "Marked fin before NTC". > Nit: why does "Dcache_miss" have an underscore and none of the others? Sure. Will change it. > >> Also perf annotate with hazard data: > >> │ static int >> │ compare(const void *p1, const void *p2) >> │ { >> 33.23 │ std r31,-8(r1) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: Load Hit Store, stall_stage: LSU, stall_reason: -, icache: L3 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: -, stall_reason: -, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> 0.84 │ stdu r1,-64(r1) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: -, stall_reason: -, icache: L1 hit} >> 0.24 │ mr r31,r1 >> │ {haz_stage: -, haz_reason: -, stall_stage: -, stall_reason: -, icache: L1 hit} >> 21.18 │ std r3,32(r31) >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> │ {haz_stage: LSU, haz_reason: ERAT Miss, stall_stage: LSU, stall_reason: Store, icache: L1 hit} >> >> >> Patches: >> - Patch #1 is a simple cleanup patch >> - Patch #2, #3, #4 implements generic and arch specific kernel >> infrastructure >> - Patch #5 enables perf record and script with hazard mode >> - Patch #6, #7, #8 enables perf report with hazard mode >> - Patch #9, #10, #11 enables perf annotate with hazard mode >> >> Note: >> - This series is based on the talk by Madhavan in LPC 2018[4]. This is >> just an early RFC to get comments about the approach and not intended >> to be merged yet. >> - I've prepared the series base on v5.6-rc3. But it depends on generic >> perf annotate fixes [5][6] which are already merged by Arnaldo in >> perf/urgent and perf/core. >> >> [1]: Book III, Section 9.4.10: >> https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0 >> [2]: https://wiki.raptorcs.com/w/images/6/6b/POWER9_PMU_UG_v12_28NOV2018_pub.pdf#G9.1106986 > > This document is also available from the "IBM Portal for OpenPOWER" under the "All IBM Material for OpenPOWER" https://www-355.ibm.com/systems/power/openpower/tgcmDocumentRepository.xhtml?aliasId=OpenPOWER, under each of the individual modules. (Well hidden, it might be said, and not a simple link like you have here.) Thanks for pointing it :) Ravi
next prev parent reply other threads:[~2020-03-05 5:06 UTC|newest] Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-02 5:23 [RFC 00/11] perf: Enhancing perf to export processor hazard information Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 01/11] powerpc/perf: Simplify ISA207_SIER macros Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 02/11] perf/core: Data structure to present hazard data Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 9:55 ` Peter Zijlstra 2020-03-02 9:55 ` Peter Zijlstra 2020-03-02 14:23 ` maddy 2020-03-02 14:23 ` maddy 2020-03-02 14:48 ` Mark Rutland 2020-03-02 14:48 ` Mark Rutland 2020-03-03 14:32 ` Ravi Bangoria 2020-03-03 14:32 ` Ravi Bangoria 2020-03-02 14:54 ` Mark Rutland 2020-03-02 14:54 ` Mark Rutland 2020-03-03 14:31 ` Ravi Bangoria 2020-03-03 14:31 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 03/11] powerpc/perf: Arch specific definitions for pipeline Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 04/11] powerpc/perf: Arch support to expose Hazard data Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 05/11] perf tools: Enable record and script to record and show hazard data Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 06/11] perf hists: Make a room for hazard info in struct hist_entry Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 07/11] perf hazard: Functions to convert generic hazard data to arch specific string Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 08/11] perf report: Enable hazard mode Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 09/11] perf annotate: Introduce type for annotation_line Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 10/11] perf annotate: Preparation for hazard Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 5:23 ` [RFC 11/11] perf annotate: Show hazard data in tui mode Ravi Bangoria 2020-03-02 5:23 ` Ravi Bangoria 2020-03-02 10:13 ` [RFC 00/11] perf: Enhancing perf to export processor hazard information Peter Zijlstra 2020-03-02 10:13 ` Peter Zijlstra 2020-03-02 20:21 ` Stephane Eranian 2020-03-02 20:21 ` Stephane Eranian 2020-03-02 22:25 ` Kim Phillips 2020-03-02 22:25 ` Kim Phillips 2020-03-05 4:46 ` Ravi Bangoria 2020-03-05 4:46 ` Ravi Bangoria 2020-03-05 22:06 ` Kim Phillips 2020-03-05 22:06 ` Kim Phillips 2020-03-11 16:00 ` Ravi Bangoria 2020-03-12 22:38 ` Kim Phillips 2020-03-12 22:38 ` Kim Phillips 2020-03-17 6:50 ` maddy 2020-03-17 6:50 ` maddy 2020-03-18 17:35 ` Kim Phillips 2020-03-18 17:35 ` Kim Phillips 2020-03-19 11:22 ` Michael Ellerman 2020-03-19 11:22 ` Michael Ellerman 2020-03-26 10:19 ` maddy 2020-03-26 10:19 ` maddy 2020-03-26 19:48 ` Kim Phillips 2020-03-26 19:48 ` Kim Phillips 2020-04-20 7:09 ` Madhavan Srinivasan 2020-04-20 7:09 ` Madhavan Srinivasan 2020-04-27 7:18 ` Madhavan Srinivasan 2020-04-27 7:18 ` Madhavan Srinivasan 2020-03-05 4:28 ` maddy 2020-03-05 4:28 ` maddy 2020-03-03 1:33 ` Andi Kleen 2020-03-03 1:33 ` Andi Kleen 2020-03-05 5:06 ` Ravi Bangoria 2020-03-05 5:06 ` Ravi Bangoria 2020-03-02 21:08 ` Paul Clarke 2020-03-02 21:08 ` Paul Clarke 2020-03-05 5:06 ` Ravi Bangoria [this message] 2020-03-05 5:06 ` Ravi Bangoria
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=b0518fdd-4993-90d4-e8b1-2330f7488d72@linux.ibm.com \ --to=ravi.bangoria@linux.ibm.com \ --cc=acme@kernel.org \ --cc=adrian.hunter@intel.com \ --cc=ak@linux.intel.com \ --cc=alexander.shishkin@linux.intel.com \ --cc=alexey.budankov@linux.intel.com \ --cc=eranian@google.com \ --cc=jolsa@redhat.com \ --cc=kan.liang@linux.intel.com \ --cc=kim.phillips@amd.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=maddy@linux.ibm.com \ --cc=mark.rutland@arm.com \ --cc=mingo@redhat.com \ --cc=namhyung@kernel.org \ --cc=paulus@samba.org \ --cc=pc@us.ibm.com \ --cc=peterz@infradead.org \ --cc=robert.richter@amd.com \ --cc=yao.jin@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.