Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

[Tanmay Jagdale]

Hi Will and Robin,

> On 2022-06-24 13:14, Will Deacon wrote:
> > On Sat, May 28, 2022 at 12:26:47AM +0530, Tanmay Jagdale wrote:
> >> The TAD PMU supports following counters that can be filtered by MPAM
> >> partition id.
> >>      - (0x1a) tad_alloc_dtg : Allocations to DTG.
> >>      - (0x1b) tad_alloc_ltg : Allocations to LTG.
> >>      - (0x1c) tad_alloc_any : Total allocations to DTG/LTG.
> >>      - (0x1d) tad_hit_dtg   : DTG hits.
> >>      - (0x1e) tad_hit_ltg   : LTG hits.
> >>      - (0x1f) tad_hit_any   : Hit in LTG/DTG.
> >>      - (0x20) tad_tag_rd    : Total tag reads.
> >>
> >> Add a new 'partid' attribute of 16-bits to get the partition id
> >> passed from perf tool. This value would be stored in config1 field
> >> of perf_event_attr structure.
> >>
> >> Example:
> >> perf stat -e tad/tad_alloc_any,partid=0x12/ <program>
> >>
> >> - Drop read of TAD_PRF since we don't have to preserve any
> >>    bit fields and always write an updated value.
> >> - Update register offsets of TAD_PRF and TAD_PFC.
> >
> > It would be great if you could document some of this under
> > Documentation/admin-guide/perf like many of the other PMU drivers have
> > done.
> 
> Especially documenting how the user obtains the required partid value to
> pass.
> 
We created MPAM partitions using the resctrl filesystem interface.
Example:
        $ cd /sys/fs/resctrl
        $ mkdir p1
        $ echo "L3:0=f" > p1/schemata (configure 4 L3 cache ways)
        $ mkdir p2
        $ echo "L3:1=ff0" > p2/schemata (configure 8 L3 cache ways)

Here directory name 'p1' creates a MPAM partid 0x1 and 'p2' creates
0x2 and so on.

Right now, there is no file which exposes the partid to userspace.
We must rely on the sequential order in which we create partitions
via resctrl and use that to derive the partid.

I'll send out a V2 documenting this.
> Thanks,
> Robin.
> 
> >> Signed-off-by: Tanmay Jagdale <tanmay@marvell.com>
> >> ---
> >>   drivers/perf/marvell_cn10k_tad_pmu.c | 23 ++++++++++++++++++-----
> >>   1 file changed, 18 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/drivers/perf/marvell_cn10k_tad_pmu.c
> b/drivers/perf/marvell_cn10k_tad_pmu.c
> >> index 282d3a071a67..f552e6bffcac 100644
> >> --- a/drivers/perf/marvell_cn10k_tad_pmu.c
> >> +++ b/drivers/perf/marvell_cn10k_tad_pmu.c
> >> @@ -18,10 +18,12 @@
> >>   #include <linux/perf_event.h>
> >>   #include <linux/platform_device.h>
> >>
> >> -#define TAD_PFC_OFFSET		0x0
> >> +#define TAD_PFC_OFFSET		0x800
> >>   #define TAD_PFC(counter)	(TAD_PFC_OFFSET | (counter << 3))
> >> -#define TAD_PRF_OFFSET		0x100
> >> +#define TAD_PRF_OFFSET		0x900
> >>   #define TAD_PRF(counter)	(TAD_PRF_OFFSET | (counter << 3))
> >> +#define TAD_PRF_MATCH_PARTID	(1 << 8)
> >> +#define TAD_PRF_PARTID_NS	(1 << 10)
> >>   #define TAD_PRF_CNTSEL_MASK	0xFF
> >>   #define TAD_MAX_COUNTERS	8
> >>
> >> @@ -86,23 +88,32 @@ static void tad_pmu_event_counter_start(struct
> perf_event *event, int flags)
> >>   	struct hw_perf_event *hwc = &event->hw;
> >>   	u32 event_idx = event->attr.config;
> >>   	u32 counter_idx = hwc->idx;
> >> +	u32 partid_filter = 0;
> >>   	u64 reg_val;
> >> +	u32 partid;
> >>   	int i;
> >>
> >>   	hwc->state = 0;
> >>
> >> +	/* Extract the partid (if any) passed by user */
> >> +	partid = event->attr.config1 & 0x3f;
> >
> > [...]
> >
> >>   PMU_FORMAT_ATTR(event, "config:0-7");
> >> +PMU_FORMAT_ATTR(partid, "config1:0-15");
> >
> > This doesn't seem to match the mask used above?
ACK.
I will send out a V2 that includes this fix.

Thanks,
Tanmay
> >
> > Will
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org

Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

[James Morse]

Hi Tanmay,

On 27/06/2022 14:18, Tanmay Jagdale wrote:
>> On 2022-06-24 13:14, Will Deacon wrote:
>>> On Sat, May 28, 2022 at 12:26:47AM +0530, Tanmay Jagdale wrote:
>>>> The TAD PMU supports following counters that can be filtered by MPAM
>>>> partition id.
>>>>      - (0x1a) tad_alloc_dtg : Allocations to DTG.
>>>>      - (0x1b) tad_alloc_ltg : Allocations to LTG.
>>>>      - (0x1c) tad_alloc_any : Total allocations to DTG/LTG.
>>>>      - (0x1d) tad_hit_dtg   : DTG hits.
>>>>      - (0x1e) tad_hit_ltg   : LTG hits.
>>>>      - (0x1f) tad_hit_any   : Hit in LTG/DTG.
>>>>      - (0x20) tad_tag_rd    : Total tag reads.
>>>>
>>>> Add a new 'partid' attribute of 16-bits to get the partition id
>>>> passed from perf tool. This value would be stored in config1 field
>>>> of perf_event_attr structure.
>>>>
>>>> Example:
>>>> perf stat -e tad/tad_alloc_any,partid=0x12/ <program>
>>>>
>>>> - Drop read of TAD_PRF since we don't have to preserve any
>>>>    bit fields and always write an updated value.
>>>> - Update register offsets of TAD_PRF and TAD_PFC.
>>>
>>> It would be great if you could document some of this under
>>> Documentation/admin-guide/perf like many of the other PMU drivers have
>>> done.
>>
>> Especially documenting how the user obtains the required partid value to
>> pass.

> We created MPAM partitions using the resctrl filesystem interface.
> Example:
>         $ cd /sys/fs/resctrl
>         $ mkdir p1
>         $ echo "L3:0=f" > p1/schemata (configure 4 L3 cache ways)
>         $ mkdir p2
>         $ echo "L3:1=ff0" > p2/schemata (configure 8 L3 cache ways)
> 
> Here directory name 'p1' creates a MPAM partid 0x1 and 'p2' creates
> 0x2 and so on.

You can't rely on this.

See the KNOWN_ISSUES file in the the mpam tree: PARTID 0 should be reserved for unknown
hardware. In fact any number of PARTID may be reserved for in-kernel users. You can't
guess what the offset might be from user-space.


> Right now, there is no file which exposes the partid to userspace.
> We must rely on the sequential order in which we create partitions
> via resctrl and use that to derive the partid.

If you dig in the MPAM tree you'll find how I intend to solve this for exposing the MPAM
counters via perf. But this is a user-space visible change to resctrl, so it will need to
wait until all the refactoring is done and the bulk of the MPAM driver is upstream.


Thanks,

James

RE: [EXT] Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

[Amit Singh Tomar]

Hi James,

-----Original Message-----
From: James Morse <james.morse@arm.com> 
Sent: Friday, July 1, 2022 10:50 PM
To: Tanmay Jagdale <tanmay@marvell.com>; Robin Murphy <robin.murphy@arm.com>; Will Deacon <will@kernel.org>
Cc: mark.rutland@arm.com; robh@kernel.org; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham <sgoutham@marvell.com>; Linu Cherian <lcherian@marvell.com>; Bharat Bhushan <bbhushan2@marvell.com>; Amit Singh Tomar <amitsinght@marvell.com>
Subject: [EXT] Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

External Email

----------------------------------------------------------------------
Hi Tanmay,

On 27/06/2022 14:18, Tanmay Jagdale wrote:
>> On 2022-06-24 13:14, Will Deacon wrote:
>>> On Sat, May 28, 2022 at 12:26:47AM +0530, Tanmay Jagdale wrote:
>>>> The TAD PMU supports following counters that can be filtered by 
>>>> MPAM partition id.
>>>>      - (0x1a) tad_alloc_dtg : Allocations to DTG.
>>>>      - (0x1b) tad_alloc_ltg : Allocations to LTG.
>>>>      - (0x1c) tad_alloc_any : Total allocations to DTG/LTG.
>>>>      - (0x1d) tad_hit_dtg   : DTG hits.
>>>>      - (0x1e) tad_hit_ltg   : LTG hits.
>>>>      - (0x1f) tad_hit_any   : Hit in LTG/DTG.
>>>>      - (0x20) tad_tag_rd    : Total tag reads.
>>>>
>>>> Add a new 'partid' attribute of 16-bits to get the partition id 
>>>> passed from perf tool. This value would be stored in config1 field 
>>>> of perf_event_attr structure.
>>>>
>>>> Example:
>>>> perf stat -e tad/tad_alloc_any,partid=0x12/ <program>
>>>>
>>>> - Drop read of TAD_PRF since we don't have to preserve any
>>>>    bit fields and always write an updated value.
>>>> - Update register offsets of TAD_PRF and TAD_PFC.
>>>
>>> It would be great if you could document some of this under 
>>> Documentation/admin-guide/perf like many of the other PMU drivers 
>>> have done.
>>
>> Especially documenting how the user obtains the required partid value 
>> to pass.

> We created MPAM partitions using the resctrl filesystem interface.
> Example:
>         $ cd /sys/fs/resctrl
>         $ mkdir p1
>         $ echo "L3:0=f" > p1/schemata (configure 4 L3 cache ways)
>         $ mkdir p2
>         $ echo "L3:1=ff0" > p2/schemata (configure 8 L3 cache ways)
> 
> Here directory name 'p1' creates a MPAM partid 0x1 and 'p2' creates
> 0x2 and so on.

You can't rely on this.

See the KNOWN_ISSUES file in the the mpam tree: PARTID 0 should be reserved for unknown hardware. In fact any number of PARTID may be reserved for in-kernel users. You can't guess what the offset might be from user-space.


> Right now, there is no file which exposes the partid to userspace.
> We must rely on the sequential order in which we create partitions via 
> resctrl and use that to derive the partid.

If you dig in the MPAM tree you'll find how I intend to solve this for exposing the MPAM counters via perf. But this is a user-space visible change to resctrl, so it will need to wait until all the refactoring is done and the bulk of the MPAM driver is upstream.

[>>] But these are non-standard (HAS_MSMON=0) monitoring counters that just happen to use MPAM partID. Therefore, IMHO we 
       should expose them via Marvell specific PMU driver, and exposing the MPAM partID to user space (may be under resctrl)
       will help this cause.
      or
      resctrl based PMU driver can still support such counters that are not comply with MPAM standard?


Thanks,
 -Amit