Re: [PATCH v4 0/2] Make sysFS functional on topologies with per core sink

[Linu Cherian <linuc.decode@gmail.com>]

Hi Suzuki,

On Sat, Nov 7, 2020 at 11:13 AM Linu Cherian <linuc.decode@gmail.com> wrote:
>
> Hi Suzuki,
>
> On Tue, Oct 27, 2020 at 6:43 PM Linu Cherian <linuc.decode@gmail.com> wrote:
> >
> > Hi Suzuki,
> >
> >
> > On Mon, Oct 26, 2020 at 11:47 PM Suzuki K Poulose
> > <suzuki.poulose@arm.com> wrote:
> > >
> > > Hi Linu,
> > >
> > > Thanks for the feedback. My responses inline.
> > >
> > > On 10/26/20 4:33 AM, Linu Cherian wrote:
> > > > Hi Suzuki,
> > > >
> > > > On Mon, Oct 5, 2020 at 4:52 PM Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> > > >>
> > > >> Hi Linu,
> > > >>
> > > >> On 09/04/2020 03:41 AM, Linu Cherian wrote:
> > > >>> This patch series tries to fix the sysfs breakage on topologies
> > > >>> with per core sink.
> > > >>>
> > > >>> Changes since v3:
> > > >>> - References to coresight_get_enabled_sink in perf interface
> > > >>>     has been removed and marked deprecated as a new patch.
> > > >>> - To avoid changes to coresight_find_sink for ease of maintenance,
> > > >>>     search function specific to sysfs usage has been added.
> > > >>> - Sysfs being the only user for coresight_get_enabled sink,
> > > >>>     reset option is removed as well.
> > > >>
> > > >> Have you tried running perf with --per-thread option ? I believe
> > > >> this will be impacted as well, as we choose a single sink at the
> > > >> moment and this may not be reachable from the other CPUs, where
> > > >> the event may be scheduled. Eventually loosing trace for the
> > > >> duration where the task is scheduled on a different CPU.
> > > >>
> > > >> Please could you try this patch and see if helps ? I have lightly
> > > >> tested this on a fast model.
> > > >
> > > > We are seeing some issues while testing with this patch.
> > > > The issue is that, always buffer allocation for the sink happens to be on the
> > > > first core in cpu mask and this doesn't match with the core on which
> > > > event is started. Please see below for additional comments.
> > >
> > > Please could you clarify the "issues" ? How is the buffer allocation
> > > a problem ?
> >
> > 1. Just realized that the issue that we are seeing with this patch is something
> > specific to our test setup, since we had some custom patches that was required
> > for supporting the secure trace buffer configuration for our silicon.
> >
> > And to be specific, our changeset was relying on the drvdata->etr_buf at the
> > time of tmc_etr_sync_perf_buffer.
> >
> > In per core case during buffer allocation,
> > the sink chosen is always for the first core, core 0.
> > Let's consider the event started on say, core 4.
> > So w.r.t drvdata of tmc_etr4,
> > drvdata->etr_buf would get initialized while starting the event.
> > And w.r.t drvdata of tmc_etr0,
> > drvdata->etr_buf would be NULL here and our custom changeset
> > was expecting to be initialized with the etr_buf.
> >
> > So will try to rebase our patches accordingly and test this again.
> >
>
> We are facing some issues while trying out perf. This doesn't appear
> to be related to your patch though. Will share the details once we
> do some initial analysis on it.
>
> Thanks.

# ./perf record -vvv -e cs_etm// --per-thread uname -a
Using CPUID 0x00000000430f0b40
Attempting to add event pmu 'cs_etm' with '' that may result in non-fatal errors
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
maps__set_modules_path_dir: cannot open
/lib/modules/5.9.0-rc5-00116-g91c9ea890e1a dir
Problems setting modules path maps, continuing anyway...
------------------------------------------------------------
perf_event_attr:
  type                             8
  size                             120
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  enable_on_exec                   1
  sample_id_all                    1
  exclude_guest                    1
------------------------------------------------------------
sys_perf_event_open: pid 3333  cpu -1  group_fd -1  flags 0x8 = 5
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             120
  config                           0x9
  { sample_period, sample_freq }   1
  sample_type                      IP|TID|IDENTIFIER
  read_format                      ID
  disabled                         1
  exclude_kernel                   1
  exclude_hv                       1
  mmap                             1
  comm                             1
  enable_on_exec                   1
  task                             1
  sample_id_all                    1
  exclude_guest                    1
  mmap2                            1
  comm_exec                        1
  context_switch                   1
  ksymbol                          1
  bpf_event                        1
------------------------------------------------------------
sys_perf_event_open: pid 3333  cpu -1  group_fd -1  flags 0x8 = 6
mmap size 589824B
AUX area mmap length 4194304
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             120
  config                           0x9
  watermark                        1
  sample_id_all                    1
  bpf_event                        1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0x8
sys_perf_event_open failed, error -22
switching off bpf_event
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             120
  config                           0x9
  watermark                        1
  sample_id_all                    1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0x8
sys_perf_event_open failed, error -22
switching off cloexec flag
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             120
  config                           0x9
  watermark                        1
  sample_id_all                    1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0
sys_perf_event_open failed, error -22
switching off sample_id_all
------------------------------------------------------------
perf_event_attr:
  type                             1
  size                             120
  config                           0x9
  watermark                        1
  { wakeup_events, wakeup_watermark } 1
------------------------------------------------------------
sys_perf_event_open: pid -1  cpu -1  group_fd -1  flags 0
sys_perf_event_open failed, error -22
Couldn't start the BPF side band thread:
BPF programs starting from now on won't be annotatable
Synthesizing auxtrace information
cannot find cgroup mount point
Couldn't synthesize cgroup events.
Control descriptor is not initialized
Linux marvell 5.9.0-rc5-00116-g91c9ea890e1a #823 SMP PREEMPT Tue Nov
10 10:49:15 IST 2020 aarch64 aarch64 aarch64 GNU/Linux
auxtrace idx 0 old 0 head 0xdd50 diff 0xdd50
[ perf record: Woken up 1 times to write data ]
symbol:init_start file:(null) line:0 offset:0 return:0 lazy:(null)
snip ..
symbol:memory_mallopt file:(null) line:0 offset:0 return:0 lazy:(null)
failed to write feature CPUDESC
failed to write feature MEM_TOPOLOGY
failed to write feature CPU_PMU_CAPS
[ perf record: Captured and wrote 0.056 MB perf.data ]

# ./perf report
0x368 [0x50]: failed to process type: 1 [Cannot allocate memory]
Error:
failed to process sample
# To display the perf.data header info, please use --header/--header-only option

============================================================================

Appreciate your help on getting some debug hints on what is going wrong.

One strange thing noted here is sys_perf_event_open, passing cpu = -1
and pid = -1,
which doesnt appear to be valid as per tools/perf/design.txt

Thanks.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel