On Sonntag, 14. März 2021 18:29:09 CET Ian Rogers wrote: > Hi Milian, > > Thanks for the email and hotspot is a very interesting project! > > On Sat, Mar 13, 2021 at 8:25 AM Milian Wolff wrote: > > On Mittwoch, 10. März 2021 17:32:25 CET Ian Rogers wrote: > > > Hi, > > > > > > I'm happy to share that the Linux Foundation has been accepted into > > > the Google Summer-of-Code 2021 and we are looking for students > > > interested in profiling, perf, etc. to apply to do projects. > > > > > > The main details on the Linux foundation application are here: > > > https://wiki.linuxfoundation.org/gsoc/google-summer-code-2021 > > > > > > Details on potential projects with perf here: > > > https://wiki.linuxfoundation.org/gsoc/2021-gsoc-perf > > > > > > Please help to share, especially to students. > > > > > > Additional project ideas, mentors and help is also appreciated. > > > > As the maintainer of hotspot [1], I'd be happy to discuss how we could > > reuse the prior art in providing a GUI around `perf report` and more. > > Considering that hotspot is already a pretty successfull project, I > > wonder why we have to reinvent the wheel to provide a gtk-based GUI - is > > it really just the framework choice - Qt vs. GTK? Or are there other > > reasons? > > There are a few issues: > - there's an observation that the gtk support in perf was bitrotting. > Having a student project in this area would hopefully improve this > situation. Agreed. > - commands like perf report even in text mode are slow, single > threaded, etc. Again, an area where a student project can help out. Again, agreed. That is actually something that I've spent a considerable amount of time on. So far, I mostly concentrated on better caching - something which has had very significant effects on the performance of hotspot for some perf.data workloads. See e.g.: https://www.kdab.com/hotspot-version-1-3-0-released/ Threading wise, in hotspot, we have basically two threads - one in the background process perfparser which does unwinding and symbolication. Then another one in the hotspot GUI process, which does the data aggregation for the various views. So far, I shied away from trying to further parallelize the former part, as unwinding and symbolication are both heavily dependend on the order in which events are processed. I.e. we have to make sure to process mmap events and the like before we process a sampling event. Additionally, making caching efficient is much easier in a singlethreaded context, which was so far much more important to get right. > - wrt UI there is also pprof (https://github.com/google/pprof) which > I think a web UI is interesting beyond just Qt, GTK, etc. Having the > code be within the perf tool means that testing can be automated, it > is easy for shared contributions, the kernel and user APIs aren't > separated by projects.. Yes, nowadays there are tons of pretty good UI options out there - esp. in the web context. I have not used pprof, but heard a lot of praise for the Mozilla profiler too: https://profiler.firefox.com/ And then there's catapult which is integrated into chromium or Perfetto which is afaik used for Android? The list probably goes on and on ;-) But in the end it would be extremely useful if perf could directly create a pre-processed, compressed and portable result file with some command. That could in turn use some easy-to-use binary format (CBOR, protobof, whatever) which could then be parsed by the GUI instead of it having to reinvent the wheel there. > > Generally, I would love to see a proper library provided by perf that > > could be used to do the `perf report` and/or `perf annotate` data > > processing. > > There is a library interface developing: > https://github.com/torvalds/linux/tree/master/tools/lib/perf Ah, that's very interesting - that didn't exist back when we started perfparser/hotspot. > But it is mainly focussed on say 'perf record' than 'perf report'. > Perhaps you could mentor a project to refactor the report APIs, which > could be integrated with hotspot? If so, we can update the GSoC wiki. While I'm honored by the invitation, I'm afraid that I don't have the capacity to fulfil this role. I've attended GSOC myself in the past, and even mentored a couple of times, so I know that doing a good job there as a mentor requires more time than I can spare currently :( That said, I am very willing to be available for discussion as an interested party. > It'd be nice to refactor things like event parsing into a library, in > part to make unit and fuzz testing easier. The event sorting is a big > source of perf's slowness and something that should be reasonable to > parallelize, an issue here is a lack of a thread pool and other > abstractions. These dependencies across projects can be something that > is tricky. In our case, sorting isn't the bottleneck at all. But it does make parallelization harder as I mentioned above. Unwinding and symbolication is usually what's much much slower compared to just sorting a couple of events in-memory. Regarding missing abstractions: I'm a C++ developer myself, and having easy access to more abstractions like hash maps and more has helped tremendously in improving the performance of our perf.data parser too. So yes, I again totally agree with you in that regards. > > As I said - I'm happy to discuss this topic in more detail with anyone > > interested in this topic. > > I'm interested! There is an IRC channel #perf see www.oftc.net, but > I'd suggest the mailing list is better for discussion. I know that BPF > is doing regular office hours. I could imagine that similar for perf > would be useful, but I also know that maintainers are busy folks. I've joined that channel now. Cheers -- Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer KDAB (Deutschland) GmbH, a KDAB Group company Tel: +49-30-521325470 KDAB - The Qt, C++ and OpenGL Experts