Re: BMC Performance Profiler

[Pasha Ghabussi <pashag@google.com>]

[-- Attachment #1: Type: text/plain, Size: 3489 bytes --]

Hello all,

We would really appreciate it if you can take a few minutes to read the
proposal sent earlier and let us know your thoughts and suggestions.

Thank you

On Mon, Oct 5, 2020 at 1:57 PM Pasha Ghabussi <pashag@google.com> wrote:

> Hello all,
>
> We would really appreciate it if you can take a few minutes to read the
> following proposal and let us know your thoughts and suggestions.
>
> We are developing a tool to investigate performance problems by looking at
> DBus traffic dumps. Current DBus inspection and visualization tools do not
> represent the DBus events similar to a typical performance profiler.
> Additionally, these tools do not address typical BMC workloads such as IPMI
> and ASIO. Hence, identifying potential performance problems requires
> inspecting the raw BMC DBus traffic, which can become a long and complex
> process. We want to add a graphical interface to webui-vue to visualize the
> DBus traffic to address the abovementioned problem.
>
> There have been DBus and IPMI performance-related discussions in the
> OpenBMC community, both of which can be helped by this work: IPMI-related
> issues started to appear as early as in 2017. One issue (#2630)
> <https://github.com/openbmc/openbmc/issues/2630> describes a problem
> related to large numbers of sensors. Its follow-up (#3098)
> <https://github.com/openbmc/openbmc/issues/3098> mentions “hostboot
> crashes due to poor IPMI performance”. Another issue (#2519)
> <https://github.com/openbmc/openbmc/issues/2519> describes a
> commonly-seen problem of IPMI taking very long to respond (> 5s).
> There are also discussions on RedFish performance
> <https://lists.ozlabs.org/pipermail/openbmc/2018-February/010735.html> on
> the mailing list; A patch
> <https://lists.ozlabs.org/pipermail/openbmc/2016-June/003380.html>
> optimized DBus performance by introducing a cache for name translation.
>
> All the performance investigations listed above involve DBus and may be
> helped by this work.
>
> We are planning to use the BMCweb file hosting functionality to access the
> DBus event dumps and visualize the events in the web UI. The available
> profiling tools such as dbus-pcap
> <https://github.com/openbmc/openbmc-tools/tree/master/amboar/obmc-scripts/dbus-pcap>,
> Wireshark <https://www.wireshark.org/>, Bustle
> <https://gitlab.freedesktop.org/bustle/bustle>, Snyh
> <https://github.com/snyh/dbus-profiler>, or DFeet
> <https://wiki.gnome.org/action/show/Apps/DFeet?action=show&redirect=DFeet>
> do not provide the exact functionality we are looking for. Our goal is to
> develop functionalities similar to other widely used profilers such as
> GPUView or VTune Profiler.
>
> One alternative solution considered was to stream DBus requests over
> websocket, but the existing websocket endpoints available on BMC webserver
> do not provide the exact information we need.
>
> Requirements and Scalability:
>
>    -
>
>    Should provide the adequate functionalities to filter, visualize the
>    events timeline, and group the DBus traffic based on multiple criteria such
>    as type, source, destination, path, interface, demon signatures, and more.
>    -
>
>    Should support capture of DBus messages using as little resources as
>    possible.
>    -
>
>    Should be able to show many (~thousands of) entries on screen
>    simultaneously
>    -
>
>    Integration with webui-vue
>
>
> Thank you
>

[-- Attachment #2: Type: text/html, Size: 14545 bytes --]