[openbmc-tools] dbus-vis: DBus capture visualization tool

[openbmc-tools] dbus-vis: DBus capture visualization tool

[Sui Chen]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

Hello,

dbus-vis is a GUI tool that does the following:
- It visualizes busctl pcap files, the same format that dbus-pcap reads and
parses.
- It somewhat half-automates the process of doing a capture on the BMC and
transferring back to the host for analysis (console access to the BMC is
required for this purpose.)

dbus-vis started from an IPMI timeline visualization tool that only
processes IPMI requests exposed on DBus. After I learned about dbus-pcap
<https://github.com/openbmc/openbmc-tools/tree/master/amboar/obmc-scripts/dbus-pcap>,
it turns out DBus visualization is a natural extension to this
visualization tool. Because this tool currently accepts the same packet
capture format that dbus-pcap uses (by using a slightly modified version of
dbus-pcap), it kind of functions like a GUI version of dbus-pcap, and is
therefore named "dbus-vis" to indicate what it does.

The main difference between this tool and other existing dbus visualizers
(such as GNU bustle) is that dbus-vis shows data in a compact timeline
format, making it possible to show >10K events simultaneously on the screen
at an acceptable frame rate, as well as making it easier to focus on DBus
performance rather than the dependency of different DBus units (that other
existing tools seem to be focusing on.)

The changes for the first commit of dbus-vis is listed below; any comment
is greatly appreciated.
https://gerrit.openbmc-project.xyz/c/openbmc/openbmc-tools/+/34263

Thanks!
Sui

[-- Attachment #2: Type: text/html, Size: 2359 bytes --]

Re: [openbmc-tools] dbus-vis: DBus capture visualization tool

[Andrew Jeffery]

Hi Sui,

On Sat, 25 Jul 2020, at 08:29, Sui Chen wrote:
> Hello,
> 
> dbus-vis is a GUI tool that does the following:
> - It visualizes busctl pcap files, the same format that dbus-pcap reads 
> and parses.
> - It somewhat half-automates the process of doing a capture on the BMC 
> and transferring back to the host for analysis (console access to the 
> BMC is required for this purpose.)
> 
> dbus-vis started from an IPMI timeline visualization tool that only 
> processes IPMI requests exposed on DBus. After I learned about 
> dbus-pcap 
> <https://github.com/openbmc/openbmc-tools/tree/master/amboar/obmc-scripts/dbus-pcap>, it turns out DBus visualization is a natural extension to this visualization tool. Because this tool currently accepts the same packet capture format that dbus-pcap uses (by using a slightly modified version of dbus-pcap), it kind of functions like a GUI version of dbus-pcap, and is therefore named "dbus-vis" to indicate what it does.
> 
> The main difference between this tool and other existing dbus 
> visualizers (such as GNU bustle) is that dbus-vis shows data in a 
> compact timeline format, making it possible to show >10K events 
> simultaneously on the screen at an acceptable frame rate, as well as 
> making it easier to focus on DBus performance rather than the 
> dependency of different DBus units (that other existing tools seem to 
> be focusing on.)
> 
> The changes for the first commit of dbus-vis is listed below; any 
> comment is greatly appreciated.
> https://gerrit.openbmc-project.xyz/c/openbmc/openbmc-tools/+/34263
> 

This sounds really interesting, thanks for sharing your work! I've added myself as a reviewer (as the author of dbus-pcap).

I've taken a brief look at your patch and think there's a bit we could do to improve dbus-pcap for your needs (e.g. adding timestamps to the json output). I'd also like to work out how we can better integrate dbus-pcap with dbus-vis so we're not maintaining a fork of the code. Maybe if we package things properly they could both be more easily installed onto a system. That way we could describe the relationship in terms of package dependencies rather than forking or using wget for the job.

I'm also thinking it might be time to start (an) independent repo(s) where we can develop the two, what we have is starting to grow a bit beyond the intended scope of openbmc-tools. Looping Brad and Andrew Geissler in here to get their thoughts.

Cheers,

Andrew

Re: [openbmc-tools] dbus-vis: DBus capture visualization tool

[Andrew Geissler]

> On Jul 24, 2020, at 10:01 PM, Andrew Jeffery <andrew@aj.id.au> wrote:
> 
<snip>
> 
> I'm also thinking it might be time to start (an) independent repo(s) where we can develop the two, what we have is starting to grow a bit beyond the intended scope of openbmc-tools. Looping Brad and Andrew Geissler in here to get their thoughts.

Unless there’s a pressing need to move it, I’d vote we just keep it where it is. Once it’s matured a bit, y’all could propose on a move to a separate repo. I wonder if this even needs to be under the openbmc namespace? Seems like a fairly generic D-Bus tool that could be useful outside of openbmc?

> 
> Cheers,
> 
> Andrew

Re: [openbmc-tools] dbus-vis: DBus capture visualization tool

[Sui Chen]

[-- Attachment #1: Type: text/plain, Size: 2511 bytes --]

Hello,

(To Andrew Jeffery)
Yes I had been using the original dbus-pcap since the start of dbus-vis;
dbus-pcap was run twice to obtain both timestamps and JSON outputs. This
approach had been working until I encountered truncated messages, where
dbus-pcap had to be modified a bit so that those truncated messages do not
get discarded.
So far the only modification I need is to keep truncated messages (just
like Wireshark does) and appending timestamps in JSON outputs (so we need
to run dbus-pcap only once.)

(To Andrew Geissler)
In my understanding, the "IPMI" part is OpenBMC-specific, as it's not very
likely that someone runs the IPMI daemon on a regular computer.
This tool was initially intended only for IPMI, so there are functions that
look for text-encoded IPMI-related DBus calls in ipmi_parse.js . dbus-pcap
was added later and it's this addition that enables more generic use.

The question of generic use is an interesting one -- dbus-vis could be used
outside of OpenBMC, and it may complement existing tools due to a few
differences: It started something that resembles a performance analysis
tool, so it focused on showing large amounts of events/information to focus
in a compact way, with limited ability to inspect individual events. DBus
traffic on the OpenBMC can easily reach hundreds of messages per second,
whereas on desktop systems this number is generally a lot lower (based on
my limited experience.) This seems to affect the design of tools as well,
as existing tools like GNU Bustle seem to be designed for showing fewer
messages per second and are more focused on finding the dependency between
DBus function calls.

Thanks,
Sui


On Tue, Jul 28, 2020 at 6:06 AM Andrew Geissler <geissonator@gmail.com>
wrote:

>
>
> > On Jul 24, 2020, at 10:01 PM, Andrew Jeffery <andrew@aj.id.au> wrote:
> >
> <snip>
> >
> > I'm also thinking it might be time to start (an) independent repo(s)
> where we can develop the two, what we have is starting to grow a bit beyond
> the intended scope of openbmc-tools. Looping Brad and Andrew Geissler in
> here to get their thoughts.
>
> Unless there’s a pressing need to move it, I’d vote we just keep it where
> it is. Once it’s matured a bit, y’all could propose on a move to a separate
> repo. I wonder if this even needs to be under the openbmc namespace? Seems
> like a fairly generic D-Bus tool that could be useful outside of openbmc?
>
> >
> > Cheers,
> >
> > Andrew
>
>

[-- Attachment #2: Type: text/html, Size: 4090 bytes --]