Re: Redfish requests for dbus-sensors data are needlessly slow - analysis

From: "Andrew Jeffery" <andrew@aj.id.au>
To: "Ed Tanous" <ed@tanous.net>, "Paul Fertser" <fercerpav@gmail.com>
Cc: Zhikui Ren <zhikui.ren@intel.com>,
	Jae Hyun Yoo <jae.hyun.yoo@linux.intel.com>,
	Andrei Tutolmin <a.tutolmin@gagar-in.com>,
	Vernon Mauery <vernon.mauery@linux.intel.com>,
	OpenBMC Maillist <openbmc@lists.ozlabs.org>,
	Timofei Nikitin <t.nikitin@gagar-in.com>,
	"Bills, Jason M" <jason.m.bills@linux.intel.com>,
	Gunnar Mills <gmills@linux.vnet.ibm.com>
Subject: Re: Redfish requests for dbus-sensors data are needlessly slow - analysis
Date: Tue, 03 Aug 2021 13:28:40 +0930	[thread overview]
Message-ID: <572c7029-682e-4930-96d4-88730eac6c5c@www.fastmail.com> (raw)
In-Reply-To: <CACWQX81MBkR3JVRDGbJJMzgGgb3E03HfZTtKu_c0m51g4hXtoA@mail.gmail.com>

On Fri, 30 Jul 2021, at 00:23, Ed Tanous wrote:
> On Thu, Jul 29, 2021 at 2:46 AM Paul Fertser <fercerpav@gmail.com> wrote:
> > I would be willing to work on doing the right thing but for that I'm
> > lacking the understanding of the complete picture involved in handling
> > all types of sensors and interfaces, so I'm kindly asking for your
> > help with it.
> >
> > diff --git a/redfish-core/lib/sensors.hpp b/redfish-core/lib/sensors.hpp
> 
> Can you please submit the below to gerrit as a WIP instead.  There's
> much better tooling there for reviewing and testing patches.  I can
> review it more there.
> 
> FWIW, it's on my list to look into the mapper shared-memory caching
> layer that @arj wrote a while back.  It might also be able to solve
> this problem;  I don't have the link offhand, but I think it was
> trying to solve a similar problem, but handled the cache invalidation
> issue as well.

Here's the link:

https://github.com/amboar/shmapper

FWIW it's not a caching layer - mapper clients directly access the 
mapper data structures through the shared memory mapping. The client 
APIs are all 'const' so you have to go out of your way to damage the 
data (or we can provide private-copy based APIs).

It would be cool to get to the point where applications can push their 
DBus schema directly into the shmapper daemon. This would remove the 
performance horrors associated with the introspection phase of the 
current implementation.

Anyway, unfortunately I haven't had time to hack on it recently. The 
current codebase is mostly the result of a hyperfocus brain-dump of 
ideas underpinned by some fuzzing. Performance and ergonomics can be 
improved a bit. The underlying ideas are also quite similar to Boost's 
Interprocess module[1], though implemented in C leveraging the sparse 
semantic parser[2] (which I think makes it easier to take advantage of 
than a Boost implementation, but there are trade-offs).

[1] https://www.boost.org/doc/libs/1_76_0/doc/html/interprocess.html
[2] https://sparse.docs.kernel.org/en/latest/

Andrew