Re: [PATCH 01/13] kdbus: add documentation

From: Andy Lutomirski <luto@amacapital.net>
To: David Herrmann <dh.herrmann@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>, "Ted Ts'o" <tytso@mit.edu>,
	Linux API <linux-api@vger.kernel.org>,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	Daniel Mack <daniel@zonque.org>,
	One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>,
	Austin S Hemmelgarn <ahferroin7@gmail.com>,
	Tom Gundersen <teg@jklm.no>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	Djalal Harouni <tixxdz@opendz.org>,
	Johannes Stezenbach <js@sig21.net>,
	Christoph Hellwig <hch@infradead.org>
Subject: Re: [PATCH 01/13] kdbus: add documentation
Date: Sun, 8 Feb 2015 08:54:42 -0800	[thread overview]
Message-ID: <CALCETrWOhw+OKUBNaTDUcqmgc21LfZ4EZ5zJvL9vzGoMd4_TmQ@mail.gmail.com> (raw)
In-Reply-To: <CANq1E4ShseFuTp0wPrHM9mFmgA-y9Kqz1m0-FmU9qALuxQ8Qvg@mail.gmail.com>

On Feb 4, 2015 4:16 PM, "David Herrmann" <dh.herrmann@gmail.com> wrote:
>
> Hi
>
> On Thu, Feb 5, 2015 at 12:03 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> > I see "latencies" of around 20 microseconds with lockdep and context
> > tracking off.  For example:
>
> Without metadata nor memfd transmission, I get 2.5us for kdbus, 1.5us
> for UDS (8k payload). With 8-byte payloads, I get 2.2us and 1.2us. I
> suspect you enabled metadata transmission, which I think is not a fair
> comparison.

I tried to disable metadata.  I may have failed.

Regardless, if metadata is very slow, then that's more reason not to
use it on send.  And if you shouldn't use it, then maybe the kernel
shouldn't provide it.

I assumed there was a context switch in there.  I can try to test
differently.  If UDS is twice as fast *with* a contest switch, then a
userspace solution should be faster.

Also, UDS can use memfds, too.

>
> A few notes on that:
>
> * kdbus is a bus layer. We don't intend to replace UDS, but improve
> dbus. Comparing roundtrip times with UDS is tempting, but in no way
> fair. To the very least, a bus layer has to perform peer-lookup, which
> UDS does not have to do. Imo, 2.5us vs. 1.5us is already pretty nice.
> Compare this to ~77us for dbus1 without marshaling.

This makes me wonder what dbus1 is doing wrong.

>
> * We have not optimized kdbus code-paths for speed, yet. Our main
> concerns are algorithmic challenges, and we believe they've been
> improved considerably with kdbus. I have constantly measured kdbus
> performance with 'perf' and flame-graphs, and there're a lot of
> possible optimizations (especially on locking). However, I think this
> can be done afterwards just fine. Neither API nor ioctl overhead has
> shown up in my measurements. If anyone has counter evidence, please
> let us know. But I'm a bit reluctant to change our API solely based on
> performance guesses.

But removal of send-time metadata can't be done after the fact.

--Andy