Re: [GIT PULL] kdbus for 4.1-rc1

[Andy Lutomirski <luto@amacapital.net>]

On Tue, Apr 21, 2015 at 3:31 AM, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
> On Mon, Apr 20, 2015 at 03:06:09PM -0700, Andy Lutomirski wrote:
>> On Mon, Apr 20, 2015 at 2:46 PM, Greg Kroah-Hartman
>> <gregkh@linuxfoundation.org> wrote:
>> > On Mon, Apr 20, 2015 at 11:16:49PM +0200, Richard Weinberger wrote:
>> >> Greg,
>> >>
>> >> Am 20.04.2015 um 22:56 schrieb Greg Kroah-Hartman:
>> >> >> In which situation on a common Linux system is the current dbus too slow today?
>> >> >> I've never seen a issue like "Oh my system is slow because dbus is
>> >> >> eating too much CPU cycles".
>> >> >
>> >> > See the original email which explained all of the things we can not do
>> >> > with D-Bus, some of which are due to speed, that can now be done with the
>> >> > kdbus code.
>> >>
>> >> okay, let's do it together.
>> >>
>> >> 1. Performance
>> >> You write:
>> >> "DBus is not used for performance sensitive applications because DBus is slow.
>> >> We want to make it fast so we can finally use it for low-latency,
>> >> high-throughput applications."
>> >>
>> >> Which applications exactly?
>> >> This reads to me like a solution for a non-existing problem.
>> >
>> > Anything that uses UDS for large buffers today can switch to using kdbus
>> > for it's data stream as it is faster.  I know the Pulse Audio people
>> > have discussed this, and there are other people as well (Enlightenment
>> > library developers, glib, wayland, etc.)  Without the code being in the
>> > kernel, no project is going to spend the time to convert their codebase
>> > to a feature that isn't accepted.
>>
>> Anything that uses UDS for large buffers today can switch to using
>> memfd over SCM_RIGHTS right now.  If SCM_RIGHTS is too slow, then we
>> can fix it along the lines that Al proposed.
>
> But that doesn't solve the latency issues.

I said memfd, not memfd bounced off a userspace daemon.  AFAICT
AF_UNIX peer-to-peer is considerably faster than kdbus, and I don't
see why memfd would change this.

>
> As has been said many times in this thread, when using UDS to build a
> better IPC for apps, you will probably end up with todays D-Bus
> userspace implementation, and not have any of the other things that we
> keep talking about kdbus having.
>
> Bringing up SCM_RIGHTS means that this is not going to be a bus system
> at all.  One principal design goal is to _not_ have peer-to-peer
> connections between all communicating parties, but rather one connection
> to a central component.  If that component is not in the kernel, it has
> to be a userspace deamon, which in turn has all of the issues that
> dbus-daemon currently has.
>

AFAICT userspace dbus-daemon has two major problems:

1. SCM_RIGHTS sucks.  That's why I proposed fixing it.

2. Performance.  But using an in-kernel bus is far from the only
solution.  I much prefer adding something simple and flexible in the
kernel so that a userspace daemon can easily and efficiently introduce
two bus users to each other.

>> >> 3. Semantics for apps with heavy data payloads
>> >>
>> >> Again, sounds like a solution for a non-existing problem.
>> >
>> > No, media apps need to share their data somehow, and kdbus provides a
>> > way to do that.  GNOME portals are one such proposed codebase that is
>> > looking to use kdbus for this, and again, so is Pulse Audio and the
>> > other groups listed above.
>>
>> AFAICT you're talking about passing data into and out of a sandbox for
>> processing or UI purposes.  We have two excellent ways to do that
>> right now: memfd and splice, depending on exactly what you're doing.
>
> That does not solve the latency issues, which is crucial for sound and
> graphics.

As above, there's only a latency issue right now if you want sound and
graphics to use a *bus*, and even that could be fixed without moving
the bus into the kernel.

>
>> >> 4. "Being in the kernel closes a lot of races which can't be fixed with
>> >>    the current userspace solutions."
>> >>
>> >> You really need a in-kernel dbus with 13k to solve that?
>> >
>> > Do you know of a smaller amount of code to solve this problem?  If so,
>> > wonderful, please show us, but we aren't playing code golf here.  We are
>> > proposing something that is well documented and easy to maintain, while
>> > still being fast and correct.  If it you think this can be done in a
>> > smaller amount of code, please show us where we are doing needless
>> > things in the patches.
>>
>> I do.  Implement something like my old SCM_IDENTITY proposal, which is
>> kind of like kdbus metadata, opt-in, over UNIX sockets.  Except that I
>> never proposed most of the absurd metadata items that kdbus is
>> proposing, and I also suggesting doing it over plain old UNIX sockets.
>
> We _want_ this metadata.  You don't, that's fine.  Calling our position
> "absurd" does not contribute to the discussion.  We are simply exporting
> data that is already accessible via /proc and other locations, and do so
> in a race-free manner, something the kernel has never been able to
> provide in the past.
>
> We do not, in any way, export any additional internal kernel state,
> again, we are merely closing a race gap that has been there.

This has been covered ad nauseum on the systemd thread, so I'm going
not going to respond here.

>
>> > Because of that, and the thread where the proposed security problems
>> > were agreed not to be a security problem, I don't see a reason anymore
>> > why this code should not be merged.
>> >
>> > With the exception of Al's code review, which is being addressed.  But
>> > that's a minor thing, not a major design flaw at all.
>>
>> My NACK stands.  A security problem was fixed,
>
> Please note that this issue was addressed in v2, which was posted many
> months ago.  It is not present in this submission at all.

That's why I said "was fixed".

>
>> but the metadata system
>> has multiple problems, each of which is independently sufficient to
>> earn my nack.
>
> If you still see a problem, please explain what it is.  At least give a
> general outline so that we can try to understand where you are coming
> from here.  On the systemd mailing list you said that your only issue
> was that you are not convinced that this is a useful feature.  But now
> you are saying you have "multiple concerns".  What are they?
>

We've only discussed creds on the systemd list.  There's still cmdline
and starttime (at least).

I've actually *submitted patches* to fix starttime, but no one seems
to care.  i'll resubmit them anyway for 4.2, since I think they're
more generally useful.

[snip]

--Andy