Re: [RFC PATCH 00/11] bpf, trace, dtrace: DTrace BPF program type implementation and sample use

[Chris Mason <clm@fb.com>]

I'm being pretty liberal with chopping down quoted material to help 
emphasize a particular opinion about how to bootstrap existing 
out-of-tree projects into the kernel.  My goal here is to talk more 
about the process and less about the technical details, so please 
forgive me if I've ignored or changed the technical meaning of anything 
below.

On 30 May 2019, at 12:15, Kris Van Hees wrote:

> On Thu, May 23, 2019 at 01:28:44PM -0700, Alexei Starovoitov wrote:
>
> ... I believe that the discussion that has been going on in other
> emails has shown that while introducing a program type that provides a
> generic (abstracted) context is a different approach from what has 
> been done
> so far, it is a new use case that provides for additional ways in 
> which BPF
> can be used.
>

[ ... ]

>
> Yes and no.  It depends on what you are trying to do with the BPF 
> program that
> is attached to the different events.  From a tracing perspective, 
> providing a
> single BPF program with an abstract context would ...

[ ... ]

>
> In this model kprobe/ksys_write and 
> tracepoint/syscalls/sys_enter_write are
> equivalent for most tracing purposes ...

[ ... ]

>
> I agree with what you are saying but I am presenting an additional use 
> case

[ ... ]

>>
>> All that aside the kernel support for shared libraries is an awesome
>> feature to have and a bunch of folks want to see it happen, but
>> it's not a blocker for 'dtrace to bpf' user space work.
>> libbpf can be taught to do this 'pseudo shared library' feature
>> while 'dtrace to bpf' side doesn't need to do anything special.

[ ... ]

This thread intermixes some abstract conceptual changes with smaller 
technical improvements, and in general it follows a familiar pattern 
other out-of-tree projects have hit while trying to adapt the kernel to 
their existing code.  Just from this one email, I quoted the abstract 
models with use cases etc, and this is often where the discussions side 
track into less productive areas.

>
> So you are basically saying that I should redesign DTrace?

In your place, I would have removed features and adapted dtrace as much 
as possible to require the absolute minimum of kernel patches, or even 
better, no patches at all.  I'd document all of the features that worked 
as expected, and underline anything either missing or suboptimal that 
needed additional kernel changes.  Then I'd focus on expanding the 
community of people using dtrace against the mainline kernel, and work 
through the series features and improvements one by one upstream over 
time.

Your current approach relies on an all-or-nothing landing of patches 
upstream, and this consistently leads to conflict every time a project 
tries it.  A more incremental approach will require bigger changes on 
the dtrace application side, but over time it'll be much easier to 
justify your kernel changes.  You won't have to talk in abstract models, 
and you'll have many more concrete examples of people asking for dtrace 
features against mainline.  Most importantly, you'll make dtrace 
available on more kernels than just the absolute latest mainline, and 
removing dependencies makes the project much easier for new users to 
try.

-chris