Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

From: Daniel Colascione <dancol@google.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Karim Yaghmour <karim.yaghmour@opersys.com>,
	Geert Uytterhoeven <geert@linux-m68k.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	Joel Fernandes <joel@joelfernandes.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexei Starovoitov <ast@kernel.org>,
	atish patra <atishp04@gmail.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Guenter Roeck <groeck@chromium.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Kees Cook <keescook@chromium.org>,
	Android Kernel Team <kernel-team@android.com>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	linux-trace-devel@vger.kernel.org,
	Manoj Rao <linux@manojrajarao.com>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Qais Yousef <qais.yousef@arm.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	Shuah Khan <shuah@kernel.org>, Yonghong Song <yhs@fb.com>
Subject: Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel
Date: Mon, 11 Mar 2019 16:58:28 -0700	[thread overview]
Message-ID: <CAKOZuetHG0O-CiO6h5tD7aht_MT70ebwWMMSPkRUHdiM3wVq0Q@mail.gmail.com> (raw)
In-Reply-To: <20190311193612.4f09bf11@oasis.local.home>

On Mon, Mar 11, 2019 at 4:37 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Sat, 9 Mar 2019 16:44:31 -0500
> Karim Yaghmour <karim.yaghmour@opersys.com> wrote:
>
>
> > Sorry, I should've been clearer. I'm including eBPF/BCC into the
> > "user-space tools" here. That was in fact my prime motivation in
> > encouraging Joel at the last LPC to look at this. I've been integrating
> > the teaching of eBPF into my AOSP debugging and performance analysis
> > class (see CC courseware here:
> > http://www.opersys.com/training/android-debug-and-performance), and it
> > was pretty messy trying to show people how to benefit from such tools
> > under Android. Joel's present set of patches would obviate this problem.
>
> I've been reading this thread and staying out for the most part. But I
> was thinking about how I could use kernel headers for things like
> kprobes, and I want to mention the pony that I would like to have :-)
>
> Are headers really needed, and more importantly, are they enough? What
> if userspace is 32bit and the kernel is 64bit. Can ebpf scripts handle
> that?

Why would eBPF care about the bit width of userspace? I've thought a
lot about inspecting user state from the kernel and have some weird
ideas in that area (e.g., why not register eBPF programs to unwind
user stacks?) --- but I think that Joel's work is independent of all
that. I'm curious what you think we can do about userspace. I wish we
could just compile the world with frame pointers, but we can't.

> What I would love, is a table that has all structures and their fields
> with information about their types, size and signed types. Like the
> format fields in the events directory. This way ebpf (and kprobes,
> internally in the kernel) could access this table and be able to know
> what the data structures of the kernel is).

Think of the headers as encoding this information and more and the C
compiler as a magical decoder ring. :-) I totally get the desire for a
metadata format a little less messy than C code, but as a practical
matter, a rich C-compilation pipeline already exists, and the
debuginfo you're proposing doesn't, except in the form of DWARF, which
I think would be even more controversial to embed in the kernel memory
image than headers are. A header blob provides a strict superset of
the information your scheme provides.