Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

From: Masami Hiramatsu <mhiramat@kernel.org>
To: Joel Fernandes <joelaf@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
	Joel Fernandes <joel@joelfernandes.org>,
	Daniel Colascione <dancol@google.com>,
	Karim Yaghmour <karim.yaghmour@opersys.com>,
	Geert Uytterhoeven <geert@linux-m68k.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexei Starovoitov <ast@kernel.org>,
	atish patra <atishp04@gmail.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Guenter Roeck <groeck@chromium.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Kees Cook <keescook@chromium.org>,
	Android Kernel Team <kernel-team@android.com>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	linux-trace-devel@vger.kernel.org,
	Manoj Rao <linux@manojrajarao.com>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Qais Yousef <qais.yousef@arm.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	Shuah Khan <shuah@kernel.org>, Yonghong Song <yhs@fb.com>
Subject: Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel
Date: Wed, 13 Mar 2019 10:18:43 +0900	[thread overview]
Message-ID: <20190313101843.800dd8d6a9babb635580ebdc@kernel.org> (raw)
In-Reply-To: <CAJWu+oqsD_wHGWfzj6POFowiY2qcu04auVzkA6Lu9R+LR4gkrQ@mail.gmail.com>

On Mon, 11 Mar 2019 18:38:49 -0700
Joel Fernandes <joelaf@google.com> wrote:

> On Mon, Mar 11, 2019 at 6:28 PM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > On Mon, 11 Mar 2019 20:39:12 -0400
> > Joel Fernandes <joel@joelfernandes.org> wrote:
> >
> > > I think even though the kernel-headers can't have information about all data
> > > structures, they do already contain a lot of data structure definitions we
> > > need already. And anything needed can/should arguably be moved to include/ if
> > > they are really needed for kernel extension by something "external" to the
> > > kernel such as kernel modules or eBPF, right?
> >
> > That's not my worry. I would like to be able to easily walk data
> > structures from within the kernel, without having to do a lot of work
> > in userspace to get that information. The kprobe_events could then be
> > passed type casts or such to access data fields of arguments to
> > functions and such.
> 
> Ok.
> 
> > > In any case, such a solution such as what Steve suggested, still cannot do
> > > what we can with headers - such as build kernel modules on the fly using the
> > > C-compiler without any auto-generation of C code from any debug artifiacts.
> > > Think systemtap working with the module-backend without any need for
> > > linux-headers package on the file system. So such a solution would still be a
> > > bit orthogonal in scope to what this proposed solution can solve IMO.
> > >
> >
> > With the information I would like to have, it would be trivial to read
> > the data to create the header files needed for modules.
> 
> But there are macros and other #define things too. We lose all of them
> and can't recreate them from just DWARF (AFAIK). Including
> include/generated/autoconf.h which #defines the CONFIG options. For
> that we either need headers, or full kernel's sources with build
> artifacts.

What kind of macros would you concern?

> I do see a use case for the debug info you are talking about as you
> mentioned for the kprobe_events argument list types, and I already
> thought about it. But it does not seem to work for all the use cases I
> am referring to here.

But the eBPF is based on kprobe-events. What kind of usage would you
expected? (with macros??)

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>