Re: [PATCH v5 1/3] Provide in-kernel headers to make extending kernel easier

From: Joel Fernandes <joel@joelfernandes.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Olof Johansson <olof@lixom.net>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Qais Yousef <qais.yousef@arm.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Manoj Rao <linux@manojrajarao.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexei Starovoitov <ast@kernel.org>,
	atish patra <atishp04@gmail.com>,
	Daniel Colascione <dancol@google.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Guenter Roeck <groeck@chromium.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Karim Yaghmour <karim.yaghmour@opersys.com>,
	Kees Cook <keescook@chromium.org>,
	Android Kernel Team <kernel-team@android.com>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	linux-trace-devel@vger.kernel.org,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Shuah Khan <shuah@kernel.org>, Yonghong Song <yhs@fb.com>
Subject: Re: [PATCH v5 1/3] Provide in-kernel headers to make extending kernel easier
Date: Thu, 11 Apr 2019 12:30:37 -0400	[thread overview]
Message-ID: <20190411163037.GA5786@google.com> (raw)
In-Reply-To: <20190411031540.ehezr6kq7ouobpzx@ast-mbp.dhcp.thefacebook.com>

On Wed, Apr 10, 2019 at 08:15:42PM -0700, Alexei Starovoitov wrote:
> On Wed, Apr 10, 2019 at 09:34:49AM -0700, Olof Johansson wrote:
> > On Wed, Apr 10, 2019 at 8:51 AM Joel Fernandes <joelaf@google.com> wrote:
> > >
> > > On Wed, Apr 10, 2019 at 11:07 AM Olof Johansson <olof@lixom.net> wrote:
> > > [snip]
> > > > > > Wouldn't it be more convenient to provide it in a standardized format
> > > > > > such that you won't have to take an additional step, and always have
> > > > > > This is that form IMO.
> ...
> > Compared to:
> >  - Extract tarball
> >  - Build and load
> >  - Remove file tree from filesystem
> 
> I think there are too many assumptions in this thread in regard to what
> is more convenient for user space.
> I think bcc should try to avoid extracting tarball into file system.
> For example libbcc can uncompress kheader.tar.xz into virtual file system
> of clang front-end. It's more or less std::map<string, string>
> with key=path, value=content of the file. Access to such in-memory
> 'files' is obviously faster than doing open/read syscalls.
> bcc already uses this approach for some bcc internal 'files' that
> it passes to clang during compilation.
> All of /proc/kheaders.tar.xz files can be passed the same way
> without extracting them into real file system.
> 
> Joel, would be great if you can share corresponding bcc patch
> that takes advantage of /proc/kheaders

That sounds good to me. Others have tested this with BCC with a manual
extraction and setting environment variable to point BCC to the extracted
location, but I can work BCC patch that makes this automatic and share it (I
was planning to do the "automatically make BCC do it" patch for the BCC
project, but first wanted this in. But I agree with Alexei, its a good idea
to do it now and share it). Masahiro had some nits in v6 that I need to
address anyway ;-) And Olof grumbled a bit about the commit message which
could be polished a bit more :-)

thanks!

 - Joel