From: hpa@zytor.com
To: Joel Fernandes <joel@joelfernandes.org>
Cc: Greg KH <gregkh@linuxfoundation.org>,
Christoph Hellwig <hch@infradead.org>,
linux-kernel@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
ast@kernel.org, atishp04@gmail.com,
Borislav Petkov <bp@alien8.de>,
dancol@google.com, Ingo Molnar <mingo@redhat.com>,
Jan Kara <jack@suse.cz>, Jonathan Corbet <corbet@lwn.net>,
karim.yaghmour@opersys.com, Kees Cook <keescook@chromium.org>,
kernel-team@android.com, linux-doc@vger.kernel.org,
Manoj Rao <linux@manojrajarao.com>,
Masahiro Yamada <yamada.masahiro@socionext.com>,
paulmck@linux.vnet.ibm.com,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
rdunlap@infradead.org, rostedt@goodmis.org,
Thomas Gleixner <tglx@linutronix.de>,
"maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)"
<x86@kernel.org>,
yhs@fb.com
Subject: Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel
Date: Sun, 20 Jan 2019 18:49:56 -0800 [thread overview]
Message-ID: <BC774ED4-C0FC-4AA6-9BAF-36930A661D9B@zytor.com> (raw)
In-Reply-To: <20190121014553.GD23827@google.com>
On January 20, 2019 5:45:53 PM PST, Joel Fernandes <joel@joelfernandes.org> wrote:
>On Sun, Jan 20, 2019 at 01:58:15PM -0800, hpa@zytor.com wrote:
>> On January 20, 2019 8:10:03 AM PST, Joel Fernandes
><joel@joelfernandes.org> wrote:
>> >On Sat, Jan 19, 2019 at 11:01:13PM -0800, hpa@zytor.com wrote:
>> >> On January 19, 2019 2:36:06 AM PST, Greg KH
>> ><gregkh@linuxfoundation.org> wrote:
>> >> >On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig
>wrote:
>> >> >> This seems like a pretty horrible idea and waste of kernel
>memory.
>> >> >
>> >> >It's only a waste if you want it to be a waste, i.e. if you load
>the
>> >> >kernel module.
>> >> >
>> >> >This really isn't any different from how /proc/config.gz works.
>> >> >
>> >> >> Just add support to kbuild to store a compressed archive in
>> >initramfs
>> >> >> and unpack it in the right place.
>> >> >
>> >> >I think the issue is that some devices do not use initramfs, or
>> >switch
>> >> >away from it after init happens or something like that. Joel has
>> >all
>> >> >of
>> >> >the looney details that he can provide.
>> >> >
>> >> >thanks,
>> >> >
>> >> >greg k-h
>> >>
>> >> Yeah, well... but it is kind of a losing game... the more
>in-kernel
>> >stuff there is the less smiley are things to actually be supported.
>> >
>> >It is better than nothing, and if this makes things a bit easier and
>> >solves
>> >real-world issues people have been having, and is optional, then I
>> >don't see
>> >why not.
>> >
>> >> Modularizing is it in some ways even crazier in the sense that at
>> >that point you are relying on the filesystem containing the module,
>> >which has to be loaded into the kernel by a root user. One could
>even
>> >wonder if a better way to do this would be to have "make
>> >modules_install" park an archive file – or even a directory as
>opposed
>> >to a symlink – with this stuff in /lib/modules. We could even
>provide a
>> >tmpfs shim which autoloads such an archive via the firmware loader;
>> >this might even be generically useful, who knows.
>> >
>> >All this seems to assume where the modules are located. In Android,
>we
>> >don't
>> >have /lib/modules. This patch generically fits into the grand scheme
>> >things
>> >and I think is just better made a part of the kernel since it is not
>> >that
>> >huge once compressed, as Dan also pointed. The more complex, and the
>> >more
>> >assumptions we make, the less likely people writing tools will get
>it
>> >right
>> >and be able to easily use it.
>> >
>> >>
>> >> Note also that initramfs contents can be built into the kernel.
>> >Extracting such content into a single-instance tmpfs would again be
>a
>> >possibility
>> >
>> >Such an approach would bloat the kernel image size though, which may
>> >not work
>> >for everyone. The module based approach, on the other hand, gives an
>> >option
>> >to the user to enable the feature, but not have it loaded into
>memory
>> >or used
>> >until it is really needed.
>> >
>> >thanks,
>> >
>> > - Joel
>>
>> Well, where are the modules? They must exist in the filesystem.
>
>The scheme of loading a module doesn't depend on _where_ the module is
>on the
>filesystem. As long as a distro knows how to load a module in its own
>way (by
>looking into whichever paths it cares about), that's all that matters.
>And
>the module contains compressed headers which saves space, vs storing it
>uncompressed on the file system.
>
>To remove complete reliance on the filesystem, there is an option of
>not
>building it as a module, and making it as a built-in.
>
>I think I see your point now - you're saying if its built-in, then it
>becomes kernel memory that is lost and unswappable. Did I get that
>right?
>I am saying that if that's a major concern, then:
>1. Don't make it a built-in, make it a module.
>2. Don't enable it at for your distro, and use a linux-headers package
>or
>whatever else you have been using so far that works for you.
>
>thanks,
>
> - Joel
My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable; otherwise nothing is gained.
I am *not* disagreeing with the problem statement!
Now, /proc isn't something that will autoload modules. A filesystem *will*, although you need to be able to mount it; furthermore, it makes it trivially to extend it (and the firmware interface provides an . easy way to feed the data to such a filesystem without having to muck with anything magic.)
Heck, we could even make it a squashfs image that can just be mounted.
So, first of all, where does Android keep its modules, and what is actually included? Is /sbin/modprobe used to load the modules, as is normal? We might even be able to address this with some fairly trivial enhancements to modprobe; specifically to search in the module paths for something that isn't a module per se.
The best scenario would be if we could simply have the tools find the location equivalent of /lib/modules/$version/source...
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
next prev parent reply other threads:[~2019-01-21 8:02 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-18 22:55 [RFC] Provide in-kernel headers for making it easy to extend the kernel Joel Fernandes
2019-01-19 8:25 ` Greg KH
2019-01-19 16:27 ` Joel Fernandes
2019-01-19 17:43 ` Daniel Colascione
2019-01-19 23:25 ` Joel Fernandes
2019-01-19 23:44 ` hpa
2019-01-20 15:58 ` Joel Fernandes
2019-03-06 23:09 ` Pavel Machek
2019-03-06 23:37 ` Daniel Colascione
2019-03-07 0:07 ` H. Peter Anvin
2019-03-07 0:33 ` Daniel Colascione
2019-03-07 1:22 ` Enrico Weigelt, metux IT consult
2019-03-07 1:49 ` Daniel Colascione
2019-03-07 20:41 ` Enrico Weigelt, metux IT consult
2019-03-07 20:55 ` Greg KH
2019-03-07 22:11 ` Enrico Weigelt, metux IT consult
2019-03-07 23:12 ` Joel Fernandes
2019-03-07 23:40 ` hpa
2019-03-08 3:16 ` Joel Fernandes
2019-03-07 1:42 ` Joel Fernandes
2019-03-07 16:24 ` Enrico Weigelt, metux IT consult
2019-03-07 0:32 ` H. Peter Anvin
2019-03-07 0:36 ` Daniel Colascione
2019-03-07 0:42 ` Enrico Weigelt, metux IT consult
2019-03-07 1:48 ` Joel Fernandes
2019-03-07 17:37 ` Enrico Weigelt, metux IT consult
2019-01-19 8:26 ` Greg KH
2019-01-19 16:27 ` Joel Fernandes
2019-01-19 10:28 ` Christoph Hellwig
2019-01-19 10:36 ` Greg KH
2019-01-19 16:26 ` Joel Fernandes
2019-01-20 7:01 ` hpa
2019-01-20 16:10 ` Joel Fernandes
2019-01-20 21:58 ` hpa
2019-01-21 1:45 ` Joel Fernandes
2019-01-21 2:49 ` hpa [this message]
2019-01-21 4:38 ` Sandeep Patil
2019-01-22 13:39 ` Joel Fernandes
2019-01-23 21:29 ` Karim Yaghmour
2019-01-23 22:37 ` Daniel Colascione
2019-01-24 2:32 ` Joel Fernandes
2019-01-24 14:18 ` Joel Fernandes
2019-01-24 18:57 ` Karim Yaghmour
2019-01-24 20:59 ` Joel Fernandes
2019-01-25 19:00 ` hpa
2019-01-25 19:15 ` Daniel Colascione
2019-01-25 19:51 ` hpa
2019-01-25 20:34 ` Daniel Colascione
2019-01-25 20:46 ` Joel Fernandes
2019-01-25 20:28 ` Joel Fernandes
2019-03-06 23:09 ` Pavel Machek
2019-03-06 23:35 ` H. Peter Anvin
2019-01-26 12:05 Norbert Lange
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BC774ED4-C0FC-4AA6-9BAF-36930A661D9B@zytor.com \
--to=hpa@zytor.com \
--cc=akpm@linux-foundation.org \
--cc=ast@kernel.org \
--cc=atishp04@gmail.com \
--cc=bp@alien8.de \
--cc=corbet@lwn.net \
--cc=dancol@google.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=joel@joelfernandes.org \
--cc=karim.yaghmour@opersys.com \
--cc=keescook@chromium.org \
--cc=kernel-team@android.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@manojrajarao.com \
--cc=mingo@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yamada.masahiro@socionext.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).