From: Rusty Russell <rusty@rustcorp.com.au>
To: "Luis R. Rodriguez" <mcgrof@kernel.org>
Cc: Filipe Manana <fdmanana@suse.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
linux-doc@vger.kernel.org, rgoldwyn@suse.com,
hare <hare@suse.com>, Jonathan Corbet <corbet@lwn.net>,
Linus Torvalds <torvalds@linux-foundation.org>,
linux-kselftest@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Dan Williams <dan.j.williams@intel.com>,
Aaron Tomlin <atomlin@redhat.com>,
rwright@hpe.com, Heinrich Schuchardt <xypron.glpk@gmx.de>,
Michal Marek <mmarek@suse.com>,
martin.wilck@suse.com, Jeff Mahoney <jeffm@suse.com>,
Ingo Molnar <mingo@redhat.com>, Petr Mladek <pmladek@suse.com>,
Dmitry Torokhov <dmitry.torokhov@gmail.com>,
Guenter Roeck <linux@roeck-us.net>,
"Eric W. Biederman" <ebiederm@xmission.com>,
shuah@kernel.org, DSterba@suse.com,
Kees Cook <keescook@chromium.org>,
Josh Poimboeuf <jpoimboe@redhat.com>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Miroslav Benes <mbenes@suse.cz>, NeilBrown <neilb@suse.com>,
linux-kernel@vger.kernel.org, David Miller <davem@davemloft.net>,
Jessica Yu <jeyu@redhat.com>,
Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>,
Julia Lawall <julia.lawall@lip6.fr>
Subject: Re: [RFC 10/10] kmod: add a sanity check on module loading
Date: Tue, 20 Dec 2016 11:23:08 +1030 [thread overview]
Message-ID: <87fuljjua3.fsf@rustcorp.com.au> (raw)
In-Reply-To: <CAB=NE6VvuA9a6hf6yoopGfUxVJQM5HyV5bNzUdsEtUV0UhbG-g@mail.gmail.com>
"Luis R. Rodriguez" <mcgrof@kernel.org> writes:
> On Dec 16, 2016 9:54 PM, "Rusty Russell" <rusty@rustcorp.com.au> wrote:
> > AFAICT the mistake here is that kmod is returning "done, OK" when the
> > module it is trying to load is already loading (but not finished
> > loading). That's the root problem; it's an attempt at optimization by
> > kmod which goes awry.
>
> This is true! To be precise though the truth of the matter is that kmod'd
> respective usermode helper: modprobe can be buggy and may lie to us. It may
> allow request_module() to return 0 but since we don't validate it, any
> assumption we make can be deadly. In the case of get_fs_type() its a null
> dereference.
Wait, what?? I can't see that in get_fs_type, which hasn't changed
since 2013. If a caller is assuming get_fs_type() doesn't return NULL,
they're broken and need fixing of course:
struct file_system_type *get_fs_type(const char *name)
{
struct file_system_type *fs;
const char *dot = strchr(name, '.');
int len = dot ? dot - name : strlen(name);
fs = __get_fs_type(name, len);
if (!fs && (request_module("fs-%.*s", len, name) == 0))
fs = __get_fs_type(name, len);
if (dot && fs && !(fs->fs_flags & FS_HAS_SUBTYPE)) {
put_filesystem(fs);
fs = NULL;
}
return fs;
}
Where does this NULL-deref is the module isn't correctly loaded?
> *Iff* we want a sanity check to verify kmod's umh is not lying to us we
> need to verify after 0 was returned that it was not lying to us. Since kmod
> accepts aliases but find_modules_all() only works on the real module name a
> validation check cannot happen when all you have are aliases.
request_module() should block until resolution, but that's fundamentally
a userspace problem. Let's not paper over it in kernelspace.
> *Iff* we are sure we don't want a validation (or another earlier
> optimization to avoid calling out to modrobe if the alias requested is
> already present, which does the time shaving I mentioned on the tests) then
> naturally no request_module() calls returning 0 can assert information
> about the requested module. I think we might need to change more code if we
> accept we cannot trust request_module() calls, or we accept userspace
> telling the kernel something may mean we sometimes crash. This later
> predicament seems rather odd so hence the patch.
>
> Perhaps in some cases validation of work from a umh is not critical in
> kernel but for request_module() I can tell you that today get_fs_type code
> currently asserts the module found can never be NULL.
OK, what am I missing in the code above?
> > Looking at the code in the kernel, we *already* get this right: block if
> > a module is still loading anyway. Once it succeeds we return -EBUSY if
> >
> > it fails we'll proceed to try to load it again.
> >
> > I don't understand what you're trying to fix with adding aliases
> > in-kernel?
>
> Two fold now:
>
> a) validation on request_module() work when an alias is used
But why?
> b) since kmod accepts aliaes, if we get aliases support, it means we could
> *also* preemptively avoid calling out to userspace for modules already
> present.
No, because once we have a module we don't request it: requesting is the
fallback case.
> >> FWIW a few things did occur to me:
> >>
> >> a) list_add_rcu() is used so new modules get added first
> >
> > Only after we're sure that there are no duplicates.
> >
> >
> OK! This is a very critical assertion. I should be able to add a debug
> WARN_ON() should two modules be on the modules list for the same module
> then ?
Yes, names must be unique.
>> b) find_module_all() returns the last module which was added as it
> traverses
>> the module list
>
>> BTW should find_module_all() use rcu to traverse?
>
> Yes; the kallsyms code does this on Oops. Not really a big issue in
> practice, but a nice fix.
>
> Ok, will bundle into my queue.
Please submit to Jessica for her module queue, as it's orthogonal
AFAICT.
> I will note though that I still think there's a bug in this code --
> upon a failure other "spinning" requests can fail, I believe this may
> be due to not having another state or informing pending modules too
> early of a failure but I haven't been able to prove this conjecture
> yet.
That's possible, but I can't see it from quickly re-checking the code.
The module should be fully usable at this point; the module's init has
been called successfully, so in the case of __get_fs_type() it should
now succeed. The module cleans up its init section, but that should be
independent.
If there is a race, it's likely to be when some other caller wakes the
queue. Moving the wakeup as soon as possible should make it easier to
trigger:
diff --git a/kernel/module.c b/kernel/module.c
index f57dd63186e6..78bd89d41a22 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -3397,6 +3397,7 @@ static noinline int do_init_module(struct module *mod)
/* Now it's a first class citizen! */
mod->state = MODULE_STATE_LIVE;
+ wake_up_all(&module_wq);
blocking_notifier_call_chain(&module_notify_list,
MODULE_STATE_LIVE, mod);
@@ -3445,7 +3446,6 @@ static noinline int do_init_module(struct module *mod)
*/
call_rcu_sched(&freeinit->rcu, do_free_init);
mutex_unlock(&module_mutex);
- wake_up_all(&module_wq);
return 0;
Thanks,
Rusty.
next prev parent reply other threads:[~2016-12-20 3:06 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-08 18:47 [RFC 00/10] kmod: stress test driver, few fixes and enhancements Luis R. Rodriguez
2016-12-08 18:47 ` [RFC 01/10] kmod: add test driver to stress test the module loader Luis R. Rodriguez
2016-12-08 20:24 ` Kees Cook
2016-12-13 21:10 ` Luis R. Rodriguez
2016-12-16 7:41 ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 02/10] module: fix memory leak on early load_module() failures Luis R. Rodriguez
2016-12-08 20:30 ` Kees Cook
2016-12-08 21:10 ` Luis R. Rodriguez
2016-12-08 21:17 ` Kees Cook
2016-12-09 17:06 ` Miroslav Benes
2016-12-16 8:51 ` Luis R. Rodriguez
2016-12-15 18:46 ` Aaron Tomlin
2016-12-08 19:48 ` [RFC 03/10] kmod: add dynamic max concurrent thread count Luis R. Rodriguez
2016-12-08 20:28 ` Kees Cook
2016-12-08 21:00 ` Luis R. Rodriguez
2016-12-14 15:38 ` Petr Mladek
2016-12-16 8:39 ` Luis R. Rodriguez
2017-01-10 19:24 ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 04/10] kmod: provide wrappers for kmod_concurrent inc/dec Luis R. Rodriguez
2016-12-08 20:29 ` Kees Cook
2016-12-08 21:08 ` Luis R. Rodriguez
2016-12-15 12:46 ` Petr Mladek
2016-12-16 8:05 ` Luis R. Rodriguez
2016-12-22 4:48 ` Jessica Yu
2017-01-06 20:54 ` Luis R. Rodriguez
2017-01-10 18:57 ` [RFC 04/10] " Luis R. Rodriguez
2017-01-11 20:08 ` Luis R. Rodriguez
2017-05-16 18:02 ` Luis R. Rodriguez
2017-05-18 2:37 ` Luis R. Rodriguez
2016-12-22 5:07 ` Jessica Yu
2017-01-10 20:28 ` Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 05/10] kmod: return -EBUSY if modprobe limit is reached Luis R. Rodriguez
2016-12-08 19:48 ` [RFC 06/10] kmod: provide sanity check on kmod_concurrent access Luis R. Rodriguez
2016-12-14 16:08 ` Petr Mladek
2016-12-14 17:12 ` Luis R. Rodriguez
2016-12-15 12:57 ` Petr Mladek
2017-01-10 20:00 ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 07/10] kmod: use simplified rate limit printk Luis R. Rodriguez
2016-12-14 16:23 ` Petr Mladek
2016-12-14 16:41 ` Joe Perches
2016-12-16 8:44 ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 08/10] sysctl: add support for unsigned int properly Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 09/10] kmod: add helpers for getting kmod count and limit Luis R. Rodriguez
2016-12-15 16:56 ` Petr Mladek
2016-12-16 7:57 ` Luis R. Rodriguez
2017-01-11 18:27 ` Luis R. Rodriguez
2016-12-08 19:49 ` [RFC 10/10] kmod: add a sanity check on module loading Luis R. Rodriguez
2016-12-09 20:03 ` Martin Wilck
2016-12-09 20:56 ` Linus Torvalds
2016-12-15 18:08 ` Luis R. Rodriguez
2016-12-15 0:27 ` Rusty Russell
2016-12-16 8:31 ` Luis R. Rodriguez
2016-12-17 3:54 ` Rusty Russell
[not found] ` <CAB=NE6VvuA9a6hf6yoopGfUxVJQM5HyV5bNzUdsEtUV0UhbG-g@mail.gmail.com>
2016-12-20 0:53 ` Rusty Russell [this message]
2016-12-20 18:52 ` Luis R. Rodriguez
2016-12-21 2:21 ` Rusty Russell
2016-12-21 13:08 ` Luis R. Rodriguez
2017-01-03 0:04 ` Rusty Russell
2017-01-06 20:36 ` Luis R. Rodriguez
2017-01-06 21:53 ` Jessica Yu
2017-01-09 20:27 ` Luis R. Rodriguez
[not found] ` <87bmvgax51.fsf@rustcorp.com.au>
2017-01-09 19:56 ` [RFC 10/10] " Luis R. Rodriguez
2017-01-06 21:03 ` Jessica Yu
2017-01-04 2:47 ` Jessica Yu
2017-01-11 19:10 ` [RFC 00/10] kmod: stress test driver, few fixes and enhancements Luis R. Rodriguez
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87fuljjua3.fsf@rustcorp.com.au \
--to=rusty@rustcorp.com.au \
--cc=DSterba@suse.com \
--cc=acme@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=atomlin@redhat.com \
--cc=corbet@lwn.net \
--cc=dan.j.williams@intel.com \
--cc=davem@davemloft.net \
--cc=dmitry.torokhov@gmail.com \
--cc=ebiederm@xmission.com \
--cc=fdmanana@suse.com \
--cc=hare@suse.com \
--cc=jeffm@suse.com \
--cc=jeyu@redhat.com \
--cc=jpoimboe@redhat.com \
--cc=julia.lawall@lip6.fr \
--cc=keescook@chromium.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux@roeck-us.net \
--cc=martin.wilck@suse.com \
--cc=mbenes@suse.cz \
--cc=mcgrof@kernel.org \
--cc=mingo@redhat.com \
--cc=mmarek@suse.com \
--cc=neilb@suse.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=pmladek@suse.com \
--cc=rgoldwyn@suse.com \
--cc=rwright@hpe.com \
--cc=shuah@kernel.org \
--cc=subashab@codeaurora.org \
--cc=torvalds@linux-foundation.org \
--cc=xypron.glpk@gmx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).