[PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Rafael J. Wysocki]

[I hope the ACKs still apply.]
---
From: Rafael J. Wysocki <rjw@sisk.pl>

At present, if a user mode helper is running while usermodehelper_pm_callback()
is executed, the helper may be frozen and the completion in
call_usermodehelper_exec() won't be completed until user space processes are
thawed.  As a result, the freezing of kernel threads may fail, which is not
desirable.

Prevent this from happening by introducing a counter of running user mode
helpers and allowing usermodehelper_pm_callback() to succeed for
action = PM_HIBERNATION_PREPARE or action = PM_SUSPEND_PREPARE only if there
are no helpers running.  [Namely, usermodehelper_pm_callback() waits for at most
RUNNING_HELPERS_TIMEOUT for the number of running helpers to become zero and
fails if that doesn't happen.]

Special thanks to Uli Luckas <u.luckas@road.de>, Pavel Machek <pavel@ucw.cz> and
Oleg Nesterov <oleg@tv-sign.ru> for reviewing the previous versions of this
patch and for very useful comments.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Uli Luckas <u.luckas@road.de>
Acked-by: Nigel Cunningham <nigel@nigel.suspend2.net>
Acked-by: Pavel Machek <pavel@ucw.cz>
---
 kernel/kmod.c |   78 ++++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 68 insertions(+), 10 deletions(-)

Index: linux-2.6.22-rc4-mm2/kernel/kmod.c
===================================================================
--- linux-2.6.22-rc4-mm2.orig/kernel/kmod.c
+++ linux-2.6.22-rc4-mm2/kernel/kmod.c
@@ -41,14 +41,6 @@ extern int max_threads;
 
 static struct workqueue_struct *khelper_wq;
 
-/*
- * If set, both call_usermodehelper_keys() and call_usermodehelper_pipe() exit
- * immediately returning -EBUSY.  Used for preventing user land processes from
- * being created after the user land has been frozen during a system-wide
- * hibernation or suspend operation.
- */
-static int usermodehelper_disabled;
-
 #ifdef CONFIG_KMOD
 
 /*
@@ -275,15 +267,55 @@ static void __call_usermodehelper(struct
 	}
 }
 
+#ifdef CONFIG_PM
+/*
+ * If set, call_usermodehelper_exec() will exit immediately returning -EBUSY
+ * (used for preventing user land processes from being created after the user
+ * land has been frozen during a system-wide hibernation or suspend operation).
+ */
+static int usermodehelper_disabled;
+
+/* Number of helpers running */
+static atomic_t running_helpers = ATOMIC_INIT(0);
+
+/*
+ * Wait queue head used by usermodehelper_pm_callback() to wait for all running
+ * helpers to finish.
+ */
+static DECLARE_WAIT_QUEUE_HEAD(running_helpers_waitq);
+
+/*
+ * Time to wait for running_helpers to become zero before the setting of
+ * usermodehelper_disabled in usermodehelper_pm_callback() fails
+ */
+#define RUNNING_HELPERS_TIMEOUT	(5 * HZ)
+
 static int usermodehelper_pm_callback(struct notifier_block *nfb,
 					unsigned long action,
 					void *ignored)
 {
+	long retval;
+
 	switch (action) {
 	case PM_HIBERNATION_PREPARE:
 	case PM_SUSPEND_PREPARE:
 		usermodehelper_disabled = 1;
-		return NOTIFY_OK;
+		smp_mb();
+		/*
+		 * From now on call_usermodehelper_exec() won't start any new
+		 * helpers, so it is sufficient if running_helpers turns out to
+		 * be zero at one point (it may be increased later, but that
+		 * doesn't matter).
+		 */
+		retval = wait_event_timeout(running_helpers_waitq,
+					atomic_read(&running_helpers) == 0,
+					RUNNING_HELPERS_TIMEOUT);
+		if (retval) {
+			return NOTIFY_OK;
+		} else {
+			usermodehelper_disabled = 0;
+			return NOTIFY_BAD;
+		}
 	case PM_POST_HIBERNATION:
 	case PM_POST_SUSPEND:
 		usermodehelper_disabled = 0;
@@ -293,6 +325,30 @@ static int usermodehelper_pm_callback(st
 	return NOTIFY_DONE;
 }
 
+static void helper_lock(void)
+{
+	atomic_inc(&running_helpers);
+	smp_mb__after_atomic_inc();
+}
+
+static void helper_unlock(void)
+{
+	if (atomic_dec_and_test(&running_helpers))
+		wake_up(&running_helpers_waitq);
+}
+
+static void register_pm_notifier_callback(void)
+{
+	pm_notifier(usermodehelper_pm_callback, 0);
+}
+#else /* CONFIG_PM */
+#define usermodehelper_disabled	0
+
+static inline void helper_lock(void) {}
+static inline void helper_unlock(void) {}
+static inline void register_pm_notifier_callback(void) {}
+#endif /* CONFIG_PM */
+
 /**
  * call_usermodehelper_setup - prepare to call a usermode helper
  * @path - path to usermode executable
@@ -397,6 +453,7 @@ int call_usermodehelper_exec(struct subp
 	DECLARE_COMPLETION_ONSTACK(done);
 	int retval;
 
+	helper_lock();
 	if (sub_info->path[0] == '\0') {
 		retval = 0;
 		goto out;
@@ -418,6 +475,7 @@ int call_usermodehelper_exec(struct subp
 
   out:
 	call_usermodehelper_freeinfo(sub_info);
+	helper_unlock();
 	return retval;
 }
 EXPORT_SYMBOL(call_usermodehelper_exec);
@@ -459,5 +517,5 @@ void __init usermodehelper_init(void)
 {
 	khelper_wq = create_singlethread_workqueue("khelper");
 	BUG_ON(!khelper_wq);
-	pm_notifier(usermodehelper_pm_callback, 0);
+	register_pm_notifier_callback();
 }

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 112 bytes --]

On Tuesday 26 June 2007 07:52:23 Rafael J. Wysocki wrote:
> [I hope the ACKs still apply.]

Mine does :>

Nigel

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Pavel Machek]

Hi!

> [I hope the ACKs still apply.]

Uhuh, not 100% sure.

> +static int usermodehelper_disabled;
> +


...

>  	case PM_HIBERNATION_PREPARE:
>  	case PM_SUSPEND_PREPARE:
>  		usermodehelper_disabled = 1;
> -		return NOTIFY_OK;
> +		smp_mb();

usermodehelper_disabled should be atomic variable, too, so we don't
have to play these ugly tricks by hand? This should not be
performance-critical, right?
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Rafael J. Wysocki]

On Monday, 25 June 2007 23:55, Pavel Machek wrote:
> Hi!
> 
> > [I hope the ACKs still apply.]
> 
> Uhuh, not 100% sure.
> 
> > +static int usermodehelper_disabled;
> > +
> 
> 
> ...
> 
> >  	case PM_HIBERNATION_PREPARE:
> >  	case PM_SUSPEND_PREPARE:
> >  		usermodehelper_disabled = 1;
> > -		return NOTIFY_OK;
> > +		smp_mb();
> 
> usermodehelper_disabled should be atomic variable, too, so we don't
> have to play these ugly tricks by hand? This should not be
> performance-critical, right?

Well, I think we'd need to add the barriers anyway.

The problem, as far as I understand it, is that the instructions can get
reordered if there are no barriers in there.

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Pavel Machek]

Hi!

> > > [I hope the ACKs still apply.]
> > 
> > Uhuh, not 100% sure.
> > 
> > > +static int usermodehelper_disabled;
> > > +
> > 
> > 
> > ...
> > 
> > >  	case PM_HIBERNATION_PREPARE:
> > >  	case PM_SUSPEND_PREPARE:
> > >  		usermodehelper_disabled = 1;
> > > -		return NOTIFY_OK;
> > > +		smp_mb();
> > 
> > usermodehelper_disabled should be atomic variable, too, so we don't
> > have to play these ugly tricks by hand? This should not be
> > performance-critical, right?
> 
> Well, I think we'd need to add the barriers anyway.
> 
> The problem, as far as I understand it, is that the instructions can get
> reordered if there are no barriers in there.

Are you sure?  I thought atomic variables have barrirers built-in.

								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Benjamin Herrenschmidt]

On Tue, 2007-06-26 at 10:48 +0200, Pavel Machek wrote:
> > The problem, as far as I understand it, is that the instructions can
> get
> > reordered if there are no barriers in there.
> 
> Are you sure?  I thought atomic variables have barrirers built-in.

bzzzt. they don't.

Ben.

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Oleg Nesterov]

On 06/26, Rafael J. Wysocki wrote:
>
> On Monday, 25 June 2007 23:55, Pavel Machek wrote:
> > 
> > >  	case PM_HIBERNATION_PREPARE:
> > >  	case PM_SUSPEND_PREPARE:
> > >  		usermodehelper_disabled = 1;
> > > -		return NOTIFY_OK;
> > > +		smp_mb();
> > 
> > usermodehelper_disabled should be atomic variable, too, so we don't
> > have to play these ugly tricks by hand? This should not be
> > performance-critical, right?
> 
> Well, I think we'd need to add the barriers anyway.
> 
> The problem, as far as I understand it, is that the instructions can get
> reordered if there are no barriers in there.

Yes, and it doesn't help if we make usermodehelper_disabled atomic_t.
atomic_xxx() operations do not imply the memory barrier semantics.

Oleg.

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Benjamin Herrenschmidt]

On Tue, 2007-06-26 at 00:27 +0200, Rafael J. Wysocki wrote:
> > >     case PM_HIBERNATION_PREPARE:
> > >     case PM_SUSPEND_PREPARE:
> > >             usermodehelper_disabled = 1;
> > > -           return NOTIFY_OK;
> > > +           smp_mb();
> > 
> > usermodehelper_disabled should be atomic variable, too, so we don't
> > have to play these ugly tricks by hand? This should not be
> > performance-critical, right?
> 
> Well, I think we'd need to add the barriers anyway.
> 
> The problem, as far as I understand it, is that the instructions can
> get
> reordered if there are no barriers in there.

That seems dodgy either way to me :-)

Just use a spinlock.

Ben.

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Rafael J. Wysocki]

On Tuesday, 3 July 2007 07:30, Benjamin Herrenschmidt wrote:
> On Tue, 2007-06-26 at 00:27 +0200, Rafael J. Wysocki wrote:
> > > >     case PM_HIBERNATION_PREPARE:
> > > >     case PM_SUSPEND_PREPARE:
> > > >             usermodehelper_disabled = 1;
> > > > -           return NOTIFY_OK;
> > > > +           smp_mb();
> > > 
> > > usermodehelper_disabled should be atomic variable, too, so we don't
> > > have to play these ugly tricks by hand? This should not be
> > > performance-critical, right?
> > 
> > Well, I think we'd need to add the barriers anyway.
> > 
> > The problem, as far as I understand it, is that the instructions can
> > get
> > reordered if there are no barriers in there.
> 
> That seems dodgy either way to me :-)
> 
> Just use a spinlock.

Around wait_for_completion()?  I don't think that's a good idea. :-)

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Rafael J. Wysocki]

On Tuesday, 3 July 2007 15:19, Rafael J. Wysocki wrote:
> On Tuesday, 3 July 2007 07:30, Benjamin Herrenschmidt wrote:
> > On Tue, 2007-06-26 at 00:27 +0200, Rafael J. Wysocki wrote:
> > > > >     case PM_HIBERNATION_PREPARE:
> > > > >     case PM_SUSPEND_PREPARE:
> > > > >             usermodehelper_disabled = 1;
> > > > > -           return NOTIFY_OK;
> > > > > +           smp_mb();
> > > > 
> > > > usermodehelper_disabled should be atomic variable, too, so we don't
> > > > have to play these ugly tricks by hand? This should not be
> > > > performance-critical, right?
> > > 
> > > Well, I think we'd need to add the barriers anyway.
> > > 
> > > The problem, as far as I understand it, is that the instructions can
> > > get
> > > reordered if there are no barriers in there.
> > 
> > That seems dodgy either way to me :-)
> > 
> > Just use a spinlock.
> 
> Around wait_for_completion()?  I don't think that's a good idea. :-)

Sorry, I mistunderstood you (I think).

Yes, I could use a spinlock for protecting usermodehelper_disabled, but
why would that be better than the current code?

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Oleg Nesterov]

On 07/03, Benjamin Herrenschmidt wrote:
>
> On Tue, 2007-06-26 at 00:27 +0200, Rafael J. Wysocki wrote:
> > > >     case PM_HIBERNATION_PREPARE:
> > > >     case PM_SUSPEND_PREPARE:
> > > >             usermodehelper_disabled = 1;
> > > > -           return NOTIFY_OK;
> > > > +           smp_mb();
> > > 
> > > usermodehelper_disabled should be atomic variable, too, so we don't
> > > have to play these ugly tricks by hand? This should not be
> > > performance-critical, right?
> > 
> > Well, I think we'd need to add the barriers anyway.
> > 
> > The problem, as far as I understand it, is that the instructions can
> > get
> > reordered if there are no barriers in there.
> 
> That seems dodgy either way to me :-)
> 
> Just use a spinlock.

Actually, spinlock_t is not suitable. Because spin_unlcok() does NOT imply
mb(). The subsequent wait_event_timeout()->atomic_read() may leak into the
critical section.

We can use set_mb(), if we don't want to play with smp_mb() by hand :)

Oleg.

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Benjamin Herrenschmidt]

> Actually, spinlock_t is not suitable. Because spin_unlcok() does NOT imply
> mb(). The subsequent wait_event_timeout()->atomic_read() may leak into the
> critical section.
> 
> We can use set_mb(), if we don't want to play with smp_mb() by hand :)

spin_unlock implies a smp_wmb() but yeah, not a full mb(), though having
a read leak into a critical section is generally not an issue.

Ben.

Re: [PATCH -mm] PM: Prevent frozen user mode helpers from failing the freezing of tasks (rev. 2)

[Uli Luckas]

On Monday, 25. June 2007, Rafael J. Wysocki wrote:
> [I hope the ACKs still apply.]
>
Mine does 

> +static void helper_lock(void)
> +{
> +       atomic_inc(&running_helpers);
> +       smp_mb__after_atomic_inc();
> +}
> +
> +static void helper_unlock(void)
> +{
> +       if (atomic_dec_and_test(&running_helpers))
> +               wake_up(&running_helpers_waitq);
> +}
> +
> +static void register_pm_notifier_callback(void)
> +{
> +       pm_notifier(usermodehelper_pm_callback, 0);
> +}
> +#else /* CONFIG_PM */

> +static inline void helper_lock(void) {}
> +static inline void helper_unlock(void) {}
> +static inline void register_pm_notifier_callback(void) {}
>

Sorry for not realising this before, but I think, we should also inline the 
actual implementations of helper_lock, helper_unlock and 
register_pm_notifier_callback

Regards
Uli


-- 

------- ROAD ...the handyPC Company - - -  ) ) )

Uli Luckas
Software Development

ROAD GmbH
Bennigsenstr. 14 | 12159 Berlin | Germany
fon: +49 (30) 230069 - 64 | fax: +49 (30) 230069 - 69
url: www.road.de

Amtsgericht Charlottenburg: HRB 96688 B
Managing directors: Hans-Peter Constien, Hubertus von Streit