linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
@ 2013-08-12 19:44 Andrew Savchenko
  2013-08-27  3:48 ` [BUG] " Andrew Savchenko
  0 siblings, 1 reply; 14+ messages in thread
From: Andrew Savchenko @ 2013-08-12 19:44 UTC (permalink / raw)
  To: linux-kernel; +Cc: linux-pm, viro, suspend-devel


[-- Attachment #1.1: Type: text/plain, Size: 2825 bytes --]

Hello,

after a kernel update from 3.5.7 to the latest stable I found that
user-space resume (from suspend-1.0 aka uswsusp) no longer works.
Kernel-space suspend and resume work fine (e.g. echo disk
> /sys/power/state), problem is with user-space support. (I need
user-space version because it supports image encryption.)

After resume (essentially linuxrc) application loads image it fails
to apply it:

========================================================
Processes could not be frozen, cannot continue resuming.
Error 11: Resource temporarily unavailable

You can now boot the system and lose the saved state
or reboot and try again.

[Notice that if you decide to reboot, you MUST NOT mount
any filesystems before a successful resume.
Resuming after some filesystems have been mounted
will badly damage these filesystems.]

Do you want to continue booting (Y/n)?
========================================================

Error code wasn't originally showed, I added it to suspend tool to
aid debugging. Essentially freeze ioctl on /dev/snapshot fails with
this error.

I bisected a commit which introduces this bug:

========================================================
commit ba4df2808a86f8b103c4db0b8807649383e9bd13 
Author: Al Viro <viro@zeniv.linux.org.uk> 
Date:   Tue Oct 2 15:29:10 2012 -0400 

    don't bother with kernel_thread/kernel_execve for launching
linuxrc 
    exec_usermodehelper_fns() will do just fine... 
    
    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> 
========================================================

In fact this commit induced/triggered at least two bugs: the first one
I'm facing now and the second one was fixed in commit
f0de17c0babe7f29381892def6b37e9181a53410:
make sure that /linuxrc has std{in,out,err}.

As a temporarily workaround for this issue I reverted all changes for
init/do_mounts_initrd.c up to the latest working commit
cb450766bcafc7bd7d40e9a5a0050745e8c68b3e considering the kernel API
changes (kernel_execve -> sys_execve). See linuxrc-workaround.patch.
I understand this isn't a proper solution, I just want to show what
code works for me.

I also found an interesting LKML discussion about s2disk and freezer
issue: http://www.spinics.net/lists/linux-nfs/msg38160.html
Maybe it is related to this bug, but patch proposed there doesn't in
my case.

Kernel config which fails with
ba4df2808a86f8b103c4db0b8807649383e9bd13 and works with
f0de17c0babe7f29381892def6b37e9181a53410 is also attached.

As this issue maybe hardware related, the system is 32-bit EEE PC
1000H with Atom N270, 2GB RAM, 750 GB SATA drive.

Additional (but probably useless) information on this bug may be found
here: https://forums.gentoo.org/viewtopic-p-7371120.html

Best regards,
Andrew Savchenko

[-- Attachment #1.2: config.xz --]
[-- Type: application/octet-stream, Size: 9816 bytes --]

[-- Attachment #1.3: linuxrc-workaround.patch --]
[-- Type: application/octet-stream, Size: 3030 bytes --]

diff --git a/init/do_mounts_initrd.c b/init/do_mounts_initrd.c
index 3e0878e..9b261ec 100644
--- a/init/do_mounts_initrd.c
+++ b/init/do_mounts_initrd.c
@@ -16,13 +16,13 @@
 #include <linux/initrd.h>
 #include <linux/sched.h>
 #include <linux/freezer.h>
-#include <linux/kmod.h>
 
 #include "do_mounts.h"
 
 unsigned long initrd_start, initrd_end;
 int initrd_below_start_ok;
 unsigned int real_root_dev;	/* do_proc_dointvec cannot handle kdev_t */
+static int __initdata old_fd, root_fd;
 static int __initdata mount_initrd = 1;
 
 static int __init no_initrd(char *str)
@@ -33,37 +33,33 @@ static int __init no_initrd(char *str)
 
 __setup("noinitrd", no_initrd);
 
-static int init_linuxrc(struct subprocess_info *info, struct cred *new)
+static int __init do_linuxrc(void *_shell)
 {
-	sys_unshare(CLONE_FS | CLONE_FILES);
-	/* stdin/stdout/stderr for /linuxrc */
-	sys_open("/dev/console", O_RDWR, 0);
-	sys_dup(0);
-	sys_dup(0);
-	/* move initrd over / and chdir/chroot in initrd root */
-	sys_chdir("/root");
-	sys_mount(".", "/", NULL, MS_MOVE, NULL);
-	sys_chroot(".");
+	static const char *argv[] = { "linuxrc", NULL, };
+	extern const char *envp_init[];
+	const char *shell = _shell;
+
+	sys_close(old_fd);sys_close(root_fd);
 	sys_setsid();
-	return 0;
+	return sys_execve(shell, argv, envp_init);
 }
 
 static void __init handle_initrd(void)
 {
-	struct subprocess_info *info;
-	static char *argv[] = { "linuxrc", NULL, };
-	extern char *envp_init[];
 	int error;
+	int pid;
 
 	real_root_dev = new_encode_dev(ROOT_DEV);
 	create_dev("/dev/root.old", Root_RAM0);
 	/* mount initrd on rootfs' /root */
 	mount_block_root("/dev/root.old", root_mountflags & ~MS_RDONLY);
 	sys_mkdir("/old", 0700);
-	sys_chdir("/old");
-
-	/* try loading default modules from initrd */
-	load_default_modules();
+	root_fd = sys_open("/", 0, 0);
+	old_fd = sys_open("/old", 0, 0);
+	/* move initrd over / and chdir/chroot in initrd root */
+	sys_chdir("/root");
+	sys_mount(".", "/", NULL, MS_MOVE, NULL);
+	sys_chroot(".");
 
 	/*
 	 * In case that a resume from disk is carried out by linuxrc or one of
@@ -71,25 +67,27 @@ static void __init handle_initrd(void)
 	 */
 	current->flags |= PF_FREEZER_SKIP;
 
-	info = call_usermodehelper_setup("/linuxrc", argv, envp_init,
-					 GFP_KERNEL, init_linuxrc, NULL, NULL);
-	if (!info)
-		return;
-	call_usermodehelper_exec(info, UMH_WAIT_PROC);
+	pid = kernel_thread(do_linuxrc, "/linuxrc", SIGCHLD);
+	if (pid > 0)
+		while (pid != sys_wait4(-1, NULL, 0, NULL))
+			yield();
 
 	current->flags &= ~PF_FREEZER_SKIP;
 
 	/* move initrd to rootfs' /old */
-	sys_mount("..", ".", NULL, MS_MOVE, NULL);
+	sys_fchdir(old_fd);
+	sys_mount("/", ".", NULL, MS_MOVE, NULL);
 	/* switch root and cwd back to / of rootfs */
-	sys_chroot("..");
+	sys_fchdir(root_fd);
+	sys_chroot(".");
+	sys_close(old_fd);
+	sys_close(root_fd);
 
 	if (new_decode_dev(real_root_dev) == Root_RAM0) {
 		sys_chdir("/old");
 		return;
 	}
 
-	sys_chdir("/");
 	ROOT_DEV = new_decode_dev(real_root_dev);
 	mount_root();
 

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-08-12 19:44 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming Andrew Savchenko
@ 2013-08-27  3:48 ` Andrew Savchenko
  2013-09-05 12:08   ` [Suspend-devel] " Pavel Machek
  2013-09-18 13:52   ` Al Viro
  0 siblings, 2 replies; 14+ messages in thread
From: Andrew Savchenko @ 2013-08-27  3:48 UTC (permalink / raw)
  To: linux-kernel; +Cc: linux-pm, viro, suspend-devel

[-- Attachment #1: Type: text/plain, Size: 3183 bytes --]

Hello,

On Mon, 12 Aug 2013 23:44:15 +0400 Andrew Savchenko wrote:
> after a kernel update from 3.5.7 to the latest stable I found that
> user-space resume (from suspend-1.0 aka uswsusp) no longer works.
> Kernel-space suspend and resume work fine (e.g. echo disk
> > /sys/power/state), problem is with user-space support. (I need
> user-space version because it supports image encryption.)
> 
> After resume (essentially linuxrc) application loads image it fails
> to apply it:
> 
> ========================================================
> Processes could not be frozen, cannot continue resuming.
> Error 11: Resource temporarily unavailable
> 
> You can now boot the system and lose the saved state
> or reboot and try again.
> 
> [Notice that if you decide to reboot, you MUST NOT mount
> any filesystems before a successful resume.
> Resuming after some filesystems have been mounted
> will badly damage these filesystems.]
> 
> Do you want to continue booting (Y/n)?
> ========================================================
> 
> Error code wasn't originally showed, I added it to suspend tool to
> aid debugging. Essentially freeze ioctl on /dev/snapshot fails with
> this error.
> 
> I bisected a commit which introduces this bug:
> 
> ========================================================
> commit ba4df2808a86f8b103c4db0b8807649383e9bd13 
> Author: Al Viro <viro@zeniv.linux.org.uk> 
> Date:   Tue Oct 2 15:29:10 2012 -0400 
> 
>     don't bother with kernel_thread/kernel_execve for launching
> linuxrc 
>     exec_usermodehelper_fns() will do just fine... 
>     
>     Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> 
> ========================================================
> 
> In fact this commit induced/triggered at least two bugs: the first one
> I'm facing now and the second one was fixed in commit
> f0de17c0babe7f29381892def6b37e9181a53410:
> make sure that /linuxrc has std{in,out,err}.
> 
> As a temporarily workaround for this issue I reverted all changes for
> init/do_mounts_initrd.c up to the latest working commit
> cb450766bcafc7bd7d40e9a5a0050745e8c68b3e considering the kernel API
> changes (kernel_execve -> sys_execve). See linuxrc-workaround.patch.
> I understand this isn't a proper solution, I just want to show what
> code works for me.
> 
> I also found an interesting LKML discussion about s2disk and freezer
> issue: http://www.spinics.net/lists/linux-nfs/msg38160.html
> Maybe it is related to this bug, but patch proposed there doesn't in
> my case.
> 
> Kernel config which fails with
> ba4df2808a86f8b103c4db0b8807649383e9bd13 and works with
> f0de17c0babe7f29381892def6b37e9181a53410 is also attached.
> 
> As this issue maybe hardware related, the system is 32-bit EEE PC
> 1000H with Atom N270, 2GB RAM, 750 GB SATA drive.
> 
> Additional (but probably useless) information on this bug may be found
> here: https://forums.gentoo.org/viewtopic-p-7371120.html

This bug is still here with 3.11-rc7 and 3.10.9.
I opened a kernel bug 60802 for this issue:
https://bugzilla.kernel.org/show_bug.cgi?id=60802

Any ideas?

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-08-27  3:48 ` [BUG] " Andrew Savchenko
@ 2013-09-05 12:08   ` Pavel Machek
  2013-09-05 12:23     ` Rafael J. Wysocki
  2013-09-18 13:52   ` Al Viro
  1 sibling, 1 reply; 14+ messages in thread
From: Pavel Machek @ 2013-09-05 12:08 UTC (permalink / raw)
  To: Andrew Savchenko, Rafael J. Wysocki
  Cc: linux-kernel, suspend-devel, viro, linux-pm

Hi!

Rafael, Al: apparently we have a regression caused by
ba4df2808a86f8b103c4db0b8807649383e9bd13 .

> > after a kernel update from 3.5.7 to the latest stable I found that
> > user-space resume (from suspend-1.0 aka uswsusp) no longer works.
> > Kernel-space suspend and resume work fine (e.g. echo disk
> > > /sys/power/state), problem is with user-space support. (I need
> > user-space version because it supports image encryption.)
> > 
> > After resume (essentially linuxrc) application loads image it fails
> > to apply it:
> > 
> > ========================================================
> > Processes could not be frozen, cannot continue resuming.
> > Error 11: Resource temporarily unavailable
> > 
> > You can now boot the system and lose the saved state
> > or reboot and try again.
> > 
> > [Notice that if you decide to reboot, you MUST NOT mount
> > any filesystems before a successful resume.
> > Resuming after some filesystems have been mounted
> > will badly damage these filesystems.]
> > 
> > Do you want to continue booting (Y/n)?
> > ========================================================
> > 
> > Error code wasn't originally showed, I added it to suspend tool to
> > aid debugging. Essentially freeze ioctl on /dev/snapshot fails with
> > this error.
> > 
> > I bisected a commit which introduces this bug:
> > 
> > ========================================================
> > commit ba4df2808a86f8b103c4db0b8807649383e9bd13 
> > Author: Al Viro <viro@zeniv.linux.org.uk> 
> > Date:   Tue Oct 2 15:29:10 2012 -0400 
> > 
> >     don't bother with kernel_thread/kernel_execve for launching
> > linuxrc 
> >     exec_usermodehelper_fns() will do just fine... 
> >     
> >     Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> 
> > ========================================================
> > 
> > In fact this commit induced/triggered at least two bugs: the first one
> > I'm facing now and the second one was fixed in commit
> > f0de17c0babe7f29381892def6b37e9181a53410:
> > make sure that /linuxrc has std{in,out,err}.
> > 
> > As a temporarily workaround for this issue I reverted all changes for
> > init/do_mounts_initrd.c up to the latest working commit
> > cb450766bcafc7bd7d40e9a5a0050745e8c68b3e considering the kernel API
> > changes (kernel_execve -> sys_execve). See linuxrc-workaround.patch.
> > I understand this isn't a proper solution, I just want to show what
> > code works for me.
> > 
> > I also found an interesting LKML discussion about s2disk and freezer
> > issue: http://www.spinics.net/lists/linux-nfs/msg38160.html
> > Maybe it is related to this bug, but patch proposed there doesn't in
> > my case.
> > 
> > Kernel config which fails with
> > ba4df2808a86f8b103c4db0b8807649383e9bd13 and works with
> > f0de17c0babe7f29381892def6b37e9181a53410 is also attached.
> > 
> > As this issue maybe hardware related, the system is 32-bit EEE PC
> > 1000H with Atom N270, 2GB RAM, 750 GB SATA drive.
> > 
> > Additional (but probably useless) information on this bug may be found
> > here: https://forums.gentoo.org/viewtopic-p-7371120.html
> 
> This bug is still here with 3.11-rc7 and 3.10.9.
> I opened a kernel bug 60802 for this issue:
> https://bugzilla.kernel.org/show_bug.cgi?id=60802
> 
> Any ideas?
> 
> Best regards,
> Andrew Savchenko



> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and 
> AppDynamics. Performance Central is your source for news, insights, 
> analysis and resources for efficient Application Performance Management. 
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk

> _______________________________________________
> Suspend-devel mailing list
> Suspend-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/suspend-devel


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-05 12:08   ` [Suspend-devel] " Pavel Machek
@ 2013-09-05 12:23     ` Rafael J. Wysocki
  2013-09-12  8:32       ` Andrew Savchenko
  0 siblings, 1 reply; 14+ messages in thread
From: Rafael J. Wysocki @ 2013-09-05 12:23 UTC (permalink / raw)
  To: Pavel Machek
  Cc: Andrew Savchenko, linux-kernel, suspend-devel, viro, linux-pm

On Thursday, September 05, 2013 02:08:11 PM Pavel Machek wrote:
> Hi!
> 
> Rafael, Al: apparently we have a regression caused by
> ba4df2808a86f8b103c4db0b8807649383e9bd13 .

I noticed that, but I'm not sure how to deal with it.

Also, s2disk still works on my test machines, so that seems to be
specific to this particular configuration.

> > > after a kernel update from 3.5.7 to the latest stable I found that
> > > user-space resume (from suspend-1.0 aka uswsusp) no longer works.
> > > Kernel-space suspend and resume work fine (e.g. echo disk
> > > > /sys/power/state), problem is with user-space support. (I need
> > > user-space version because it supports image encryption.)
> > > 
> > > After resume (essentially linuxrc) application loads image it fails
> > > to apply it:
> > > 
> > > ========================================================
> > > Processes could not be frozen, cannot continue resuming.
> > > Error 11: Resource temporarily unavailable
> > > 
> > > You can now boot the system and lose the saved state
> > > or reboot and try again.
> > > 
> > > [Notice that if you decide to reboot, you MUST NOT mount
> > > any filesystems before a successful resume.
> > > Resuming after some filesystems have been mounted
> > > will badly damage these filesystems.]
> > > 
> > > Do you want to continue booting (Y/n)?
> > > ========================================================
> > > 
> > > Error code wasn't originally showed, I added it to suspend tool to
> > > aid debugging. Essentially freeze ioctl on /dev/snapshot fails with
> > > this error.
> > > 
> > > I bisected a commit which introduces this bug:
> > > 
> > > ========================================================
> > > commit ba4df2808a86f8b103c4db0b8807649383e9bd13 
> > > Author: Al Viro <viro@zeniv.linux.org.uk> 
> > > Date:   Tue Oct 2 15:29:10 2012 -0400 
> > > 
> > >     don't bother with kernel_thread/kernel_execve for launching
> > > linuxrc 
> > >     exec_usermodehelper_fns() will do just fine... 
> > >     
> > >     Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> 
> > > ========================================================
> > > 
> > > In fact this commit induced/triggered at least two bugs: the first one
> > > I'm facing now and the second one was fixed in commit
> > > f0de17c0babe7f29381892def6b37e9181a53410:
> > > make sure that /linuxrc has std{in,out,err}.
> > > 
> > > As a temporarily workaround for this issue I reverted all changes for
> > > init/do_mounts_initrd.c up to the latest working commit
> > > cb450766bcafc7bd7d40e9a5a0050745e8c68b3e considering the kernel API
> > > changes (kernel_execve -> sys_execve). See linuxrc-workaround.patch.
> > > I understand this isn't a proper solution, I just want to show what
> > > code works for me.
> > > 
> > > I also found an interesting LKML discussion about s2disk and freezer
> > > issue: http://www.spinics.net/lists/linux-nfs/msg38160.html
> > > Maybe it is related to this bug, but patch proposed there doesn't in
> > > my case.
> > > 
> > > Kernel config which fails with
> > > ba4df2808a86f8b103c4db0b8807649383e9bd13 and works with
> > > f0de17c0babe7f29381892def6b37e9181a53410 is also attached.
> > > 
> > > As this issue maybe hardware related, the system is 32-bit EEE PC
> > > 1000H with Atom N270, 2GB RAM, 750 GB SATA drive.
> > > 
> > > Additional (but probably useless) information on this bug may be found
> > > here: https://forums.gentoo.org/viewtopic-p-7371120.html
> > 
> > This bug is still here with 3.11-rc7 and 3.10.9.
> > I opened a kernel bug 60802 for this issue:
> > https://bugzilla.kernel.org/show_bug.cgi?id=60802
> > 
> > Any ideas?
> > 
> > Best regards,
> > Andrew Savchenko
> 
> 
> 
> > ------------------------------------------------------------------------------
> > Introducing Performance Central, a new site from SourceForge and 
> > AppDynamics. Performance Central is your source for news, insights, 
> > analysis and resources for efficient Application Performance Management. 
> > Visit us today!
> > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> 
> > _______________________________________________
> > Suspend-devel mailing list
> > Suspend-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/suspend-devel
> 
> 
> 
-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-05 12:23     ` Rafael J. Wysocki
@ 2013-09-12  8:32       ` Andrew Savchenko
  2013-09-18 13:02         ` Pavel Machek
  0 siblings, 1 reply; 14+ messages in thread
From: Andrew Savchenko @ 2013-09-12  8:32 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Pavel Machek, linux-kernel, suspend-devel, viro, linux-pm

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

Hello,

On Thu, 05 Sep 2013 14:23:25 +0200 Rafael J. Wysocki wrote:
> On Thursday, September 05, 2013 02:08:11 PM Pavel Machek wrote:
> > Hi!
> > 
> > Rafael, Al: apparently we have a regression caused by
> > ba4df2808a86f8b103c4db0b8807649383e9bd13 .
> 
> I noticed that, but I'm not sure how to deal with it.
> 
> Also, s2disk still works on my test machines, so that seems to be
> specific to this particular configuration.

Is there any way to debug this issue further than just bisecting
commit? Perhaps suspend ioctl should be traced to see why it fails,
but I don't know what exactly inside the kernel should be debugged
and how.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-12  8:32       ` Andrew Savchenko
@ 2013-09-18 13:02         ` Pavel Machek
  0 siblings, 0 replies; 14+ messages in thread
From: Pavel Machek @ 2013-09-18 13:02 UTC (permalink / raw)
  To: Andrew Savchenko
  Cc: Rafael J. Wysocki, linux-kernel, suspend-devel, viro, linux-pm

On Thu 2013-09-12 12:32:17, Andrew Savchenko wrote:
> Hello,
> 
> On Thu, 05 Sep 2013 14:23:25 +0200 Rafael J. Wysocki wrote:
> > On Thursday, September 05, 2013 02:08:11 PM Pavel Machek wrote:
> > > Hi!
> > > 
> > > Rafael, Al: apparently we have a regression caused by
> > > ba4df2808a86f8b103c4db0b8807649383e9bd13 .
> > 
> > I noticed that, but I'm not sure how to deal with it.
> > 
> > Also, s2disk still works on my test machines, so that seems to be
> > specific to this particular configuration.
> 
> Is there any way to debug this issue further than just bisecting
> commit? Perhaps suspend ioctl should be traced to see why it fails,
> but I don't know what exactly inside the kernel should be debugged
> and how.

If revert of ba4df2808a86f8b103c4db0b8807649383e9bd13 fixes it, then I
guess it should be just reverted...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-08-27  3:48 ` [BUG] " Andrew Savchenko
  2013-09-05 12:08   ` [Suspend-devel] " Pavel Machek
@ 2013-09-18 13:52   ` Al Viro
  2013-09-18 15:21     ` Al Viro
  2013-09-18 18:40     ` Andrew Savchenko
  1 sibling, 2 replies; 14+ messages in thread
From: Al Viro @ 2013-09-18 13:52 UTC (permalink / raw)
  To: Andrew Savchenko; +Cc: linux-kernel, linux-pm, suspend-devel

On Tue, Aug 27, 2013 at 07:48:43AM +0400, Andrew Savchenko wrote:
> > Additional (but probably useless) information on this bug may be found
> > here: https://forums.gentoo.org/viewtopic-p-7371120.html

Something's very fishy there:

[quote]
Digging into suspend-utils code shows that the following ioctl fails on      
"/dev/snapshot":                                                             
                                                                                
                                    Code:                                       
                          ioctl(dev, _IO(3, 1), 0);                          
[end quote]

but that's _not_ anything freeze-related - that's HDIO_GETGEO, and with zero
as last argument it will fail, no matter what.  With EFAULT, if nothing
else...

Which ioctl() it really is?  A bit further down you write "I modified suspend
code to see errno, so freeze on /dev/snapshot fails [with EAGAIN]", so you
have isolated the call in question.  Could you quote the actual code?

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-18 13:52   ` Al Viro
@ 2013-09-18 15:21     ` Al Viro
  2013-09-18 18:40     ` Andrew Savchenko
  1 sibling, 0 replies; 14+ messages in thread
From: Al Viro @ 2013-09-18 15:21 UTC (permalink / raw)
  To: Andrew Savchenko; +Cc: linux-kernel, linux-pm, suspend-devel

On Wed, Sep 18, 2013 at 02:52:39PM +0100, Al Viro wrote:
> On Tue, Aug 27, 2013 at 07:48:43AM +0400, Andrew Savchenko wrote:
> > > Additional (but probably useless) information on this bug may be found
> > > here: https://forums.gentoo.org/viewtopic-p-7371120.html
> 
> Something's very fishy there:
> 
> [quote]
> Digging into suspend-utils code shows that the following ioctl fails on      
> "/dev/snapshot":                                                             
>                                                                                 
>                                     Code:                                       
>                           ioctl(dev, _IO(3, 1), 0);                          
> [end quote]
> 
> but that's _not_ anything freeze-related - that's HDIO_GETGEO, and with zero
> as last argument it will fail, no matter what.  With EFAULT, if nothing
> else...
> 
> Which ioctl() it really is?  A bit further down you write "I modified suspend
> code to see errno, so freeze on /dev/snapshot fails [with EAGAIN]", so you
> have isolated the call in question.  Could you quote the actual code?

*scratches head*  _IO('3', 1), perhaps?  At least that would make sense in
such context...  Assuming that's the case, slap
	printk(KERN_INFO "freeze_process() => %d", error);
after the call of freeze_process() in kernel/power/user.c along with
	printk(KERN_INFO "__usermodehelper_disable() => %d", error);
and
	printk(KERN_INFO "try_to_freeze_tasks() => %d", error);
in kernel/power/process.c:freeze_process(), after the calls of
__usermodehelper_disable() and try_to_freeze_tasks() resp.

FWIW, I suspect that it's __usermodehelper_disable() - it does
        retval = wait_event_timeout(running_helpers_waitq,
                                        atomic_read(&running_helpers) == 0,
                                        RUNNING_HELPERS_TIMEOUT);
and returns -EAGAIN on timeout.  I'm not familiar with swsusp code, but
it smells like we end up waiting for linuxrc itself to finish.

Pavel, any suggestions?  If SNAPSHOT_FREEZE really wants everything run
via usermodehelper gone for some reason, what makes /linuxrc different
from e.g /sbin/modprobe?

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-18 13:52   ` Al Viro
  2013-09-18 15:21     ` Al Viro
@ 2013-09-18 18:40     ` Andrew Savchenko
  2013-09-18 19:16       ` Al Viro
  1 sibling, 1 reply; 14+ messages in thread
From: Andrew Savchenko @ 2013-09-18 18:40 UTC (permalink / raw)
  To: Al Viro; +Cc: linux-kernel, linux-pm, suspend-devel

[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]

Hello,

On Wed, 18 Sep 2013 14:52:39 +0100 Al Viro wrote:
> On Tue, Aug 27, 2013 at 07:48:43AM +0400, Andrew Savchenko wrote:
> > > Additional (but probably useless) information on this bug may be found
> > > here: https://forums.gentoo.org/viewtopic-p-7371120.html
> 
> Something's very fishy there:
> 
> [quote]
> Digging into suspend-utils code shows that the following ioctl fails on      
> "/dev/snapshot":                                                             
>                                                                                 
>                                     Code:                                       
>                           ioctl(dev, _IO(3, 1), 0);                          
> [end quote]
> 
> but that's _not_ anything freeze-related - that's HDIO_GETGEO, and with zero
> as last argument it will fail, no matter what.  With EFAULT, if nothing
> else...
> 
> Which ioctl() it really is?  A bit further down you write "I modified suspend
> code to see errno, so freeze on /dev/snapshot fails [with EAGAIN]", so you
> have isolated the call in question.  Could you quote the actual code?

Actual code is from suspend-utils tree, swsusp.h:

static inline int freeze(int dev)
{
        return ioctl(dev, SNAPSHOT_FREEZE, 0);
}

And from suspend_ioctls.h:
#define SNAPSHOT_IOC_MAGIC      '3'
#define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)

My mistake, should be '3' instead of 3.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-18 18:40     ` Andrew Savchenko
@ 2013-09-18 19:16       ` Al Viro
  2013-09-18 22:13         ` Andrew Savchenko
  2013-09-24  0:21         ` [Suspend-devel] " Pavel Machek
  0 siblings, 2 replies; 14+ messages in thread
From: Al Viro @ 2013-09-18 19:16 UTC (permalink / raw)
  To: Andrew Savchenko; +Cc: linux-kernel, linux-pm, suspend-devel

On Wed, Sep 18, 2013 at 10:40:32PM +0400, Andrew Savchenko wrote:

> And from suspend_ioctls.h:
> #define SNAPSHOT_IOC_MAGIC      '3'
> #define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)
> 
> My mistake, should be '3' instead of 3.

OK...  The thing to test, then, is what does __usermodehelper_disable()
return to freeze_processes().  If that's where this -EAGAIN comes from,
we at least have a plausible theory re what's going on.

freeze_processes() uses __usermodehelper_disable() to stop any new userland
processes spawned by UMH (modprobe, etc.) and waits for ones it might be
waiting for to complete.  Then it does try_to_freeze_tasks(), which
freezes remaining userland, carefully skipping the current thread.
However, it misses the possibility that current thread might have been
spawned by something that had been launched by UMH, with UMH waiting
for it.  Which is the case of everything spawned by linuxrc.

I'd try something like diff below, but I'm *NOT* familiar with swsusp at
all; it's not for mainline until ACKed by swsusp folks.

diff --git a/kernel/kmod.c b/kernel/kmod.c
index fb32636..d968882 100644
--- a/kernel/kmod.c
+++ b/kernel/kmod.c
@@ -571,7 +571,8 @@ int call_usermodehelper_exec(struct subprocess_info *sub_info, int wait)
 	DECLARE_COMPLETION_ONSTACK(done);
 	int retval = 0;
 
-	helper_lock();
+	if (!(current->flags & PF_FREEZER_SKIP))
+		helper_lock();
 	if (!khelper_wq || usermodehelper_disabled) {
 		retval = -EBUSY;
 		goto out;
@@ -611,7 +612,8 @@ wait_done:
 out:
 	call_usermodehelper_freeinfo(sub_info);
 unlock:
-	helper_unlock();
+	if (!(current->flags & PF_FREEZER_SKIP))
+		helper_unlock();
 	return retval;
 }
 EXPORT_SYMBOL(call_usermodehelper_exec);

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-18 19:16       ` Al Viro
@ 2013-09-18 22:13         ` Andrew Savchenko
  2013-09-24  0:21         ` [Suspend-devel] " Pavel Machek
  1 sibling, 0 replies; 14+ messages in thread
From: Andrew Savchenko @ 2013-09-18 22:13 UTC (permalink / raw)
  To: Al Viro; +Cc: linux-kernel, linux-pm, suspend-devel

[-- Attachment #1: Type: text/plain, Size: 2002 bytes --]

On Wed, 18 Sep 2013 20:16:07 +0100 Al Viro wrote:
> On Wed, Sep 18, 2013 at 10:40:32PM +0400, Andrew Savchenko wrote:
> 
> > And from suspend_ioctls.h:
> > #define SNAPSHOT_IOC_MAGIC      '3'
> > #define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)
> > 
> > My mistake, should be '3' instead of 3.
> 
> OK...  The thing to test, then, is what does __usermodehelper_disable()
> return to freeze_processes().  If that's where this -EAGAIN comes from,
> we at least have a plausible theory re what's going on.
> 
> freeze_processes() uses __usermodehelper_disable() to stop any new userland
> processes spawned by UMH (modprobe, etc.) and waits for ones it might be
> waiting for to complete.  Then it does try_to_freeze_tasks(), which
> freezes remaining userland, carefully skipping the current thread.
> However, it misses the possibility that current thread might have been
> spawned by something that had been launched by UMH, with UMH waiting
> for it.  Which is the case of everything spawned by linuxrc.
> 
> I'd try something like diff below, but I'm *NOT* familiar with swsusp at
> all; it's not for mainline until ACKed by swsusp folks.
> 
> diff --git a/kernel/kmod.c b/kernel/kmod.c
> index fb32636..d968882 100644
> --- a/kernel/kmod.c
> +++ b/kernel/kmod.c
> @@ -571,7 +571,8 @@ int call_usermodehelper_exec(struct subprocess_info *sub_info, int wait)
>  	DECLARE_COMPLETION_ONSTACK(done);
>  	int retval = 0;
>  
> -	helper_lock();
> +	if (!(current->flags & PF_FREEZER_SKIP))
> +		helper_lock();
>  	if (!khelper_wq || usermodehelper_disabled) {
>  		retval = -EBUSY;
>  		goto out;
> @@ -611,7 +612,8 @@ wait_done:
>  out:
>  	call_usermodehelper_freeinfo(sub_info);
>  unlock:
> -	helper_unlock();
> +	if (!(current->flags & PF_FREEZER_SKIP))
> +		helper_unlock();
>  	return retval;
>  }
>  EXPORT_SYMBOL(call_usermodehelper_exec);

With this patch and 3.11.1 kernel resume works fine.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-18 19:16       ` Al Viro
  2013-09-18 22:13         ` Andrew Savchenko
@ 2013-09-24  0:21         ` Pavel Machek
  2013-10-17 21:35           ` Rafael J. Wysocki
  1 sibling, 1 reply; 14+ messages in thread
From: Pavel Machek @ 2013-09-24  0:21 UTC (permalink / raw)
  To: Al Viro, Rafael J. Wysocki
  Cc: Andrew Savchenko, suspend-devel, linux-kernel, linux-pm

Hi!

> > And from suspend_ioctls.h:
> > #define SNAPSHOT_IOC_MAGIC      '3'
> > #define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)
> > 
> > My mistake, should be '3' instead of 3.
> 
> OK...  The thing to test, then, is what does __usermodehelper_disable()
> return to freeze_processes().  If that's where this -EAGAIN comes from,
> we at least have a plausible theory re what's going on.
> 
> freeze_processes() uses __usermodehelper_disable() to stop any new userland
> processes spawned by UMH (modprobe, etc.) and waits for ones it might be
> waiting for to complete.  Then it does try_to_freeze_tasks(), which
> freezes remaining userland, carefully skipping the current thread.
> However, it misses the possibility that current thread might have been
> spawned by something that had been launched by UMH, with UMH waiting
> for it.  Which is the case of everything spawned by linuxrc.
> 
> I'd try something like diff below, but I'm *NOT* familiar with swsusp at
> all; it's not for mainline until ACKed by swsusp folks.
> 
> diff --git a/kernel/kmod.c b/kernel/kmod.c
> index fb32636..d968882 100644
> --- a/kernel/kmod.c
> +++ b/kernel/kmod.c
> @@ -571,7 +571,8 @@ int call_usermodehelper_exec(struct subprocess_info *sub_info, int wait)
>  	DECLARE_COMPLETION_ONSTACK(done);
>  	int retval = 0;
>  
> -	helper_lock();
> +	if (!(current->flags & PF_FREEZER_SKIP))
> +		helper_lock();
>  	if (!khelper_wq || usermodehelper_disabled) {
>  		retval = -EBUSY;
>  		goto out;
> @@ -611,7 +612,8 @@ wait_done:
>  out:
>  	call_usermodehelper_freeinfo(sub_info);
>  unlock:
> -	helper_unlock();
> +	if (!(current->flags & PF_FREEZER_SKIP))
> +		helper_unlock();
>  	return retval;
>  }
>  EXPORT_SYMBOL(call_usermodehelper_exec);

PF_FREEZER_SKIP flag is manipulated at about 1000 places, so I'm not
sure this will nest correctly. They seem to be in form of 

|= FREEZER_SKIP
schedule()
&= ~FREEZER_SKIP

so this should be safe, but...

								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-09-24  0:21         ` [Suspend-devel] " Pavel Machek
@ 2013-10-17 21:35           ` Rafael J. Wysocki
  2016-05-22  8:47             ` Andrew Savchenko
  0 siblings, 1 reply; 14+ messages in thread
From: Rafael J. Wysocki @ 2013-10-17 21:35 UTC (permalink / raw)
  To: Pavel Machek, Al Viro
  Cc: Andrew Savchenko, suspend-devel, linux-kernel, linux-pm

Sorry for the huge delay.

On Tuesday, September 24, 2013 02:21:11 AM Pavel Machek wrote:
> Hi!
> 
> > > And from suspend_ioctls.h:
> > > #define SNAPSHOT_IOC_MAGIC      '3'
> > > #define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)
> > > 
> > > My mistake, should be '3' instead of 3.
> > 
> > OK...  The thing to test, then, is what does __usermodehelper_disable()
> > return to freeze_processes().  If that's where this -EAGAIN comes from,
> > we at least have a plausible theory re what's going on.
> > 
> > freeze_processes() uses __usermodehelper_disable() to stop any new userland
> > processes spawned by UMH (modprobe, etc.) and waits for ones it might be
> > waiting for to complete.  Then it does try_to_freeze_tasks(), which
> > freezes remaining userland, carefully skipping the current thread.
> > However, it misses the possibility that current thread might have been
> > spawned by something that had been launched by UMH, with UMH waiting
> > for it.  Which is the case of everything spawned by linuxrc.
> > 
> > I'd try something like diff below, but I'm *NOT* familiar with swsusp at
> > all; it's not for mainline until ACKed by swsusp folks.
> > 
> > diff --git a/kernel/kmod.c b/kernel/kmod.c
> > index fb32636..d968882 100644
> > --- a/kernel/kmod.c
> > +++ b/kernel/kmod.c
> > @@ -571,7 +571,8 @@ int call_usermodehelper_exec(struct subprocess_info *sub_info, int wait)
> >  	DECLARE_COMPLETION_ONSTACK(done);
> >  	int retval = 0;
> >  
> > -	helper_lock();
> > +	if (!(current->flags & PF_FREEZER_SKIP))
> > +		helper_lock();
> >  	if (!khelper_wq || usermodehelper_disabled) {
> >  		retval = -EBUSY;
> >  		goto out;
> > @@ -611,7 +612,8 @@ wait_done:
> >  out:
> >  	call_usermodehelper_freeinfo(sub_info);
> >  unlock:
> > -	helper_unlock();
> > +	if (!(current->flags & PF_FREEZER_SKIP))
> > +		helper_unlock();
> >  	return retval;
> >  }
> >  EXPORT_SYMBOL(call_usermodehelper_exec);
> 
> PF_FREEZER_SKIP flag is manipulated at about 1000 places, so I'm not
> sure this will nest correctly.

This is not exactly correct unless 1000 is about 50.  And none of them leads to
call_usermodehelper_exec() as far as I can say.

> They seem to be in form of 
> 
> |= FREEZER_SKIP
> schedule()
> &= ~FREEZER_SKIP
> 
> so this should be safe, but...

I think the patch is correct, so

Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Thanks!

-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Suspend-devel] [BUG] 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming
  2013-10-17 21:35           ` Rafael J. Wysocki
@ 2016-05-22  8:47             ` Andrew Savchenko
  0 siblings, 0 replies; 14+ messages in thread
From: Andrew Savchenko @ 2016-05-22  8:47 UTC (permalink / raw)
  To: Rafael J. Wysocki, Pavel Machek, Al Viro
  Cc: suspend-devel, linux-kernel, linux-pm


[-- Attachment #1.1: Type: text/plain, Size: 2892 bytes --]

Hi,

On Thu, 17 Oct 2013 23:35:12 +0200 Rafael J. Wysocki wrote:
> Sorry for the huge delay.
> 
> On Tuesday, September 24, 2013 02:21:11 AM Pavel Machek wrote:
> > Hi!
> > 
> > > > And from suspend_ioctls.h:
> > > > #define SNAPSHOT_IOC_MAGIC      '3'
> > > > #define SNAPSHOT_FREEZE                 _IO(SNAPSHOT_IOC_MAGIC, 1)
> > > > 
> > > > My mistake, should be '3' instead of 3.
> > > 
> > > OK...  The thing to test, then, is what does __usermodehelper_disable()
> > > return to freeze_processes().  If that's where this -EAGAIN comes from,
> > > we at least have a plausible theory re what's going on.
> > > 
> > > freeze_processes() uses __usermodehelper_disable() to stop any new userland
> > > processes spawned by UMH (modprobe, etc.) and waits for ones it might be
> > > waiting for to complete.  Then it does try_to_freeze_tasks(), which
> > > freezes remaining userland, carefully skipping the current thread.
> > > However, it misses the possibility that current thread might have been
> > > spawned by something that had been launched by UMH, with UMH waiting
> > > for it.  Which is the case of everything spawned by linuxrc.
> > > 
> > > I'd try something like diff below, but I'm *NOT* familiar with swsusp at
> > > all; it's not for mainline until ACKed by swsusp folks.
> > > 
> > > diff --git a/kernel/kmod.c b/kernel/kmod.c
> > > index fb32636..d968882 100644
> > > --- a/kernel/kmod.c
> > > +++ b/kernel/kmod.c
> > > @@ -571,7 +571,8 @@ int call_usermodehelper_exec(struct subprocess_info *sub_info, int wait)
> > >  	DECLARE_COMPLETION_ONSTACK(done);
> > >  	int retval = 0;
> > >  
> > > -	helper_lock();
> > > +	if (!(current->flags & PF_FREEZER_SKIP))
> > > +		helper_lock();
> > >  	if (!khelper_wq || usermodehelper_disabled) {
> > >  		retval = -EBUSY;
> > >  		goto out;
> > > @@ -611,7 +612,8 @@ wait_done:
> > >  out:
> > >  	call_usermodehelper_freeinfo(sub_info);
> > >  unlock:
> > > -	helper_unlock();
> > > +	if (!(current->flags & PF_FREEZER_SKIP))
> > > +		helper_unlock();
> > >  	return retval;
> > >  }
> > >  EXPORT_SYMBOL(call_usermodehelper_exec);
> > 
> > PF_FREEZER_SKIP flag is manipulated at about 1000 places, so I'm not
> > sure this will nest correctly.
> 
> This is not exactly correct unless 1000 is about 50.  And none of them leads to
> call_usermodehelper_exec() as far as I can say.
> 
> > They seem to be in form of 
> > 
> > |= FREEZER_SKIP
> > schedule()
> > &= ~FREEZER_SKIP
> > 
> > so this should be safe, but...
> 
> I think the patch is correct, so
> 
> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> 
> Thanks!
 
The problem is still here with 4.6.0 kernel. Patch is updated to
reflect code base changes while retaining original logics. Works
fine for me here (4.6.0, x86, Atom N270, EeePC 1000H).

Best regards,
Andrew Savchenko

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: linuxrc-usermodhelper_disable-4.6.0.patch --]
[-- Type: text/x-diff; name="linuxrc-usermodhelper_disable-4.6.0.patch", Size: 610 bytes --]

--- linux-4.6.0/kernel/kmod.c.orig	2016-05-16 01:43:13.000000000 +0300
+++ linux-4.6.0/kernel/kmod.c	2016-05-22 09:49:12.654159691 +0300
@@ -561,7 +561,8 @@
 		call_usermodehelper_freeinfo(sub_info);
 		return -EINVAL;
 	}
-	helper_lock();
+	if (!(current->flags & PF_FREEZER_SKIP))
+		helper_lock();
 	if (usermodehelper_disabled) {
 		retval = -EBUSY;
 		goto out;
@@ -595,7 +596,8 @@
 out:
 	call_usermodehelper_freeinfo(sub_info);
 unlock:
-	helper_unlock();
+	if (!(current->flags & PF_FREEZER_SKIP))
+		helper_unlock();
 	return retval;
 }
 EXPORT_SYMBOL(call_usermodehelper_exec);

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2016-05-22  8:47 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-12 19:44 3.7-rc regression bisected: s2disk fails to resume image: Processes could not be frozen, cannot continue resuming Andrew Savchenko
2013-08-27  3:48 ` [BUG] " Andrew Savchenko
2013-09-05 12:08   ` [Suspend-devel] " Pavel Machek
2013-09-05 12:23     ` Rafael J. Wysocki
2013-09-12  8:32       ` Andrew Savchenko
2013-09-18 13:02         ` Pavel Machek
2013-09-18 13:52   ` Al Viro
2013-09-18 15:21     ` Al Viro
2013-09-18 18:40     ` Andrew Savchenko
2013-09-18 19:16       ` Al Viro
2013-09-18 22:13         ` Andrew Savchenko
2013-09-24  0:21         ` [Suspend-devel] " Pavel Machek
2013-10-17 21:35           ` Rafael J. Wysocki
2016-05-22  8:47             ` Andrew Savchenko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).