[ 00/23] [Suspend2] Freezer Upgrade Patches

[ 00/23] [Suspend2] Freezer Upgrade Patches

[Nigel Cunningham]

Hi everyone.

This set of patches represents the freezer upgrade patches from Suspend2.

The key features of this changeset are:

- Use of Christoph Lameter's todo list notifiers, which help with SMP
  cleanness.
- Splitting the freezing of kernel and userspace processes. Freezing
  currently suffers from a race because userspace processes can be
  submitting work for kernel threads, thereby stopping them from
  responding to freeze messages in a timely manner. The freezer can
  thus give up when it doesn't really need to. (This is not normally
  a problem only because load is not usually high).
- The use of bdev freezing to ensure filesystems are properly frozen,
  thereby increasing the integrity of on-disk data in the case where
  a resume doesn't occur. This is also helpful in the case of Suspend2,
  where we don't atomically copy all memory, instead writing LRU pages
  separately.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

--
Nigel Cunningham   nigel at suspend2 dot net   http://suspend2.net

[ 01/23] [Suspend2] Make workqueues freezeable.

[Nigel Cunningham]

Prior to this patch, kernel threads and workqueues are unconditionally
unfreezeable. This patch reverses that behaviour, making the default
for kernel processes to be frozen. New variations of the routines for
starting kernel threads and workqueues (containing _nofreeze_) allow
threads that need to run during suspend to be made nofreeze again.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 block/ll_rw_blk.c             |    2 -
 drivers/acpi/osl.c            |    2 -
 drivers/char/hvc_console.c    |    2 -
 drivers/char/hvcs.c           |    2 -
 drivers/input/serio/serio.c   |    2 -
 drivers/md/dm-crypt.c         |    2 -
 drivers/scsi/hosts.c          |    2 -
 drivers/scsi/lpfc/lpfc_init.c |    2 -
 drivers/usb/net/pegasus.c     |    2 -
 include/linux/kthread.h       |   27 ++++++++----
 include/linux/workqueue.h     |    9 +++-
 kernel/audit.c                |    3 +
 kernel/kthread.c              |   94 ++++++++++++++++++++++++++++++++++++++---
 kernel/sched.c                |    1 
 kernel/softirq.c              |    3 -
 kernel/workqueue.c            |   37 +++++++++++-----
 16 files changed, 149 insertions(+), 43 deletions(-)

diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 8e27d0a..e6c2c5d 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -3435,7 +3435,7 @@ int __init blk_dev_init(void)
 {
 	int i;
 
-	kblockd_workqueue = create_workqueue("kblockd");
+	kblockd_workqueue = create_nofreeze_workqueue("kblockd");
 	if (!kblockd_workqueue)
 		panic("Failed to create kblockd\n");
 
diff --git a/drivers/acpi/osl.c b/drivers/acpi/osl.c
index 20c9a37..1b3ee90 100644
--- a/drivers/acpi/osl.c
+++ b/drivers/acpi/osl.c
@@ -91,7 +91,7 @@ acpi_status acpi_os_initialize1(void)
 		       "Access to PCI configuration space unavailable\n");
 		return AE_NULL_ENTRY;
 	}
-	kacpid_wq = create_singlethread_workqueue("kacpid");
+	kacpid_wq = create_nofreeze_singlethread_workqueue("kacpid");
 	BUG_ON(!kacpid_wq);
 
 	return AE_OK;
diff --git a/drivers/char/hvc_console.c b/drivers/char/hvc_console.c
index 1994a92..44b5cff 100644
--- a/drivers/char/hvc_console.c
+++ b/drivers/char/hvc_console.c
@@ -839,7 +839,7 @@ int __init hvc_init(void)
 
 	/* Always start the kthread because there can be hotplug vty adapters
 	 * added later. */
-	hvc_task = kthread_run(khvcd, NULL, "khvcd");
+	hvc_task = kthread_nofreeze_run(khvcd, NULL, "khvcd");
 	if (IS_ERR(hvc_task)) {
 		panic("Couldn't create kthread for console.\n");
 		put_tty_driver(hvc_driver);
diff --git a/drivers/char/hvcs.c b/drivers/char/hvcs.c
index 831eb4e..4302c36 100644
--- a/drivers/char/hvcs.c
+++ b/drivers/char/hvcs.c
@@ -1404,7 +1404,7 @@ static int __init hvcs_module_init(void)
 		return -ENOMEM;
 	}
 
-	hvcs_task = kthread_run(khvcsd, NULL, "khvcsd");
+	hvcs_task = kthread_nofreeze_run(khvcsd, NULL, "khvcsd");
 	if (IS_ERR(hvcs_task)) {
 		printk(KERN_ERR "HVCS: khvcsd creation failed.  Driver not loaded.\n");
 		kfree(hvcs_pi_buff);
diff --git a/drivers/input/serio/serio.c b/drivers/input/serio/serio.c
index 2f76813..edd7254 100644
--- a/drivers/input/serio/serio.c
+++ b/drivers/input/serio/serio.c
@@ -901,7 +901,7 @@ irqreturn_t serio_interrupt(struct serio
 
 static int __init serio_init(void)
 {
-	serio_task = kthread_run(serio_thread, NULL, "kseriod");
+	serio_task = kthread_nofreeze_run(serio_thread, NULL, "kseriod");
 	if (IS_ERR(serio_task)) {
 		printk(KERN_ERR "serio: Failed to start kseriod\n");
 		return PTR_ERR(serio_task);
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index e7a650f..634c82b 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -928,7 +928,7 @@ static int __init dm_crypt_init(void)
 	if (!_crypt_io_pool)
 		return -ENOMEM;
 
-	_kcryptd_workqueue = create_workqueue("kcryptd");
+	_kcryptd_workqueue = create_nofreeze_workqueue("kcryptd");
 	if (!_kcryptd_workqueue) {
 		r = -ENOMEM;
 		DMERR(PFX "couldn't create kcryptd");
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index 5881079..2f245f0 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -227,7 +227,7 @@ int scsi_add_host(struct Scsi_Host *shos
 	if (shost->transportt->create_work_queue) {
 		snprintf(shost->work_q_name, KOBJ_NAME_LEN, "scsi_wq_%d",
 			shost->host_no);
-		shost->work_q = create_singlethread_workqueue(
+		shost->work_q = create_nofreeze_singlethread_workqueue(
 					shost->work_q_name);
 		if (!shost->work_q)
 			goto out_free_shost_data;
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index b7a603a..02734fc 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -1521,7 +1521,7 @@ lpfc_pci_probe_one(struct pci_dev *pdev,
 	phba->work_ha_mask |= (HA_RXMASK << (LPFC_ELS_RING * 4));
 
 	/* Startup the kernel thread for this host adapter. */
-	phba->worker_thread = kthread_run(lpfc_do_work, phba,
+	phba->worker_thread = kthread_nofreeze_run(lpfc_do_work, phba,
 				       "lpfc_worker_%d", phba->brd_no);
 	if (IS_ERR(phba->worker_thread)) {
 		error = PTR_ERR(phba->worker_thread);
diff --git a/drivers/usb/net/pegasus.c b/drivers/usb/net/pegasus.c
index 156a2f1..d6a1474 100644
--- a/drivers/usb/net/pegasus.c
+++ b/drivers/usb/net/pegasus.c
@@ -1451,7 +1451,7 @@ static int __init pegasus_init(void)
 	pr_info("%s: %s, " DRIVER_DESC "\n", driver_name, DRIVER_VERSION);
 	if (devid)
 		parse_id(devid);
-	pegasus_workqueue = create_singlethread_workqueue("pegasus");
+	pegasus_workqueue = create_nofreeze_singlethread_workqueue("pegasus");
 	if (!pegasus_workqueue)
 		return -ENOMEM;
 	return usb_register(&pegasus_driver);
diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index ebdd41f..965e836 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -23,10 +23,20 @@
  *
  * Returns a task_struct or ERR_PTR(-ENOMEM).
  */
+struct task_struct *__kthread_create(int (*threadfn)(void *data),
+				   void *data,
+				   unsigned long freezer_flags,
+				   const char namefmt[],
+				   va_list * args);
+
 struct task_struct *kthread_create(int (*threadfn)(void *data),
 				   void *data,
 				   const char namefmt[], ...);
 
+struct task_struct *kthread_nofreeze_create(int (*threadfn)(void *data),
+				   void *data,
+				   const char namefmt[], ...);
+
 /**
  * kthread_run: create and wake a thread.
  * @threadfn: the function to run until signal_pending(current).
@@ -35,14 +45,15 @@ struct task_struct *kthread_create(int (
  *
  * Description: Convenient wrapper for kthread_create() followed by
  * wake_up_process().  Returns the kthread, or ERR_PTR(-ENOMEM). */
-#define kthread_run(threadfn, data, namefmt, ...)			   \
-({									   \
-	struct task_struct *__k						   \
-		= kthread_create(threadfn, data, namefmt, ## __VA_ARGS__); \
-	if (!IS_ERR(__k))						   \
-		wake_up_process(__k);					   \
-	__k;								   \
-})
+
+extern struct task_struct * kthread_run(int (*threadfn)(void *data),
+					void *data,
+					const char namefmt[], ...);
+
+extern struct task_struct * kthread_nofreeze_run(int (*threadfn)(void *data),
+					void *data,
+					const char namefmt[], ...);
+
 
 /**
  * kthread_bind: bind a just-created kthread to a cpu.
diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index 86b1113..fb1b51e 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -51,9 +51,12 @@ struct work_struct {
 	} while (0)
 
 extern struct workqueue_struct *__create_workqueue(const char *name,
-						    int singlethread);
-#define create_workqueue(name) __create_workqueue((name), 0)
-#define create_singlethread_workqueue(name) __create_workqueue((name), 1)
+						    int singlethread,
+						    unsigned long freezer_flag);
+#define create_workqueue(name) __create_workqueue((name), 0, 0)
+#define create_nofreeze_workqueue(name) __create_workqueue((name), 0, PF_NOFREEZE)
+#define create_singlethread_workqueue(name) __create_workqueue((name), 1, 0)
+#define create_nofreeze_singlethread_workqueue(name) __create_workqueue((name), 1, PF_NOFREEZE)
 
 extern void destroy_workqueue(struct workqueue_struct *wq);
 
diff --git a/kernel/audit.c b/kernel/audit.c
index 0a813d2..0cd1b16 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -288,6 +288,9 @@ static int kauditd_thread(void *dummy)
 			}
 		} else {
 			DECLARE_WAITQUEUE(wait, current);
+			
+			try_to_freeze();
+
 			set_current_state(TASK_INTERRUPTIBLE);
 			add_wait_queue(&kauditd_wait, &wait);
 
diff --git a/kernel/kthread.c b/kernel/kthread.c
index e75950a..36f82da 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -25,6 +25,7 @@ struct kthread_create_info
 	/* Information passed to kthread() from keventd. */
 	int (*threadfn)(void *data);
 	void *data;
+	unsigned long freezer_flags;
 	struct completion started;
 
 	/* Result passed back to kthread_create() from keventd. */
@@ -86,6 +87,10 @@ static int kthread(void *_create)
 	/* By default we can run anywhere, unlike keventd. */
 	set_cpus_allowed(current, CPU_MASK_ALL);
 
+	/* Set our freezer flags */
+	current->flags &= ~PF_NOFREEZE;
+	current->flags |= (create->freezer_flags & PF_NOFREEZE);
+
 	/* OK, tell user we're spawned, wait for stop or wakeup */
 	__set_current_state(TASK_INTERRUPTIBLE);
 	complete(&create->started);
@@ -119,16 +124,18 @@ static void keventd_create_kthread(void 
 	complete(&create->done);
 }
 
-struct task_struct *kthread_create(int (*threadfn)(void *data),
+struct task_struct *__kthread_create(int (*threadfn)(void *data),
 				   void *data,
+				   unsigned long freezer_flags,
 				   const char namefmt[],
-				   ...)
+				   va_list * args)
 {
 	struct kthread_create_info create;
 	DECLARE_WORK(work, keventd_create_kthread, &create);
 
 	create.threadfn = threadfn;
 	create.data = data;
+	create.freezer_flags = freezer_flags;
 	init_completion(&create.started);
 	init_completion(&create.done);
 
@@ -141,18 +148,89 @@ struct task_struct *kthread_create(int (
 		queue_work(helper_wq, &work);
 		wait_for_completion(&create.done);
 	}
-	if (!IS_ERR(create.result)) {
-		va_list args;
-		va_start(args, namefmt);
+	if (!IS_ERR(create.result))
 		vsnprintf(create.result->comm, sizeof(create.result->comm),
-			  namefmt, args);
-		va_end(args);
-	}
+			  namefmt, *args);
 
 	return create.result;
 }
+
+struct task_struct *kthread_create(int (*threadfn)(void *data),
+				   void *data,
+				   const char namefmt[], ...)
+{
+	struct task_struct * result;
+
+	va_list args;
+	va_start(args, namefmt);
+	result = __kthread_create(threadfn, data, 0, namefmt, &args);
+	va_end(args);
+	return result;
+}
+
 EXPORT_SYMBOL(kthread_create);
 
+struct task_struct *kthread_nofreeze_create(int (*threadfn)(void *data),
+				   void *data,
+				   const char namefmt[], ...)
+{
+	struct task_struct * result;
+
+	va_list args;
+	va_start(args, namefmt);
+	result = __kthread_create(threadfn, data, PF_NOFREEZE, namefmt, &args);
+	va_end(args);
+	return result;
+}
+
+EXPORT_SYMBOL(kthread_nofreeze_create);
+
+/**
+ * kthread_run: create and wake a thread.
+ * @threadfn: the function to run until signal_pending(current).
+ * @data: data ptr for @threadfn.
+ * @namefmt: printf-style name for the thread.
+ *
+ * Description: Convenient wrapper for kthread_create() followed by
+ * wake_up_process().  Returns the kthread, or ERR_PTR(-ENOMEM).
+ **/
+struct task_struct * kthread_run(int (*threadfn)(void *data),
+		void *data,
+		const char namefmt[], ...)
+{
+	struct task_struct *__k;
+	va_list args;
+
+	va_start(args, namefmt);
+	__k = __kthread_create(threadfn, data, 0, namefmt, &args);
+	va_end(args);
+
+	if(!IS_ERR(__k))
+		wake_up_process(__k);
+
+	return __k;
+}
+
+EXPORT_SYMBOL(kthread_run);
+
+struct task_struct * kthread_nofreeze_run(int (*threadfn)(void *data),
+		void *data,
+		const char namefmt[], ...)
+{
+	struct task_struct *__k;
+	va_list args;
+
+	va_start(args, namefmt);
+	__k = __kthread_create(threadfn, data, PF_NOFREEZE, namefmt, &args);
+	va_end(args);
+
+	if(!IS_ERR(__k))
+		wake_up_process(__k);
+
+	return __k;
+}
+EXPORT_SYMBOL(kthread_nofreeze_run);
+
 void kthread_bind(struct task_struct *k, unsigned int cpu)
 {
 	BUG_ON(k->state != TASK_INTERRUPTIBLE);
diff --git a/kernel/sched.c b/kernel/sched.c
index 3ee2ae4..ad3c9ac 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4805,7 +4805,6 @@ static int migration_call(struct notifie
 		p = kthread_create(migration_thread, hcpu, "migration/%d",cpu);
 		if (IS_ERR(p))
 			return NOTIFY_BAD;
-		p->flags |= PF_NOFREEZE;
 		kthread_bind(p, cpu);
 		/* Must be high prio: stop_machine expects to yield to it. */
 		rq = task_rq_lock(p, &flags);
diff --git a/kernel/softirq.c b/kernel/softirq.c
index ad3295c..25cae0d 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -350,7 +350,6 @@ void __init softirq_init(void)
 static int ksoftirqd(void * __bind_cpu)
 {
 	set_user_nice(current, 19);
-	current->flags |= PF_NOFREEZE;
 
 	set_current_state(TASK_INTERRUPTIBLE);
 
@@ -456,7 +455,7 @@ static int __devinit cpu_callback(struct
 	case CPU_UP_PREPARE:
 		BUG_ON(per_cpu(tasklet_vec, hotcpu).list);
 		BUG_ON(per_cpu(tasklet_hi_vec, hotcpu).list);
-		p = kthread_create(ksoftirqd, hcpu, "ksoftirqd/%d", hotcpu);
+		p = kthread_nofreeze_create(ksoftirqd, hcpu, "ksoftirqd/%d", hotcpu);
 		if (IS_ERR(p)) {
 			printk("ksoftirqd for %i failed\n", hotcpu);
 			return NOTIFY_BAD;
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index b052e2c..124b6b8 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -191,8 +191,6 @@ static int worker_thread(void *__cwq)
 	struct k_sigaction sa;
 	sigset_t blocked;
 
-	current->flags |= PF_NOFREEZE;
-
 	set_user_nice(current, -5);
 
 	/* Block and flush all signals */
@@ -213,6 +211,7 @@ static int worker_thread(void *__cwq)
 			schedule();
 		else
 			__set_current_state(TASK_RUNNING);
+		try_to_freeze();
 		remove_wait_queue(&cwq->more_work, &wait);
 
 		if (!list_empty(&cwq->worklist))
@@ -282,7 +281,8 @@ void fastcall flush_workqueue(struct wor
 }
 
 static struct task_struct *create_workqueue_thread(struct workqueue_struct *wq,
-						   int cpu)
+						   int cpu,
+						   unsigned long freezer_flags)
 {
 	struct cpu_workqueue_struct *cwq = per_cpu_ptr(wq->cpu_wq, cpu);
 	struct task_struct *p;
@@ -296,10 +296,21 @@ static struct task_struct *create_workqu
 	init_waitqueue_head(&cwq->more_work);
 	init_waitqueue_head(&cwq->work_done);
 
-	if (is_single_threaded(wq))
-		p = kthread_create(worker_thread, cwq, "%s", wq->name);
-	else
-		p = kthread_create(worker_thread, cwq, "%s/%d", wq->name, cpu);
+	if (is_single_threaded(wq)) {
+		if (freezer_flags)
+			p = kthread_nofreeze_create(worker_thread, cwq,
+					"%s", wq->name);
+		else
+			p = kthread_create(worker_thread, cwq,
+					"%s", wq->name);
+	} else {
+		if (freezer_flags)
+			p = kthread_nofreeze_create(worker_thread, cwq,
+					"%s/%d", wq->name, cpu);
+		else
+			p = kthread_create(worker_thread, cwq,
+					"%s/%d", wq->name, cpu);
+	}
 	if (IS_ERR(p))
 		return NULL;
 	cwq->thread = p;
@@ -307,7 +318,8 @@ static struct task_struct *create_workqu
 }
 
 struct workqueue_struct *__create_workqueue(const char *name,
-					    int singlethread)
+					    int singlethread,
+					    unsigned long freezer_flags)
 {
 	int cpu, destroy = 0;
 	struct workqueue_struct *wq;
@@ -328,7 +340,8 @@ struct workqueue_struct *__create_workqu
 	lock_cpu_hotplug();
 	if (singlethread) {
 		INIT_LIST_HEAD(&wq->list);
-		p = create_workqueue_thread(wq, singlethread_cpu);
+		p = create_workqueue_thread(wq, singlethread_cpu,
+				freezer_flags);
 		if (!p)
 			destroy = 1;
 		else
@@ -338,7 +351,7 @@ struct workqueue_struct *__create_workqu
 		list_add(&wq->list, &workqueues);
 		spin_unlock(&workqueue_lock);
 		for_each_online_cpu(cpu) {
-			p = create_workqueue_thread(wq, cpu);
+			p = create_workqueue_thread(wq, cpu, freezer_flags);
 			if (p) {
 				kthread_bind(p, cpu);
 				wake_up_process(p);
@@ -529,7 +542,7 @@ static int __devinit workqueue_cpu_callb
 	case CPU_UP_PREPARE:
 		/* Create a new workqueue thread for it. */
 		list_for_each_entry(wq, &workqueues, list) {
-			if (!create_workqueue_thread(wq, hotcpu)) {
+			if (!create_workqueue_thread(wq, hotcpu, 0)) {
 				printk("workqueue for %i failed\n", hotcpu);
 				return NOTIFY_BAD;
 			}
@@ -572,7 +585,7 @@ void init_workqueues(void)
 {
 	singlethread_cpu = first_cpu(cpu_possible_map);
 	hotcpu_notifier(workqueue_cpu_callback, 0);
-	keventd_wq = create_workqueue("events");
+	keventd_wq = create_nofreeze_workqueue("events");
 	BUG_ON(!keventd_wq);
 }
 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 02/23] [Suspend2] Add new include/linux/freezer.h header.

[Nigel Cunningham]

This patch implements a new freezer header file, separated from
suspend.h and sched.h so that 95 percent of the kernel doesn't need
to be recompiled if/when we modify the freezer implementation.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 include/linux/freezer.h |   28 ++++++++++++++++++++++++++++
 1 files changed, 28 insertions(+), 0 deletions(-)

diff --git a/include/linux/freezer.h b/include/linux/freezer.h
new file mode 100644
index 0000000..5663f24
--- /dev/null
+++ b/include/linux/freezer.h
@@ -0,0 +1,28 @@
+/* Freezer declarations */
+
+#define FREEZER_ON 0
+#define ABORT_FREEZING 1
+
+#define FREEZER_KERNEL_THREADS 0
+#define FREEZER_ALL_THREADS 1
+
+#ifdef CONFIG_PM
+extern unsigned long freezer_state;
+
+#define test_freezer_state(bit) test_bit(bit, &freezer_state)
+#define set_freezer_state(bit) set_bit(bit, &freezer_state)
+#define clear_freezer_state(bit) clear_bit(bit, &freezer_state)
+
+#define freezer_is_on() (test_freezer_state(FREEZER_ON))
+
+extern void do_freeze_process(struct notifier_block *nl);
+
+#else
+
+#define test_freezer_state(bit) (0)
+#define set_freezer_state(bit) do { } while(0)
+#define clear_freezer_state(bit) do { } while(0)
+
+#define freezer_is_on() (0)
+
+#endif

--
Nigel Cunningham		nigel at suspend2 dot net

[ 03/23] [Suspend2] Allow a notifier to remove itself from the notifier list.

[Nigel Cunningham]

This is Christoph Lameter's original patch, posted to LKML some time ago,
but as yet unmerged.

Allow a notifier to remove itself from the notifier list.

If the next pointer is retrieved before notifier_call then the notifier can
remove itself from the chain.

Signed-off-by: Christoph Lameter <christoph@lameter.com>
Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/sys.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/kernel/sys.c b/kernel/sys.c
index d09cac2..46455fa 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -174,15 +174,18 @@ int __kprobes notifier_call_chain(struct
 {
 	int ret=NOTIFY_DONE;
 	struct notifier_block *nb = *n;
+	struct notifier_block *next;
 
 	while(nb)
 	{
-		ret=nb->notifier_call(nb,val,v);
+		/* Determining next here allows the notifier to unregister itself */
+		next = nb->next;
+		ret = nb->notifier_call(nb,val,v);
 		if(ret&NOTIFY_STOP_MASK)
 		{
 			return ret;
 		}
-		nb=nb->next;
+		nb = next;
 	}
 	return ret;
 }

--
Nigel Cunningham		nigel at suspend2 dot net

[ 04/23] [Suspend2] Todo notifier for processes.

[Nigel Cunningham]

(This is Christoph Lameter's patch, previously posted to LKML. This
revision only updates it to apply against the current codebase).

Introduce a todo notifier in the task_struct so that a task can be told to
do certain things. Abuse the suspend hooks try_to_freeze, freezing and
refrigerator to establish checkpoints where the todo list is processed.

This will break software suspend (next patch fixes and cleans up
software suspend).

This patch has been updated so as to apply to current git. Since that's
the only change, I've kept Christoph's original Signed-off-by.

Signed-off-by: Christoph Lameter <christoph@lameter.com>
Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 arch/arm/kernel/signal.c                  |    2 -
 arch/frv/kernel/signal.c                  |    2 -
 arch/h8300/kernel/signal.c                |    2 -
 arch/i386/kernel/io_apic.c                |    2 -
 arch/i386/kernel/signal.c                 |    2 -
 arch/m32r/kernel/signal.c                 |    2 -
 arch/mips/kernel/irixsig.c                |    2 -
 arch/mips/kernel/signal32.c               |    2 -
 arch/powerpc/kernel/signal_32.c           |    2 -
 arch/sh/kernel/signal.c                   |    2 -
 arch/sh64/kernel/signal.c                 |    2 -
 arch/x86_64/kernel/signal.c               |    2 -
 drivers/block/pktcdvd.c                   |    3 -
 drivers/ieee1394/nodemgr.c                |    2 -
 drivers/input/gameport/gameport.c         |    2 -
 drivers/input/serio/serio.c               |    8 +++
 drivers/macintosh/therm_adt746x.c         |    2 -
 drivers/media/dvb/dvb-core/dvb_frontend.c |    2 -
 drivers/media/video/msp3400-driver.c      |    2 -
 drivers/media/video/tvaudio.c             |    2 -
 drivers/media/video/video-buf-dvb.c       |    2 -
 drivers/net/irda/stir4200.c               |    4 +
 drivers/net/wireless/airo.c               |    2 -
 drivers/pcmcia/cs.c                       |    2 -
 drivers/pnp/pnpbios/core.c                |    2 -
 drivers/usb/core/hub.c                    |    2 -
 drivers/usb/gadget/file_storage.c         |    2 -
 drivers/usb/storage/usb.c                 |    2 -
 drivers/w1/w1.c                           |    4 +
 fs/afs/kafsasyncd.c                       |    2 -
 fs/afs/kafstimod.c                        |    2 -
 fs/jbd/journal.c                          |    4 +
 fs/jffs/intrep.c                          |    2 -
 fs/jffs2/background.c                     |    2 -
 fs/jfs/jfs_logmgr.c                       |    4 +
 fs/jfs/jfs_txnmgr.c                       |    8 +--
 fs/lockd/clntlock.c                       |    1 
 fs/lockd/clntproc.c                       |    2 -
 fs/lockd/svc.c                            |    2 +
 fs/xfs/linux-2.6/xfs_buf.c                |    4 +
 fs/xfs/linux-2.6/xfs_super.c              |    2 -
 include/linux/sched.h                     |   80 ++++++++---------------------
 kernel/audit.c                            |    2 -
 kernel/sched.c                            |    2 -
 kernel/signal.c                           |    4 +
 kernel/workqueue.c                        |    2 -
 mm/page_alloc.c                           |    1 
 mm/pdflush.c                              |    2 -
 mm/vmscan.c                               |    2 -
 net/rxrpc/krxiod.c                        |    2 -
 net/rxrpc/krxsecd.c                       |    2 -
 net/rxrpc/krxtimod.c                      |    2 -
 net/sunrpc/sched.c                        |    4 +
 net/sunrpc/svcsock.c                      |    4 +
 54 files changed, 94 insertions(+), 119 deletions(-)

diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c
index a0cd0a9..7bbcf0f 100644
--- a/arch/arm/kernel/signal.c
+++ b/arch/arm/kernel/signal.c
@@ -644,7 +644,7 @@ static int do_signal(sigset_t *oldset, s
 	if (!user_mode(regs))
 		return 0;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (current->ptrace & PT_SINGLESTEP)
diff --git a/arch/frv/kernel/signal.c b/arch/frv/kernel/signal.c
index 679c1d5..5d1567f 100644
--- a/arch/frv/kernel/signal.c
+++ b/arch/frv/kernel/signal.c
@@ -500,7 +500,7 @@ static void do_signal(void)
 	if (!user_mode(__frame))
 		return;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (test_thread_flag(TIF_RESTORE_SIGMASK))
diff --git a/arch/h8300/kernel/signal.c b/arch/h8300/kernel/signal.c
index f13d5e8..acce2c7 100644
--- a/arch/h8300/kernel/signal.c
+++ b/arch/h8300/kernel/signal.c
@@ -516,7 +516,7 @@ asmlinkage int do_signal(struct pt_regs 
 	if ((regs->ccr & 0x10))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	current->thread.esp0 = (unsigned long) regs;
diff --git a/arch/i386/kernel/io_apic.c b/arch/i386/kernel/io_apic.c
index f2dd218..760da02 100644
--- a/arch/i386/kernel/io_apic.c
+++ b/arch/i386/kernel/io_apic.c
@@ -578,7 +578,7 @@ static int balanced_irq(void *unused)
 
 	for ( ; ; ) {
 		time_remaining = schedule_timeout_interruptible(time_remaining);
-		try_to_freeze();
+		try_todo_list();
 		if (time_after(jiffies,
 				prev_balance_time+balanced_irq_interval)) {
 			preempt_disable();
diff --git a/arch/i386/kernel/signal.c b/arch/i386/kernel/signal.c
index 963616d..75f017e 100644
--- a/arch/i386/kernel/signal.c
+++ b/arch/i386/kernel/signal.c
@@ -582,7 +582,7 @@ static void fastcall do_signal(struct pt
 	if (!user_mode(regs))
 		return;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (test_thread_flag(TIF_RESTORE_SIGMASK))
diff --git a/arch/m32r/kernel/signal.c b/arch/m32r/kernel/signal.c
index 71763f7..5311cce 100644
--- a/arch/m32r/kernel/signal.c
+++ b/arch/m32r/kernel/signal.c
@@ -370,7 +370,7 @@ int do_signal(struct pt_regs *regs, sigs
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze()) 
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/arch/mips/kernel/irixsig.c b/arch/mips/kernel/irixsig.c
index 08273a2..0a86eb4 100644
--- a/arch/mips/kernel/irixsig.c
+++ b/arch/mips/kernel/irixsig.c
@@ -185,7 +185,7 @@ asmlinkage int do_irix_signal(sigset_t *
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/arch/mips/kernel/signal32.c b/arch/mips/kernel/signal32.c
index 98b185b..62d28ff 100644
--- a/arch/mips/kernel/signal32.c
+++ b/arch/mips/kernel/signal32.c
@@ -822,7 +822,7 @@ int do_signal32(sigset_t *oldset, struct
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c
index 3747ab0..0bd53de 100644
--- a/arch/powerpc/kernel/signal_32.c
+++ b/arch/powerpc/kernel/signal_32.c
@@ -1132,7 +1132,7 @@ int do_signal(sigset_t *oldset, struct p
 	int signr, ret;
 
 #ifdef CONFIG_PPC32
-	if (try_to_freeze()) {
+	if (try_todo_list()) {
 		signr = 0;
 		if (!signal_pending(current))
 			goto no_signal;
diff --git a/arch/sh/kernel/signal.c b/arch/sh/kernel/signal.c
index b475c4d..e6df050 100644
--- a/arch/sh/kernel/signal.c
+++ b/arch/sh/kernel/signal.c
@@ -578,7 +578,7 @@ int do_signal(struct pt_regs *regs, sigs
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/arch/sh64/kernel/signal.c b/arch/sh64/kernel/signal.c
index 3ea8929..8727da0 100644
--- a/arch/sh64/kernel/signal.c
+++ b/arch/sh64/kernel/signal.c
@@ -696,7 +696,7 @@ int do_signal(struct pt_regs *regs, sigs
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/arch/x86_64/kernel/signal.c b/arch/x86_64/kernel/signal.c
index 5876df1..f086f3c 100644
--- a/arch/x86_64/kernel/signal.c
+++ b/arch/x86_64/kernel/signal.c
@@ -443,7 +443,7 @@ int do_signal(struct pt_regs *regs, sigs
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c
index 93affee..0774b13 100644
--- a/drivers/block/pktcdvd.c
+++ b/drivers/block/pktcdvd.c
@@ -1255,8 +1255,7 @@ static int kcdrwd(void *foobar)
 			residue = schedule_timeout(min_sleep_time);
 			VPRINTK("kcdrwd: wake up\n");
 
-			/* make swsusp happy with our thread */
-			try_to_freeze();
+			try_todo_list();
 
 			list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) {
 				if (!pkt->sleep_time)
diff --git a/drivers/ieee1394/nodemgr.c b/drivers/ieee1394/nodemgr.c
index 082c7fd..b2766e9 100644
--- a/drivers/ieee1394/nodemgr.c
+++ b/drivers/ieee1394/nodemgr.c
@@ -1581,7 +1581,7 @@ static int nodemgr_host_thread(void *__h
 
 		if (down_interruptible(&hi->reset_sem) ||
 		    down_interruptible(&nodemgr_serialize)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			printk("NodeMgr: received unexpected signal?!\n" );
 			break;
diff --git a/drivers/input/gameport/gameport.c b/drivers/input/gameport/gameport.c
index b765a15..19271d5 100644
--- a/drivers/input/gameport/gameport.c
+++ b/drivers/input/gameport/gameport.c
@@ -440,7 +440,7 @@ static int gameport_thread(void *nothing
 		gameport_handle_event();
 		wait_event_interruptible(gameport_wait,
 			kthread_should_stop() || !list_empty(&gameport_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "gameport: kgameportd exiting\n");
diff --git a/drivers/input/serio/serio.c b/drivers/input/serio/serio.c
index edd7254..8e0c32b 100644
--- a/drivers/input/serio/serio.c
+++ b/drivers/input/serio/serio.c
@@ -312,6 +312,12 @@ static void serio_handle_event(void)
 
 		serio_remove_duplicate_events(event);
 		serio_free_event(event);
+
+		if (unlikely(todo_list_active())) {
+  			up(&serio_sem);
+			try_todo_list();
+  			down(&serio_sem);
+		}
 	}
 
 	up(&serio_sem);
@@ -375,7 +381,7 @@ static int serio_thread(void *nothing)
 		serio_handle_event();
 		wait_event_interruptible(serio_wait,
 			kthread_should_stop() || !list_empty(&serio_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "serio: kseriod exiting\n");
diff --git a/drivers/macintosh/therm_adt746x.c b/drivers/macintosh/therm_adt746x.c
index 5ebfd1d..97d2ad7 100644
--- a/drivers/macintosh/therm_adt746x.c
+++ b/drivers/macintosh/therm_adt746x.c
@@ -337,7 +337,7 @@ static int monitor_task(void *arg)
 	struct thermostat* th = arg;
 
 	while(!kthread_should_stop()) {
-		try_to_freeze();
+		try_todo_list();
 		msleep_interruptible(2000);
 
 #ifndef DEBUG
diff --git a/drivers/media/dvb/dvb-core/dvb_frontend.c b/drivers/media/dvb/dvb-core/dvb_frontend.c
index 771f32d..f01d545 100644
--- a/drivers/media/dvb/dvb-core/dvb_frontend.c
+++ b/drivers/media/dvb/dvb-core/dvb_frontend.c
@@ -520,7 +520,7 @@ static int dvb_frontend_thread(void *dat
 			break;
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		if (down_interruptible(&fepriv->sem))
 			break;
diff --git a/drivers/media/video/msp3400-driver.c b/drivers/media/video/msp3400-driver.c
index 69ed369..4a4b2fe 100644
--- a/drivers/media/video/msp3400-driver.c
+++ b/drivers/media/video/msp3400-driver.c
@@ -398,7 +398,7 @@ int msp_sleep(struct msp_state *state, i
 	}
 
 	remove_wait_queue(&state->wq, &wait);
-	try_to_freeze();
+	try_todo_list();
 	return state->restart;
 }
 
diff --git a/drivers/media/video/tvaudio.c b/drivers/media/video/tvaudio.c
index 6d03b9b..0d9e473 100644
--- a/drivers/media/video/tvaudio.c
+++ b/drivers/media/video/tvaudio.c
@@ -283,7 +283,7 @@ static int chip_thread(void *data)
 			schedule();
 		}
 		remove_wait_queue(&chip->wq, &wait);
-		try_to_freeze();
+		try_todo_list();
 		if (chip->done || signal_pending(current))
 			break;
 		v4l_dbg(1, debug, &chip->c, "%s: thread wakeup\n", chip->c.name);
diff --git a/drivers/media/video/video-buf-dvb.c b/drivers/media/video/video-buf-dvb.c
index 0a4004a..2568065 100644
--- a/drivers/media/video/video-buf-dvb.c
+++ b/drivers/media/video/video-buf-dvb.c
@@ -62,7 +62,7 @@ static int videobuf_dvb_thread(void *dat
 			break;
 		if (kthread_should_stop())
 			break;
-		try_to_freeze();
+		try_todo_list();
 
 		/* feed buffer data to demux */
 		if (buf->state == STATE_DONE)
diff --git a/drivers/net/irda/stir4200.c b/drivers/net/irda/stir4200.c
index 31867e4..f48f86e 100644
--- a/drivers/net/irda/stir4200.c
+++ b/drivers/net/irda/stir4200.c
@@ -762,7 +762,7 @@ static int stir_transmit_thread(void *ar
 	{
 #ifdef CONFIG_PM
 		/* if suspending, then power off and wait */
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active())) {
 			if (stir->receiving)
 				receive_stop(stir);
 			else
@@ -770,7 +770,7 @@ static int stir_transmit_thread(void *ar
 
 			write_reg(stir, REG_CTRL1, CTRL1_TXPWD|CTRL1_RXPWD);
 
-			refrigerator();
+			run_todo_list();
 
 			if (change_speed(stir, stir->speed))
 				break;
diff --git a/drivers/net/wireless/airo.c b/drivers/net/wireless/airo.c
index a4c7ae9..db505c6 100644
--- a/drivers/net/wireless/airo.c
+++ b/drivers/net/wireless/airo.c
@@ -2910,7 +2910,7 @@ static int airo_thread(void *data) {
 			flush_signals(current);
 
 		/* make swsusp happy with our thread */
-		try_to_freeze();
+		try_todo_list();
 
 		if (test_bit(JOB_DIE, &ai->flags))
 			break;
diff --git a/drivers/pcmcia/cs.c b/drivers/pcmcia/cs.c
index 613f2f1..2cafae0 100644
--- a/drivers/pcmcia/cs.c
+++ b/drivers/pcmcia/cs.c
@@ -681,7 +681,7 @@ static int pccardd(void *__skt)
 			break;
 
 		schedule();
-		try_to_freeze();
+		try_todo_list();
 	}
 	/* make sure we are running before we exit */
 	set_current_state(TASK_RUNNING);
diff --git a/drivers/pnp/pnpbios/core.c b/drivers/pnp/pnpbios/core.c
index b154b3f..f6ad731 100644
--- a/drivers/pnp/pnpbios/core.c
+++ b/drivers/pnp/pnpbios/core.c
@@ -170,7 +170,7 @@ static int pnp_dock_thread(void * unused
 		msleep_interruptible(2000);
 
 		if(signal_pending(current)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			break;
 		}
diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index 650d5ee..dc232f0 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -2870,7 +2870,7 @@ static int hub_thread(void *__unused)
 		wait_event_interruptible(khubd_wait,
 				!list_empty(&hub_event_list) ||
 				kthread_should_stop());
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop() || !list_empty(&hub_event_list));
 
 	pr_debug("%s: khubd exiting\n", usbcore_name);
diff --git a/drivers/usb/gadget/file_storage.c b/drivers/usb/gadget/file_storage.c
index de59c58..2ffca51 100644
--- a/drivers/usb/gadget/file_storage.c
+++ b/drivers/usb/gadget/file_storage.c
@@ -1557,7 +1557,7 @@ static int sleep_thread(struct fsg_dev *
 
 	/* Wait until a signal arrives or we are woken up */
 	for (;;) {
-		try_to_freeze();
+		try_todo_list();
 		set_current_state(TASK_INTERRUPTIBLE);
 		if (signal_pending(current)) {
 			rc = -EINTR;
diff --git a/drivers/usb/storage/usb.c b/drivers/usb/storage/usb.c
index dbcf239..0d6bcaa 100644
--- a/drivers/usb/storage/usb.c
+++ b/drivers/usb/storage/usb.c
@@ -860,7 +860,7 @@ retry:
 		wait_event_interruptible_timeout(us->delay_wait,
 				test_bit(US_FLIDX_DISCONNECTING, &us->flags),
 				delay_use * HZ);
-		if (try_to_freeze())
+		if (try_todo_list())
 			goto retry;
 	}
 
diff --git a/drivers/w1/w1.c b/drivers/w1/w1.c
index 024206c..7e8d624 100644
--- a/drivers/w1/w1.c
+++ b/drivers/w1/w1.c
@@ -720,7 +720,7 @@ static int w1_control(void *data)
 	while (!control_needs_exit || have_to_wait) {
 		have_to_wait = 0;
 
-		try_to_freeze();
+		try_todo_list();
 		msleep_interruptible(w1_control_timeout * 1000);
 
 		if (signal_pending(current))
@@ -796,7 +796,7 @@ int w1_process(void *data)
 	allow_signal(SIGTERM);
 
 	while (!test_bit(W1_MASTER_NEED_EXIT, &dev->flags)) {
-		try_to_freeze();
+		try_todo_list();
 		msleep_interruptible(w1_timeout * 1000);
 
 		if (signal_pending(current))
diff --git a/fs/afs/kafsasyncd.c b/fs/afs/kafsasyncd.c
index 7ac07d0..cd0cbda 100644
--- a/fs/afs/kafsasyncd.c
+++ b/fs/afs/kafsasyncd.c
@@ -116,7 +116,7 @@ static int kafsasyncd(void *arg)
 		remove_wait_queue(&kafsasyncd_sleepq, &myself);
 		set_current_state(TASK_RUNNING);
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
diff --git a/fs/afs/kafstimod.c b/fs/afs/kafstimod.c
index 65bc05a..127b2ac 100644
--- a/fs/afs/kafstimod.c
+++ b/fs/afs/kafstimod.c
@@ -91,7 +91,7 @@ static int kafstimod(void *arg)
 			complete_and_exit(&kafstimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
diff --git a/fs/jbd/journal.c b/fs/jbd/journal.c
index e4b516a..c7113af 100644
--- a/fs/jbd/journal.c
+++ b/fs/jbd/journal.c
@@ -153,7 +153,7 @@ loop:
 	}
 
 	wake_up(&journal->j_wait_done_commit);
-	if (freezing(current)) {
+	if (todo_list_active()) {
 		/*
 		 * The simpler the better. Flushing journal isn't a
 		 * good idea, because that depends on threads that may
@@ -161,7 +161,7 @@ loop:
 		 */
 		jbd_debug(1, "Now suspending kjournald\n");
 		spin_unlock(&journal->j_state_lock);
-		refrigerator();
+		run_todo_list();
 		spin_lock(&journal->j_state_lock);
 	} else {
 		/*
diff --git a/fs/jffs/intrep.c b/fs/jffs/intrep.c
index b2e9542..f077572 100644
--- a/fs/jffs/intrep.c
+++ b/fs/jffs/intrep.c
@@ -3391,7 +3391,7 @@ jffs_garbage_collect_thread(void *ptr)
 			siginfo_t info;
 			unsigned long signr = 0;
 
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 
 			spin_lock_irq(&current->sighand->siglock);
diff --git a/fs/jffs2/background.c b/fs/jffs2/background.c
index 7b77a95..a8b4fe3 100644
--- a/fs/jffs2/background.c
+++ b/fs/jffs2/background.c
@@ -96,7 +96,7 @@ static int jffs2_garbage_collect_thread(
 			schedule();
 		}
 
-		if (try_to_freeze())
+		if (try_todo_list())
 			continue;
 
 		cond_resched();
diff --git a/fs/jfs/jfs_logmgr.c b/fs/jfs/jfs_logmgr.c
index d27bac6..436e1e3 100644
--- a/fs/jfs/jfs_logmgr.c
+++ b/fs/jfs/jfs_logmgr.c
@@ -2362,9 +2362,9 @@ int jfsIOWait(void *arg)
 			lbmStartIO(bp);
 			spin_lock_irq(&log_redrive_lock);
 		}
-		if (freezing(current)) {
+		if (todo_list_active()) {
 			spin_unlock_irq(&log_redrive_lock);
-			refrigerator();
+			run_todo_list();
 		} else {
 			add_wait_queue(&jfs_IO_thread_wait, &wq);
 			set_current_state(TASK_INTERRUPTIBLE);
diff --git a/fs/jfs/jfs_txnmgr.c b/fs/jfs/jfs_txnmgr.c
index 2ddb6b8..7758373 100644
--- a/fs/jfs/jfs_txnmgr.c
+++ b/fs/jfs/jfs_txnmgr.c
@@ -2793,9 +2793,9 @@ int jfs_lazycommit(void *arg)
 		/* In case a wakeup came while all threads were active */
 		jfs_commit_thread_waking = 0;
 
-		if (freezing(current)) {
+		if (todo_list_active()) {
 			LAZY_UNLOCK(flags);
-			refrigerator();
+			run_todo_list();
 		} else {
 			DECLARE_WAITQUEUE(wq, current);
 
@@ -2992,9 +2992,9 @@ int jfs_sync(void *arg)
 		/* Add anon_list2 back to anon_list */
 		list_splice_init(&TxAnchor.anon_list2, &TxAnchor.anon_list);
 
-		if (freezing(current)) {
+		if (todo_list_active()) {
 			TXN_UNLOCK();
-			refrigerator();
+			run_todo_list();
 		} else {
 			DECLARE_WAITQUEUE(wq, current);
 
diff --git a/fs/lockd/clntlock.c b/fs/lockd/clntlock.c
index 3eaf6e7..df8a823 100644
--- a/fs/lockd/clntlock.c
+++ b/fs/lockd/clntlock.c
@@ -237,6 +237,7 @@ restart:
 
 		fl->fl_u.nfs_fl.flags &= ~NFS_LCK_RECLAIM;
 		nlmclnt_reclaim(host, fl);
+		try_todo_list();
 		if (signalled())
 			break;
 		goto restart;
diff --git a/fs/lockd/clntproc.c b/fs/lockd/clntproc.c
index 1455240..96daffe 100644
--- a/fs/lockd/clntproc.c
+++ b/fs/lockd/clntproc.c
@@ -311,7 +311,7 @@ static int nlm_wait_on_grace(wait_queue_
 	prepare_to_wait(queue, &wait, TASK_INTERRUPTIBLE);
 	if (!signalled ()) {
 		schedule_timeout(NLMCLNT_GRACE_WAIT);
-		try_to_freeze();
+		try_todo_list();
 		if (!signalled ())
 			status = 0;
 	}
diff --git a/fs/lockd/svc.c b/fs/lockd/svc.c
index 71a30b4..6fb8e0c 100644
--- a/fs/lockd/svc.c
+++ b/fs/lockd/svc.c
@@ -138,6 +138,8 @@ lockd(struct svc_rqst *rqstp)
 	while ((nlmsvc_users || !signalled()) && nlmsvc_pid == current->pid) {
 		long timeout = MAX_SCHEDULE_TIMEOUT;
 
+		try_todo_list();
+  
 		if (signalled()) {
 			flush_signals(current);
 			if (nlmsvc_ops) {
diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c
index e44b7c1..6696eba 100644
--- a/fs/xfs/linux-2.6/xfs_buf.c
+++ b/fs/xfs/linux-2.6/xfs_buf.c
@@ -1677,9 +1677,9 @@ xfsbufd(
 
 	INIT_LIST_HEAD(&tmp);
 	do {
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active(current))) {
 			set_bit(XBT_FORCE_SLEEP, &target->bt_flags);
-			refrigerator();
+			run_todo_list();
 		} else {
 			clear_bit(XBT_FORCE_SLEEP, &target->bt_flags);
 		}
diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c
index f22e426..cec9c38 100644
--- a/fs/xfs/linux-2.6/xfs_super.c
+++ b/fs/xfs/linux-2.6/xfs_super.c
@@ -578,7 +578,7 @@ xfssyncd(
 	for (;;) {
 		timeleft = schedule_timeout_interruptible(timeleft);
 		/* swsusp */
-		try_to_freeze();
+		try_todo_list();
 		if (kthread_should_stop() && list_empty(&vfsp->vfs_sync_list))
 			break;
 
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 0cfcd1c..7d566dd 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -35,6 +35,7 @@
 #include <linux/topology.h>
 #include <linux/seccomp.h>
 #include <linux/rcupdate.h>
+#include <linux/notifier.h>
 
 #include <linux/auxvec.h>	/* For AT_VECTOR_SIZE */
 
@@ -817,7 +818,10 @@ struct task_struct {
 	int (*notifier)(void *priv);
 	void *notifier_data;
 	sigset_t *notifier_mask;
-	
+
+	/* todo list to be executed in the context of this thread */
+	struct notifier_block *todo;
+
 	void *security;
 	struct audit_context *audit_context;
 	seccomp_t seccomp;
@@ -1390,79 +1394,37 @@ extern long sched_getaffinity(pid_t pid,
 
 extern void normalize_rt_tasks(void);
 
-#ifdef CONFIG_PM
-/*
- * Check if a process has been frozen
- */
-static inline int frozen(struct task_struct *p)
-{
-	return p->flags & PF_FROZEN;
-}
-
-/*
- * Check if there is a request to freeze a process
- */
-static inline int freezing(struct task_struct *p)
-{
-	return p->flags & PF_FREEZE;
-}
-
 /*
- * Request that a process be frozen
- * FIXME: SMP problem. We may not modify other process' flags!
+ * Check if there is a todo list request
  */
-static inline void freeze(struct task_struct *p)
+static inline int todo_list_active(void)
 {
-	p->flags |= PF_FREEZE;
+	return current->todo != NULL;
 }
 
-/*
- * Wake up a frozen process
- */
-static inline int thaw_process(struct task_struct *p)
-{
-	if (frozen(p)) {
-		p->flags &= ~PF_FROZEN;
-		wake_up_process(p);
-		return 1;
-	}
-	return 0;
-}
-
-/*
- * freezing is complete, mark process as frozen
- */
-static inline void frozen_process(struct task_struct *p)
+static inline void run_todo_list(void)
 {
-	p->flags = (p->flags & ~PF_FREEZE) | PF_FROZEN;
+	notifier_call_chain(&current->todo, 0, current);
 }
 
-extern void refrigerator(void);
-extern int freeze_processes(void);
-extern void thaw_processes(void);
-
-static inline int try_to_freeze(void)
+static inline int try_todo_list(void)
 {
-	if (freezing(current)) {
-		refrigerator();
+	if (todo_list_active()) {
+		run_todo_list();
 		return 1;
 	} else
 		return 0;
 }
-#else
-static inline int frozen(struct task_struct *p) { return 0; }
-static inline int freezing(struct task_struct *p) { return 0; }
-static inline void freeze(struct task_struct *p) { BUG(); }
-static inline int thaw_process(struct task_struct *p) { return 1; }
-static inline void frozen_process(struct task_struct *p) { BUG(); }
-
-static inline void refrigerator(void) {}
-static inline int freeze_processes(void) { BUG(); return 0; }
-static inline void thaw_processes(void) {}
 
-static inline int try_to_freeze(void) { return 0; }
+/*
+ * Compatibility definitions to use the suspend checkpoints for the task todo
+ * list. These may be removed once all uses of try_to_free,  refrigerator and
+ * freezing have been removed.
+ */
+#define try_to_freeze try_todo_list
+#define refrigerator run_todo_list
+#define freezing(p) todo_list_active()
 
-#endif /* CONFIG_PM */
 #endif /* __KERNEL__ */
 
 #endif
diff --git a/kernel/audit.c b/kernel/audit.c
index 0cd1b16..7af8794 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -289,7 +289,7 @@ static int kauditd_thread(void *dummy)
 		} else {
 			DECLARE_WAITQUEUE(wait, current);
 			
-			try_to_freeze();
+			try_todo_list();
 
 			set_current_state(TASK_INTERRUPTIBLE);
 			add_wait_queue(&kauditd_wait, &wait);
diff --git a/kernel/sched.c b/kernel/sched.c
index ad3c9ac..7e685bc 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -4589,7 +4589,7 @@ static int migration_thread(void *data)
 		struct list_head *head;
 		migration_req_t *req;
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_irq(&rq->lock);
 
diff --git a/kernel/signal.c b/kernel/signal.c
index d3efafd..4b367b3 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -214,7 +214,7 @@ static inline int has_pending_signals(si
 fastcall void recalc_sigpending_tsk(struct task_struct *t)
 {
 	if (t->signal->group_stop_count > 0 ||
-	    (freezing(t)) ||
+	    (t->todo) ||
 	    PENDING(&t->pending, &t->blocked) ||
 	    PENDING(&t->signal->shared_pending, &t->blocked))
 		set_tsk_thread_flag(t, TIF_SIGPENDING);
@@ -2307,7 +2307,7 @@ sys_rt_sigtimedwait(const sigset_t __use
 
 			timeout = schedule_timeout_interruptible(timeout);
 
-			try_to_freeze();
+			try_todo_list();
 			spin_lock_irq(&current->sighand->siglock);
 			sig = dequeue_signal(current, &these, &info);
 			current->blocked = current->real_blocked;
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 124b6b8..74d2334 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -211,7 +211,7 @@ static int worker_thread(void *__cwq)
 			schedule();
 		else
 			__set_current_state(TASK_RUNNING);
-		try_to_freeze();
+		try_todo_list();
 		remove_wait_queue(&cwq->more_work, &wait);
 
 		if (!list_empty(&cwq->worklist))
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index df54e2f..f9151b9 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1030,6 +1030,7 @@ rebalance:
 			do_retry = 1;
 	}
 	if (do_retry) {
+		try_todo_list();
 		blk_congestion_wait(WRITE, HZ/50);
 		goto rebalance;
 	}
diff --git a/mm/pdflush.c b/mm/pdflush.c
index c4b6d0a..d1417da 100644
--- a/mm/pdflush.c
+++ b/mm/pdflush.c
@@ -106,7 +106,7 @@ static int __pdflush(struct pdflush_work
 		spin_unlock_irq(&pdflush_lock);
 
 		schedule();
-		if (try_to_freeze()) {
+		if (try_todo_list()) {
 			spin_lock_irq(&pdflush_lock);
 			continue;
 		}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 2e34b61..7bc80c3 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1463,7 +1463,7 @@ static int kswapd(void *p)
 	for ( ; ; ) {
 		unsigned long new_order;
 
-		try_to_freeze();
+		try_todo_list();
 
 		prepare_to_wait(&pgdat->kswapd_wait, &wait, TASK_INTERRUPTIBLE);
 		new_order = pgdat->kswapd_max_order;
diff --git a/net/rxrpc/krxiod.c b/net/rxrpc/krxiod.c
index dada34a..be76d85 100644
--- a/net/rxrpc/krxiod.c
+++ b/net/rxrpc/krxiod.c
@@ -138,7 +138,7 @@ static int rxrpc_krxiod(void *arg)
 
 		_debug("### End Work");
 
-		try_to_freeze();
+		try_todo_list();
 
                 /* discard pending signals */
 		rxrpc_discard_my_signals();
diff --git a/net/rxrpc/krxsecd.c b/net/rxrpc/krxsecd.c
index 1aadd02..38a587f 100644
--- a/net/rxrpc/krxsecd.c
+++ b/net/rxrpc/krxsecd.c
@@ -107,7 +107,7 @@ static int rxrpc_krxsecd(void *arg)
 
 		_debug("### End Inbound Calls");
 
-		try_to_freeze();
+		try_todo_list();
 
                 /* discard pending signals */
 		rxrpc_discard_my_signals();
diff --git a/net/rxrpc/krxtimod.c b/net/rxrpc/krxtimod.c
index 3e74669..8ca7ff3 100644
--- a/net/rxrpc/krxtimod.c
+++ b/net/rxrpc/krxtimod.c
@@ -90,7 +90,7 @@ static int krxtimod(void *arg)
 			complete_and_exit(&krxtimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		rxrpc_discard_my_signals();
diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c
index 7415406..1b2fb95 100644
--- a/net/sunrpc/sched.c
+++ b/net/sunrpc/sched.c
@@ -671,6 +671,9 @@ static int __rpc_execute(struct rpc_task
 
 		/* sync task: sleep here */
 		dprintk("RPC: %4d sync task going to sleep\n", task->tk_pid);
+
+		try_todo_list();
+
 		/* Note: Caller should be using rpc_clnt_sigmask() */
 		status = out_of_line_wait_on_bit(&task->tk_runstate,
 				RPC_TASK_QUEUED, rpc_wait_bit_interruptible,
@@ -711,6 +714,7 @@ static int __rpc_execute(struct rpc_task
 int
 rpc_execute(struct rpc_task *task)
 {
+	try_todo_list();
 	rpc_set_active(task);
 	rpc_set_running(task);
 	return __rpc_execute(task);
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 5058062..d840ddd 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1177,7 +1177,7 @@ svc_recv(struct svc_serv *serv, struct s
 	arg->len = (pages-1)*PAGE_SIZE;
 	arg->tail[0].iov_len = 0;
 
-	try_to_freeze();
+	try_todo_list();
 	cond_resched();
 	if (signalled())
 		return -EINTR;
@@ -1219,7 +1219,7 @@ svc_recv(struct svc_serv *serv, struct s
 
 		schedule_timeout(timeout);
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_bh(&serv->sv_lock);
 		remove_wait_queue(&rqstp->rq_wait, &wait);

--
Nigel Cunningham		nigel at suspend2 dot net

[ 05/23] [Suspend2] Make the freezer use todo lists.

[Nigel Cunningham]

(This is Christoph Lameter's original patch, slightly modified. I don't
remove PF_FROZEN because it will still be used by later modifications.
Christoph's original comment follows).

Make the suspend code use the todo list in the task_struct.

This patch makes the suspend code SMP clean by removing the current usage
of PF_FREEZE and PF_FROZEN. Instead it relies on the new notification
handler in the task_struct, a completion handler and an atomic counter for
the number of processes frozen.

All the logic is local to the suspend code and no longer requires changes
to other kernel components.

Signed-off-by: Christoph Lameter <christoph@lameter.com>
Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 Documentation/power/kernel_threads.txt |   10 +-
 Documentation/power/swsusp.txt         |    5 +
 include/linux/sched.h                  |    1 
 kernel/power/process.c                 |  134 ++++++++++++++++----------------
 4 files changed, 77 insertions(+), 73 deletions(-)

diff --git a/Documentation/power/kernel_threads.txt b/Documentation/power/kernel_threads.txt
index fb57784..5edd749 100644
--- a/Documentation/power/kernel_threads.txt
+++ b/Documentation/power/kernel_threads.txt
@@ -4,15 +4,15 @@ KERNEL THREADS
 Freezer
 
 Upon entering a suspended state the system will freeze all
-tasks. This is done by delivering pseudosignals. This affects
-kernel threads, too. To successfully freeze a kernel thread
-the thread has to check for the pseudosignal and enter the
-refrigerator. Code to do this looks like this:
+tasks. This is done by making all processes execute a notifier.
+This affects kernel threads, too. To successfully freeze a kernel thread
+the thread has to check for the notifications and call the notifier
+chain for the process. Code to do this looks like this:
 
 	do {
 		hub_events();
 		wait_event_interruptible(khubd_wait, !list_empty(&hub_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!signal_pending(current));
 
 from drivers/usb/core/hub.c::hub_thread()
diff --git a/Documentation/power/swsusp.txt b/Documentation/power/swsusp.txt
index 08c79d4..2de0e65 100644
--- a/Documentation/power/swsusp.txt
+++ b/Documentation/power/swsusp.txt
@@ -135,7 +135,8 @@ should be sent to the mailing list avail
 website, and not to the Linux Kernel Mailing List. We are working
 toward merging suspend2 into the mainline kernel.
 
-Q: A kernel thread must voluntarily freeze itself (call 'refrigerator').
+Q: A kernel thread must work on the todo list (call 'run_todo_list')
+to enter the refrigerator.
 I found some kernel threads that don't do it, and they don't freeze
 so the system can't sleep. Is this a known behavior?
 
@@ -144,7 +145,7 @@ place where the thread is safe to be fro
 should be held at that point and it must be safe to sleep there), and
 add:
 
-       try_to_freeze();
+       try_todo_list();
 
 If the thread is needed for writing the image to storage, you should
 instead set the PF_NOFREEZE process flag when creating the thread (and
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 7d566dd..991c5e0 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -924,7 +924,6 @@ static inline void put_task_struct(struc
 #define PF_MEMALLOC	0x00000800	/* Allocating memory */
 #define PF_FLUSHER	0x00001000	/* responsible for disk writeback */
 #define PF_USED_MATH	0x00002000	/* if unset the fpu must be initialized before use */
-#define PF_FREEZE	0x00004000	/* this task is being frozen for suspend now */
 #define PF_NOFREEZE	0x00008000	/* this thread should not be frozen */
 #define PF_FROZEN	0x00010000	/* frozen for system suspend */
 #define PF_FSTRANS	0x00020000	/* inside a filesystem transaction */
diff --git a/kernel/power/process.c b/kernel/power/process.c
index 28de118..a788186 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -18,6 +18,8 @@
  */
 #define TIMEOUT	(6 * HZ)
 
+DECLARE_COMPLETION(thaw);
+static atomic_t nr_frozen;
 
 static inline int freezeable(struct task_struct * p)
 {
@@ -31,27 +33,68 @@ static inline int freezeable(struct task
 	return 1;
 }
 
-/* Refrigerator is place where frozen processes are stored :-). */
-void refrigerator(void)
+/*
+ * Invoked by the task todo list notifier when the task to be
+ * frozen is running.
+ */
+static int freeze_process(struct notifier_block *nl, unsigned long x, void *v)
 {
 	/* Hmm, should we be allowed to suspend when there are realtime
 	   processes around? */
 	long save;
 	save = current->state;
-	pr_debug("%s entered refrigerator\n", current->comm);
+	pr_debug("%s frozen\n", current->comm);
+	current->flags |= PF_FROZEN;
+	notifier_chain_unregister(&current->todo, nl);
+	kfree(nl);
 	printk("=");
 
-	frozen_process(current);
 	spin_lock_irq(&current->sighand->siglock);
 	recalc_sigpending(); /* We sent fake signal, clean it up */
+	atomic_inc(&nr_frozen);
 	spin_unlock_irq(&current->sighand->siglock);
 
-	while (frozen(current)) {
-		current->state = TASK_UNINTERRUPTIBLE;
-		schedule();
-	}
-	pr_debug("%s left refrigerator\n", current->comm);
+	wait_for_completion(&thaw);
+	current->flags &= ~PF_FROZEN;
+	atomic_dec(&nr_frozen);
+	pr_debug("%s thawed\n", current->comm);
 	current->state = save;
+	return 0;
+}
+
+void thaw_processes(void)
+{
+	printk("Restarting tasks..");
+	complete_all(&thaw);
+	while (atomic_read(&nr_frozen) > 0)
+		schedule();
+	printk("done\n");
+}
+
+static inline void freeze(struct task_struct *p)
+{
+	unsigned long flags;
+
+	/*
+	 * We only freeze if the todo list is empty to avoid
+	 * putting multiple freeze handlers on the todo list.
+	 */
+	if (!p->todo) {
+		struct notifier_block *n;
+
+		n = kmalloc(sizeof(struct notifier_block),
+					GFP_ATOMIC);
+		if (n) {
+			n->notifier_call = freeze_process;
+			n->priority = 0;
+			notifier_chain_register(&p->todo, n);
+		}
+ 	}
+	/* Make the process work on its todo list */
+	spin_lock_irqsave(&p->sighand->siglock, flags);
+	recalc_sigpending();
+	signal_wake_up(p, 0);
+	spin_unlock_irqrestore(&p->sighand->siglock, flags);
 }
 
 /* 0 = success, else # of processes that we failed to stop */
@@ -60,75 +103,36 @@ int freeze_processes(void)
 	int todo;
 	unsigned long start_time;
 	struct task_struct *g, *p;
-	unsigned long flags;
 
-	printk( "Stopping tasks: " );
+	atomic_set(&nr_frozen, 0);
+	INIT_COMPLETION(thaw);
+
+	printk("Stopping tasks: ");
 	start_time = jiffies;
 	do {
 		todo = 0;
 		read_lock(&tasklist_lock);
 		do_each_thread(g, p) {
-			if (!freezeable(p))
-				continue;
-			if (frozen(p))
-				continue;
-
-			freeze(p);
-			spin_lock_irqsave(&p->sighand->siglock, flags);
-			signal_wake_up(p, 0);
-			spin_unlock_irqrestore(&p->sighand->siglock, flags);
-			todo++;
+			if (freezeable(p)) {
+				if (!(p->flags & PF_FROZEN))
+					freeze(p);
+				todo++;
+			}
 		} while_each_thread(g, p);
 		read_unlock(&tasklist_lock);
 		yield();			/* Yield is okay here */
 		if (todo && time_after(jiffies, start_time + TIMEOUT)) {
-			printk( "\n" );
-			printk(KERN_ERR " stopping tasks failed (%d tasks remaining)\n", todo );
-			break;
+			printk("\n");
+			printk(KERN_ERR " stopping tasks failed"
+					"(%d tasks remaining)\n",
+					todo - atomic_read(&nr_frozen));
+			thaw_processes();
+			return todo;
 		}
-	} while(todo);
+	} while (atomic_read(&nr_frozen) < todo);
 
-	/* This does not unfreeze processes that are already frozen
-	 * (we have slightly ugly calling convention in that respect,
-	 * and caller must call thaw_processes() if something fails),
-	 * but it cleans up leftover PF_FREEZE requests.
-	 */
-	if (todo) {
-		read_lock(&tasklist_lock);
-		do_each_thread(g, p)
-			if (freezing(p)) {
-				pr_debug("  clean up: %s\n", p->comm);
-				p->flags &= ~PF_FREEZE;
-				spin_lock_irqsave(&p->sighand->siglock, flags);
-				recalc_sigpending_tsk(p);
-				spin_unlock_irqrestore(&p->sighand->siglock, flags);
-			}
-		while_each_thread(g, p);
-		read_unlock(&tasklist_lock);
-		return todo;
-	}
-
-	printk( "|\n" );
+	printk("|\n");
 	BUG_ON(in_atomic());
 	return 0;
 }
 
-void thaw_processes(void)
-{
-	struct task_struct *g, *p;
-
-	printk( "Restarting tasks..." );
-	read_lock(&tasklist_lock);
-	do_each_thread(g, p) {
-		if (!freezeable(p))
-			continue;
-		if (!thaw_process(p))
-			printk(KERN_INFO " Strange, %s not stopped\n", p->comm );
-	} while_each_thread(g, p);
-
-	read_unlock(&tasklist_lock);
-	schedule();
-	printk( " done\n" );
-}
-
-EXPORT_SYMBOL(refrigerator);

--
Nigel Cunningham		nigel at suspend2 dot net

[ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Nigel Cunningham]

Disable usermode helper invocations when the freezer is on. This avoids
deadlocks due to hotplug events occuring while processes are frozen.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/kmod.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/kernel/kmod.c b/kernel/kmod.c
index 51a8920..12afa2c 100644
--- a/kernel/kmod.c
+++ b/kernel/kmod.c
@@ -36,6 +36,7 @@
 #include <linux/mount.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
+#include <linux/freezer.h>
 #include <asm/uaccess.h>
 
 extern int max_threads;
@@ -249,6 +250,9 @@ int call_usermodehelper_keys(char *path,
 	if (!khelper_wq)
 		return -EBUSY;
 
+	if (freezer_is_on())
+		return 0;
+
 	if (path[0] == '\0')
 		return 0;
 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 07/23] [Suspend2] Add which-to-thaw parameter to thaw_processes() calls.

[Nigel Cunningham]

Modify the existing invocations of thaw_processes so that they use the new
parameter specifying which processes (kernel only or all) to thaw.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/disk.c |    5 +++--
 kernel/power/main.c |    7 ++++---
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/kernel/power/disk.c b/kernel/power/disk.c
index e24446f..4fb8af1 100644
--- a/kernel/power/disk.c
+++ b/kernel/power/disk.c
@@ -10,6 +10,7 @@
  */
 
 #include <linux/suspend.h>
+#include <linux/freezer.h>
 #include <linux/syscalls.h>
 #include <linux/reboot.h>
 #include <linux/string.h>
@@ -106,7 +107,7 @@ static int prepare_processes(void)
 	if (!(error = swsusp_shrink_memory()))
 		return 0;
 thaw:
-	thaw_processes();
+	thaw_processes(FREEZER_ALL_THREADS);
 	enable_nonboot_cpus();
 	pm_restore_console();
 	return error;
@@ -115,7 +116,7 @@ thaw:
 static void unprepare_processes(void)
 {
 	platform_finish();
-	thaw_processes();
+	thaw_processes(FREEZER_ALL_THREADS);
 	enable_nonboot_cpus();
 	pm_restore_console();
 }
diff --git a/kernel/power/main.c b/kernel/power/main.c
index d253f3a..6f854e4 100644
--- a/kernel/power/main.c
+++ b/kernel/power/main.c
@@ -9,6 +9,7 @@
  */
 
 #include <linux/suspend.h>
+#include <linux/freezer.h>
 #include <linux/kobject.h>
 #include <linux/string.h>
 #include <linux/delay.h>
@@ -95,7 +96,7 @@ static int suspend_prepare(suspend_state
 	if (pm_ops->finish)
 		pm_ops->finish(state);
  Thaw:
-	thaw_processes();
+	thaw_processes(FREEZER_ALL_THREADS);
  Enable_cpu:
 	enable_nonboot_cpus();
 	pm_restore_console();
@@ -103,7 +104,7 @@ static int suspend_prepare(suspend_state
 }
 
 
-static int suspend_enter(suspend_state_t state)
+int suspend_enter(suspend_state_t state)
 {
 	int error = 0;
 	unsigned long flags;
@@ -135,7 +136,7 @@ static void suspend_finish(suspend_state
 	device_resume();
 	if (pm_ops && pm_ops->finish)
 		pm_ops->finish(state);
-	thaw_processes();
+	thaw_processes(FREEZER_ALL_THREADS);
 	enable_nonboot_cpus();
 	pm_restore_console();
 }

--
Nigel Cunningham		nigel at suspend2 dot net

[ 08/23] [Suspend2] New freezer explanation for kernel/power/process.c

[Nigel Cunningham]

Modify kernel/power/process.c header to describe how the new freezer
implementation works.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   26 +++++++++++++++++++++++---
 1 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index a788186..6da0445 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -1,8 +1,28 @@
 /*
- * drivers/power/process.c - Functions for starting/stopping processes on 
- *                           suspend transitions.
+ * kernel/power/process.c
  *
- * Originally from swsusp.
+ * Copyright (C) 1998-2001 Gabor Kuti <seasons@fornax.hu>
+ * Copyright (C) 1998,2001,2002 Pavel Machek <pavel@suse.cz>
+ * Copyright (C) 2002-2003 Florent Chabaud <fchabaud@free.fr>
+ * Copyright (C) 2002-2006 Nigel Cunningham <nigel@suspend2.net>
+ *
+ * This file is released under the GPLv2.
+ *
+ * It contains the routines used to freeze processes during a suspend
+ * cycle (or perhaps in future, a process migration).
+ *
+ * When quiescing the whole system, one process initiates this
+ * functionality by calling freeze_processes(). First, userspace
+ * is frozen. Then bdevs for mounted filesystems are frozen so
+ * as to ensure pending I/O is flushed and stopped. Finally,
+ * kernel threads which are no marked as unfreezeable (perhaps
+ * because they're needed for doing I/O) are frozen. Splitting
+ * kernel threads from userspace also improves reliability,
+ * because it ensures that a userspace thread doesn't deadlock,
+ * waiting for a kernel thread that has already been frozen to
+ * process some request, and because it stops the cause of most
+ * work for kernel threads (ie userspace), allowing those thread
+ * to complete their work more quickly.
  */
 
 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 09/23] [Suspend2] Quieten the freezer for normal operation.

[Nigel Cunningham]

Quieten the freezer by replacing printks with a local #define
that can be enabled if and when debugging is needed.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   19 +++++++++++++------
 1 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 6da0445..09fc9ca 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -33,6 +33,13 @@
 #include <linux/suspend.h>
 #include <linux/module.h>
 
+#if 0
+//#ifdef CONFIG_PM_DEBUG
+#define freezer_message(msg, a...) do { printk(msg, ##a); } while(0)
+#else
+#define freezer_message(msg, a...) do { } while(0)
+#endif
+
 /* 
  * Timeout for stopping processes
  */
@@ -67,7 +74,7 @@ static int freeze_process(struct notifie
 	current->flags |= PF_FROZEN;
 	notifier_chain_unregister(&current->todo, nl);
 	kfree(nl);
-	printk("=");
+	freezer_message("=");
 
 	spin_lock_irq(&current->sighand->siglock);
 	recalc_sigpending(); /* We sent fake signal, clean it up */
@@ -84,11 +91,11 @@ static int freeze_process(struct notifie
 
 void thaw_processes(void)
 {
-	printk("Restarting tasks..");
+	freezer_message("Restarting tasks..");
 	complete_all(&thaw);
 	while (atomic_read(&nr_frozen) > 0)
 		schedule();
-	printk("done\n");
+	freezer_message("done\n");
 }
 
 static inline void freeze(struct task_struct *p)
@@ -127,7 +134,7 @@ int freeze_processes(void)
 	atomic_set(&nr_frozen, 0);
 	INIT_COMPLETION(thaw);
 
-	printk("Stopping tasks: ");
+	freezer_message("Stopping tasks: ");
 	start_time = jiffies;
 	do {
 		todo = 0;
@@ -142,7 +149,7 @@ int freeze_processes(void)
 		read_unlock(&tasklist_lock);
 		yield();			/* Yield is okay here */
 		if (todo && time_after(jiffies, start_time + TIMEOUT)) {
-			printk("\n");
+			freezer_message("\n");
 			printk(KERN_ERR " stopping tasks failed"
 					"(%d tasks remaining)\n",
 					todo - atomic_read(&nr_frozen));
@@ -151,7 +158,7 @@ int freeze_processes(void)
 		}
 	} while (atomic_read(&nr_frozen) < todo);
 
-	printk("|\n");
+	freezer_message("|\n");
 	BUG_ON(in_atomic());
 	return 0;
 }

--
Nigel Cunningham		nigel at suspend2 dot net

[ 10/23] [Suspend2] Add support for freezing filesystem bdevs.

[Nigel Cunningham]

Add support for freezing and thawing bdevs of filesystems. Filesystems
are frozen in reverse order so that nested filesystems don't cause
deadlocks.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 09fc9ca..0e377ed 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -48,6 +48,56 @@
 DECLARE_COMPLETION(thaw);
 static atomic_t nr_frozen;
 
+struct frozen_fs
+{
+	struct list_head fsb_list;
+	struct super_block *sb;
+};
+
+LIST_HEAD(frozen_fs_list);
+
+void freezer_make_fses_rw(void)
+{
+	struct frozen_fs *fs, *next_fs;
+
+	list_for_each_entry_safe(fs, next_fs, &frozen_fs_list, fsb_list) {
+		thaw_bdev(fs->sb->s_bdev, fs->sb);
+
+		list_del(&fs->fsb_list);
+		kfree(fs);
+	}
+}
+
+/* 
+ * Done after userspace is frozen, so there should be no danger of
+ * fses being unmounted while we're in here.
+ */
+int freezer_make_fses_ro(void)
+{
+	struct frozen_fs *fs;
+	struct super_block *sb;
+
+	/* Generate the list */
+	list_for_each_entry(sb, &super_blocks, s_list) {
+		if (!sb->s_root || !sb->s_bdev ||
+		    (sb->s_frozen == SB_FREEZE_TRANS) ||
+		    (sb->s_flags & MS_RDONLY))
+			continue;
+
+		fs = kmalloc(sizeof(struct frozen_fs), GFP_ATOMIC);
+		fs->sb = sb;
+		list_add_tail(&fs->fsb_list, &frozen_fs_list);
+	};
+
+	/* Do the freezing in reverse order so filesystems dependant
+	 * upon others are frozen in the right order. (Eg loopback
+	 * on ext3). */
+	list_for_each_entry_reverse(fs, &frozen_fs_list, fsb_list)
+		freeze_bdev(fs->sb->s_bdev);
+
+	return 0;
+}
+
 static inline int freezeable(struct task_struct * p)
 {
 	if ((p == current) || 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 11/23] [Suspend2] Modify freezeable for freezing kernel threads separately.

[Nigel Cunningham]

Modify freezeable so that it takes an extra argument indicating
whether we are interested in all threads, or just userspace ones,
and ignores threads that are already in the refrigerator.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 0e377ed..444a163 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -98,14 +98,24 @@ int freezer_make_fses_ro(void)
 	return 0;
 }
 
-static inline int freezeable(struct task_struct * p)
+/*
+ * freezeable
+ *
+ * Description:	Determine whether a process should be frozen yet.
+ * Parameters:	struct task_struct *	The process to consider.
+ * 		int			Boolean - 0 = userspace else all.
+ * Returns:	int			0 if don't freeze yet, otherwise do.
+ */
+static int freezeable(struct task_struct * p, int all_freezable)
 {
 	if ((p == current) || 
+	    (p->flags & PF_FROZEN) ||
 	    (p->flags & PF_NOFREEZE) ||
 	    (p->exit_state == EXIT_ZOMBIE) ||
 	    (p->exit_state == EXIT_DEAD) ||
 	    (p->state == TASK_STOPPED) ||
-	    (p->state == TASK_TRACED))
+	    (p->state == TASK_TRACED) ||
+	    (!p->mm && !all_freezable))
 		return 0;
 	return 1;
 }

--
Nigel Cunningham		nigel at suspend2 dot net

[ 12/23] [Suspend2] Split freezing of threads according to whether user/kernelspace.

[Nigel Cunningham]

Split the freezing of threads according to whether they're userspace or
kernelspace threads. We use separate completion handlers, thereby allowing
for the possibility of thawing kernel space without thawing userspace.
This will be used to allow memory to be freed without it racing against
userspace.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   71 +++++++++++++++++++++++++++++++++++-------------
 1 files changed, 52 insertions(+), 19 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 444a163..a3aca9a 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -45,8 +45,11 @@
  */
 #define TIMEOUT	(6 * HZ)
 
-DECLARE_COMPLETION(thaw);
-static atomic_t nr_frozen;
+
+DECLARE_COMPLETION(kernelspace_thaw);
+DECLARE_COMPLETION(userspace_thaw);
+static atomic_t nr_userspace_frozen;
+static atomic_t nr_kernelspace_frozen;
 
 struct frozen_fs
 {
@@ -120,32 +123,62 @@ static int freezeable(struct task_struct
 	return 1;
 }
 
+static void __freeze_process(struct completion *completion_handler,
+				atomic_t *nr_frozen)
+{
+	long save;
+
+	freezer_message("%s (%d) frozen.\n",
+			current->comm, current->pid);
+	save = current->state;
+	
+	atomic_inc(nr_frozen);
+	wait_for_completion(completion_handler);
+	atomic_dec(nr_frozen);
+	
+	current->state = save;
+	freezer_message("%s (%d) leaving freezer.\n",
+			current->comm, current->pid);
+}
+
 /*
  * Invoked by the task todo list notifier when the task to be
  * frozen is running.
  */
 static int freeze_process(struct notifier_block *nl, unsigned long x, void *v)
 {
-	/* Hmm, should we be allowed to suspend when there are realtime
-	   processes around? */
-	long save;
-	save = current->state;
-	pr_debug("%s frozen\n", current->comm);
+	unsigned long flags;
+
+	might_sleep();
+
+	/* Locking to handle race against waking the process in
+	 * freeze threads. */
+	spin_lock_irqsave(&current->sighand->siglock, flags);
 	current->flags |= PF_FROZEN;
-	notifier_chain_unregister(&current->todo, nl);
-	kfree(nl);
-	freezer_message("=");
-
-	spin_lock_irq(&current->sighand->siglock);
-	recalc_sigpending(); /* We sent fake signal, clean it up */
-	atomic_inc(&nr_frozen);
-	spin_unlock_irq(&current->sighand->siglock);
 
-	wait_for_completion(&thaw);
+	if (nl)
+		notifier_chain_unregister(&current->todo, nl);
+
+	recalc_sigpending();
+	spin_unlock_irqrestore(&current->sighand->siglock, flags);
+
+	if (nl)
+		kfree(nl);
+
+	if (test_freezer_state(FREEZER_ON)) {
+		if (current->mm)
+			__freeze_process(&userspace_thaw, &nr_userspace_frozen);
+		else
+			__freeze_process(&kernelspace_thaw,
+					&nr_kernelspace_frozen);
+	}
+
+	spin_lock_irqsave(&current->sighand->siglock, flags);
+	recalc_sigpending();
+	spin_unlock_irqrestore(&current->sighand->siglock, flags);
+
 	current->flags &= ~PF_FROZEN;
-	atomic_dec(&nr_frozen);
-	pr_debug("%s thawed\n", current->comm);
-	current->state = save;
+
 	return 0;
 }
 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 13/23] [Suspend2] Add support for thawing just kernel threads or all threads.

[Nigel Cunningham]

Modify the thaw_processes routine so that it takes and implements a
parameter saying whether to thaw all processes, or just kernel space.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   25 +++++++++++++++++++------
 1 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index a3aca9a..dffe645 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -182,13 +182,26 @@ static int freeze_process(struct notifie
 	return 0;
 }
 
-void thaw_processes(void)
+void thaw_processes(int do_all_threads)
 {
-	freezer_message("Restarting tasks..");
-	complete_all(&thaw);
-	while (atomic_read(&nr_frozen) > 0)
-		schedule();
-	freezer_message("done\n");
+	if (do_all_threads) {
+		clear_freezer_state(FREEZER_ON);
+		clear_freezer_state(ABORT_FREEZING);
+	}
+
+	complete_all(&kernelspace_thaw);
+	while (atomic_read(&nr_kernelspace_frozen) > 0)
+		yield();
+
+	init_completion(&kernelspace_thaw);
+	freezer_make_fses_rw();
+
+	if (do_all_threads) {
+		complete_all(&userspace_thaw);
+		while (atomic_read(&nr_userspace_frozen) > 0)
+			yield();
+		init_completion(&userspace_thaw);
+	}
 }
 
 static inline void freeze(struct task_struct *p)

--
Nigel Cunningham		nigel at suspend2 dot net

[ 14/23] [Suspend2] Helper for counting freezeable threads of a type.

[Nigel Cunningham]

Add a helper, which returns the number of threads of the given type
(userspace only or all) that are still to be frozen.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index dffe645..d5d052a 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -204,6 +204,29 @@ void thaw_processes(int do_all_threads)
 	}
 }
 
+/*
+ * num_freezeable
+ *
+ * Description:	Determine how many processes of our type are still to be
+ * 		frozen. As a side effect, update the progress bar too.
+ * Parameters:	int	Which type we are trying to freeze.
+ * 		int	Whether we are displaying our progress.
+ */
+static int num_freezeable(int do_all_threads)
+{
+	struct task_struct *g, *p;
+	int todo_this_type = 0;
+
+	read_lock(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (freezeable(p, do_all_threads))
+			todo_this_type++;
+	} while_each_thread(g, p);
+	read_unlock(&tasklist_lock);
+
+	return todo_this_type;
+}
+
 static inline void freeze(struct task_struct *p)
 {
 	unsigned long flags;

--
Nigel Cunningham		nigel at suspend2 dot net

[ 15/23] [Suspend2] Helper for counting uninterruptible threads of a type.

[Nigel Cunningham]

Add a helper which counts the number of patches of a type (all
or userspace only) which are in TASK_UNINTERRUPTIBLE state.
These tasks are signalled (just in case they leave that state at
a later point), but we do not consider freezing to have failed
if and when they do not enter the freezer.

Note that when they eventually leave TASK_UNINTERRUPTIBLE state,
they will enter the refrigerator, but will immediately exit if
we no longer want to freeze at that point.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index d5d052a..9cfafa7 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -227,6 +227,29 @@ static int num_freezeable(int do_all_thr
 	return todo_this_type;
 }
 
+/*
+ * num_uninterruptible
+ *
+ * Description:	Determine how many processes of our type are in state
+ * 		task uninterruptible.
+ * Parameters:	int	Which type we are trying to freeze.
+ */
+static int num_uninterruptible(int do_all_threads)
+{
+	struct task_struct *g, *p;
+	int count = 0;
+
+	read_lock(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (freezeable(p, do_all_threads) &&
+			p->state == TASK_UNINTERRUPTIBLE)
+			count++;
+	} while_each_thread(g, p);
+	read_unlock(&tasklist_lock);
+
+	return count;
+}
+
 static inline void freeze(struct task_struct *p)
 {
 	unsigned long flags;

--
Nigel Cunningham		nigel at suspend2 dot net

[ 16/23] [Suspend2] Helper to signal all threads of a type.

[Nigel Cunningham]

Add a helper to do the actual signalling of processes, telling them to
enter the refrigerator.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   25 +++++++++++++++++++++++++
 1 files changed, 25 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 9cfafa7..2857c8b 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -250,6 +250,31 @@ static int num_uninterruptible(int do_al
 	return count;
 }
 
+/*
+ * Tell threads of the type to enter the freezer.
+ */
+static void signal_threads(int do_all_threads)
+{
+	struct task_struct *g, *p;
+	struct notifier_block *n;
+
+	read_lock(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (!freezeable(p, do_all_threads))
+			continue;
+
+		n = kmalloc(sizeof(struct notifier_block),
+					GFP_ATOMIC);
+
+		if (n) {
+			n->notifier_call = freeze_process;
+			n->priority = 0;
+			notifier_chain_register(&p->todo, n);
+ 		}
+	} while_each_thread(g, p);
+	read_unlock(&tasklist_lock);
+}
+
 static inline void freeze(struct task_struct *p)
 {
 	unsigned long flags;

--
Nigel Cunningham		nigel at suspend2 dot net

[ 17/23] [Suspend2] Helper to prod processes that should have frozen but haven't.

[Nigel Cunningham]

Add a helper which prods processes of the type we are currently
trying to freeze, seeking to get them to run their todo lists.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 2857c8b..b2a9147 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -275,6 +275,29 @@ static void signal_threads(int do_all_th
 	read_unlock(&tasklist_lock);
 }
 
+/*
+ * Prod processes that haven't entered the refrigerator yet.
+ */
+static void prod_processes(int do_all_threads)
+{
+	struct task_struct *g, *p;
+	unsigned long flags;
+
+	read_lock(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (!freezeable(p, do_all_threads))
+			continue;
+			
+		spin_lock_irqsave(&p->sighand->siglock, flags);
+		if (!(p->flags & PF_FROZEN)) {
+			recalc_sigpending();
+			signal_wake_up(p, 0);
+		}
+		spin_unlock_irqrestore(&p->sighand->siglock, flags);
+	} while_each_thread(g, p);
+	read_unlock(&tasklist_lock);
+}
+
 static inline void freeze(struct task_struct *p)
 {
 	unsigned long flags;

--
Nigel Cunningham		nigel at suspend2 dot net

[ 18/23] [Suspend2] Helper: Did we fail to freeze all threads of a type?

[Nigel Cunningham]

Add a helper which checks whether we failed to freeze all the processes
of a given type (ignoring uninterruptible threads). If such processes
are found, we notify the user.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   40 ++++++++++++++++++++++++++++++++++++++++
 1 files changed, 40 insertions(+), 0 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index b2a9147..4322155 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -298,6 +298,46 @@ static void prod_processes(int do_all_th
 	read_unlock(&tasklist_lock);
 }
 
+/*
+ * Freezer failure.
+ *
+ * Check whether we failed to freeze all the processes that
+ * should be frozen. If we find a task that failed to freeze,
+ * we give useful information on what failed and how.
+ */
+static int freezer_failure(int do_all_threads)
+{
+	int result = 0;
+	struct task_struct *g, *p;
+
+	read_lock(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (!freezeable(p, do_all_threads) || 
+				p->state == TASK_UNINTERRUPTIBLE) 
+			continue;
+
+		if (!result) {
+			printk(KERN_ERR	"Stopping tasks failed.\n");
+			printk(KERN_ERR "Tasks that refused to be "
+			 "refrigerated and haven't since exited:\n");
+			set_freezer_state(ABORT_FREEZING);
+			result = 1;
+		}
+
+		if ((freezing(p))) {
+			printk(" - %s (#%d) signalled but "
+				"didn't enter refrigerator.\n",
+				p->comm, p->pid);
+		} else
+			printk(" - %s (#%d) signalled "
+				"and todo list empty.\n",
+				p->comm, p->pid);
+	} while_each_thread(g, p);
+	read_unlock(&tasklist_lock);
+
+	return result;
+}
+
 static inline void freeze(struct task_struct *p)
 {
 	unsigned long flags;

--
Nigel Cunningham		nigel at suspend2 dot net

[ 19/23] [Suspend2] Freeze all threads of a type.

[Nigel Cunningham]

Add a helper, which seeks to freeze all threads of a given type. If the
type is kernel threads, it first freezes filesystems. Separate timeouts
are used for periodically prodding processes that haven't yet entered
the refrigerator, and for the timeout after which we consider failure
to have occured.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   65 ++++++++++++++++++++++++++++++------------------
 1 files changed, 41 insertions(+), 24 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 4322155..711ac1a 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -40,11 +40,9 @@
 #define freezer_message(msg, a...) do { } while(0)
 #endif
 
-/* 
- * Timeout for stopping processes
- */
-#define TIMEOUT	(6 * HZ)
-
+/* Timeouts when freezing */
+#define FREEZER_TOTAL_TIMEOUT (5 * HZ)
+#define FREEZER_CHECK_TIMEOUT (HZ / 10)
 
 DECLARE_COMPLETION(kernelspace_thaw);
 DECLARE_COMPLETION(userspace_thaw);
@@ -338,30 +336,49 @@ static int freezer_failure(int do_all_th
 	return result;
 }
 
-static inline void freeze(struct task_struct *p)
+/*
+ * freeze_threads
+ *
+ * Freeze a set of threads having particular attributes.
+ *
+ * Types:
+ * 2: User threads.
+ * 3: Kernel threads.
+ */
+static int freeze_threads(int do_all_threads)
 {
-	unsigned long flags;
+	int result = 0, still_to_do;
+	unsigned long start_time = jiffies;
+
+	if (do_all_threads)	
+		freezer_make_fses_ro();
+
+	signal_threads(do_all_threads);
+
+	/* Watch them do it, wake them if they ignore us. */
+	do {
+		prod_processes(do_all_threads);
+
+		set_task_state(current, TASK_INTERRUPTIBLE);
+		schedule_timeout(FREEZER_CHECK_TIMEOUT);
+
+		still_to_do = num_freezeable(do_all_threads) - 
+			num_uninterruptible(do_all_threads);
+
+	} while(still_to_do && (!test_freezer_state(ABORT_FREEZING)) &&
+		!time_after(jiffies, start_time + FREEZER_TOTAL_TIMEOUT));
 
 	/*
-	 * We only freeze if the todo list is empty to avoid
-	 * putting multiple freeze handlers on the todo list.
+	 * Did we time out? See if we failed to freeze processes as well.
+	 *
 	 */
-	if (!p->todo) {
-		struct notifier_block *n;
+	if ((time_after(jiffies, start_time + FREEZER_TOTAL_TIMEOUT))
+			&& (still_to_do))
+		result = freezer_failure(do_all_threads);
 
-		n = kmalloc(sizeof(struct notifier_block),
-					GFP_ATOMIC);
-		if (n) {
-			n->notifier_call = freeze_process;
-			n->priority = 0;
-			notifier_chain_register(&p->todo, n);
-		}
- 	}
-	/* Make the process work on its todo list */
-	spin_lock_irqsave(&p->sighand->siglock, flags);
-	recalc_sigpending();
-	signal_wake_up(p, 0);
-	spin_unlock_irqrestore(&p->sighand->siglock, flags);
+	BUG_ON(in_atomic());
+	
+	return 0; 
 }
 
 /* 0 = success, else # of processes that we failed to stop */

--
Nigel Cunningham		nigel at suspend2 dot net

[ 20/23] [Suspend2] Modify process.c includes and export freezer state.

[Nigel Cunningham]

Modify kernel/power/process #includes and add the exported freezer_state
variable. (The symbol is exported so that modules can use the
freezer_is_on() macro).

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index 711ac1a..aad2aa5 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -28,10 +28,15 @@
 
 #undef DEBUG
 
-#include <linux/smp_lock.h>
-#include <linux/interrupt.h>
 #include <linux/suspend.h>
+#include <linux/freezer.h>
 #include <linux/module.h>
+#include <linux/mount.h>
+#include <linux/namespace.h>
+#include <linux/buffer_head.h>
+#include <asm/tlbflush.h>
+
+unsigned long freezer_state = 0;
 
 #if 0
 //#ifdef CONFIG_PM_DEBUG
@@ -420,3 +425,4 @@ int freeze_processes(void)
 	return 0;
 }
 
+EXPORT_SYMBOL(freezer_state);

--
Nigel Cunningham		nigel at suspend2 dot net

[ 21/23] [Suspend2] Remove unused DEBUG undef.

[Nigel Cunningham]

Remove a now-unneeded #undef DEBUG.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/process.c |    3 ---
 1 files changed, 0 insertions(+), 3 deletions(-)

diff --git a/kernel/power/process.c b/kernel/power/process.c
index aad2aa5..9aa2fa0 100644
--- a/kernel/power/process.c
+++ b/kernel/power/process.c
@@ -25,9 +25,6 @@
  * to complete their work more quickly.
  */
 
-
-#undef DEBUG
-
 #include <linux/suspend.h>
 #include <linux/freezer.h>
 #include <linux/module.h>

--
Nigel Cunningham		nigel at suspend2 dot net

[ 22/23] [Suspend2] Modify swsusp to thaw kernel threads while eating memory.

[Nigel Cunningham]

Modify swsusp so that while trying to eat memory, it allows kernel threads
to run. This avoids a deadlock that could otherwise occur if access to a
filesystem is needed while freeing the memory.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 kernel/power/main.c   |    4 ++++
 kernel/power/swsusp.c |    3 +++
 2 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/kernel/power/main.c b/kernel/power/main.c
index 6f854e4..2fed3dc 100644
--- a/kernel/power/main.c
+++ b/kernel/power/main.c
@@ -72,6 +72,8 @@ static int suspend_prepare(suspend_state
 		goto Thaw;
 	}
 
+	thaw_processes(FREEZER_KERNEL_THREADS);
+
 	if ((free_pages = nr_free_pages()) < FREE_PAGE_NUMBER) {
 		pr_debug("PM: free some memory\n");
 		shrink_all_memory(FREE_PAGE_NUMBER - free_pages);
@@ -82,6 +84,8 @@ static int suspend_prepare(suspend_state
 		}
 	}
 
+	freeze_processes();
+
 	if (pm_ops->prepare) {
 		if ((error = pm_ops->prepare(state)))
 			goto Thaw;
diff --git a/kernel/power/swsusp.c b/kernel/power/swsusp.c
index 55a18d2..3bc835a 100644
--- a/kernel/power/swsusp.c
+++ b/kernel/power/swsusp.c
@@ -575,6 +575,8 @@ int swsusp_shrink_memory(void)
 	unsigned int i = 0;
 	char *p = "-\\|/";
 
+	thaw_processes(FREEZER_KERNEL_THREADS);
+
 	printk("Shrinking memory...  ");
 	do {
 		size = 2 * count_highmem_pages();
@@ -598,6 +600,7 @@ int swsusp_shrink_memory(void)
 	} while (tmp > 0);
 	printk("\bdone (%lu pages freed)\n", pages);
 
+	freeze_processes();
 	return 0;
 }
 

--
Nigel Cunningham		nigel at suspend2 dot net

[ 23/23] [Suspend2] Don't scan LRU while freezer is on.

[Nigel Cunningham]

Stop processes from initiating scanning of the LRU while the freezer is on.
This is primarily for the benefit of Suspend2, which could conceivably
trigger LRU scanning through this path while writing the image, and would
thus generate an inconsistent image and through it all sorts of trouble.

Signed-off-by: Nigel Cunningham <nigel@suspend2.net>

 mm/page_alloc.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index f9151b9..713d773 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -25,6 +25,7 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/suspend.h>
+#include <linux/freezer.h>
 #include <linux/pagevec.h>
 #include <linux/blkdev.h>
 #include <linux/slab.h>
@@ -957,8 +958,8 @@ restart:
 
 	/* This allocation should allow future memory freeing. */
 
-	if (((p->flags & PF_MEMALLOC) || unlikely(test_thread_flag(TIF_MEMDIE)))
-			&& !in_interrupt()) {
+	if ((((p->flags & PF_MEMALLOC) || unlikely(test_thread_flag(TIF_MEMDIE))) && 
+				!in_interrupt()) || (test_freezer_state(FREEZER_ON))) {
 		if (!(gfp_mask & __GFP_NOMEMALLOC)) {
 nofail_alloc:
 			/* go through the zonelist yet again, ignoring mins */

--
Nigel Cunningham		nigel at suspend2 dot net

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Pavel Machek]

Hi!

> This set of patches represents the freezer upgrade patches from Suspend2.
> 
> The key features of this changeset are:
> 
> - Use of Christoph Lameter's todo list notifiers, which help with SMP
>   cleanness.
> - Splitting the freezing of kernel and userspace processes. Freezing
>   currently suffers from a race because userspace processes can be
>   submitting work for kernel threads, thereby stopping them from
>   responding to freeze messages in a timely manner. The freezer can
>   thus give up when it doesn't really need to. (This is not normally
>   a problem only because load is not usually high).

Exactly. 6 seconds should be enough for signal to be delivered.

> - The use of bdev freezing to ensure filesystems are properly frozen,
>   thereby increasing the integrity of on-disk data in the case where
>   a resume doesn't occur. This is also helpful in the case of Suspend2,
>   where we don't atomically copy all memory, instead writing LRU pages
>   separately.

We have talked about this before... we alredy have sync there, and
that's enough. Your patch only makes code more complex for no gain.

								Pavel
-- 
Thanks, Sharp!

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Nigel Cunningham]

Hi.

On Thursday 26 January 2006 21:55, Pavel Machek wrote:
> Hi!
>
> > This set of patches represents the freezer upgrade patches from Suspend2.
> >
> > The key features of this changeset are:
> >
> > - Use of Christoph Lameter's todo list notifiers, which help with SMP
> >   cleanness.
> > - Splitting the freezing of kernel and userspace processes. Freezing
> >   currently suffers from a race because userspace processes can be
> >   submitting work for kernel threads, thereby stopping them from
> >   responding to freeze messages in a timely manner. The freezer can
> >   thus give up when it doesn't really need to. (This is not normally
> >   a problem only because load is not usually high).
>
> Exactly. 6 seconds should be enough for signal to be delivered.
>
> > - The use of bdev freezing to ensure filesystems are properly frozen,
> >   thereby increasing the integrity of on-disk data in the case where
> >   a resume doesn't occur. This is also helpful in the case of Suspend2,
> >   where we don't atomically copy all memory, instead writing LRU pages
> >   separately.
>
> We have talked about this before... we alredy have sync there, and
> that's enough. Your patch only makes code more complex for no gain.

Yes, we have. Sorry for failing to back up my statement of the need for the 
changes with numbers. I'll do a couple of hundred attempts under load with 
and without the changes, and send a further message saying how many times the 
current implementation fails.

Regards,

Nigel
-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Rafael J. Wysocki]

Hi,

On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> Hi everyone.
> 
> This set of patches represents the freezer upgrade patches from Suspend2.
> 
> The key features of this changeset are:
> 
> - Use of Christoph Lameter's todo list notifiers, which help with SMP
>   cleanness.
> - Splitting the freezing of kernel and userspace processes. Freezing
>   currently suffers from a race because userspace processes can be
>   submitting work for kernel threads, thereby stopping them from
>   responding to freeze messages in a timely manner. The freezer can
>   thus give up when it doesn't really need to. (This is not normally
>   a problem only because load is not usually high).

Could you please describe specific situation?

> - The use of bdev freezing to ensure filesystems are properly frozen,
>   thereby increasing the integrity of on-disk data in the case where
>   a resume doesn't occur. This is also helpful in the case of Suspend2,
>   where we don't atomically copy all memory, instead writing LRU pages
>   separately.

Is this also needed when we do atomically copy all memory?

Greetings,
Rafael

Re: [ 01/23] [Suspend2] Make workqueues freezeable.

[Rafael J. Wysocki]

Hi,

On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> 
> Prior to this patch, kernel threads and workqueues are unconditionally
> unfreezeable. This patch reverses that behaviour, making the default
> for kernel processes to be frozen. New variations of the routines for
> starting kernel threads and workqueues (containing _nofreeze_) allow
> threads that need to run during suspend to be made nofreeze again.

This looks like "let's make everything freezable and hunt for things that
must not be frozen" kind of approach, but isn't it error-prone?  I mean,
for example, if someone creates a kernel thread that in fact must not
be frozen, but forgets to use the _nofreeze_ call, things will break for
some people and the problem will be worse than the current one,
it seems.

Greetings,
Rafael

Re: [ 01/23] [Suspend2] Make workqueues freezeable.

[Nigel Cunningham]

Hi Rafael.

On Friday 27 January 2006 09:17, Rafael J. Wysocki wrote:
> Hi,
>
> On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> > Prior to this patch, kernel threads and workqueues are unconditionally
> > unfreezeable. This patch reverses that behaviour, making the default
> > for kernel processes to be frozen. New variations of the routines for
> > starting kernel threads and workqueues (containing _nofreeze_) allow
> > threads that need to run during suspend to be made nofreeze again.
>
> This looks like "let's make everything freezable and hunt for things that
> must not be frozen" kind of approach, but isn't it error-prone?  I mean,
> for example, if someone creates a kernel thread that in fact must not
> be frozen, but forgets to use the _nofreeze_ call, things will break for
> some people and the problem will be worse than the current one,
> it seems.

You're right that the default is now to freeze threads. This is mostly because 
the majority of kernel threads can and should be frozen because they're not 
needed. If a thread is frozen which shouldn't be, the user (which will 
hopefully be the developer testing their work) will experience a deadlock 
when they try to suspend. If, on the other hand, we let threads run that 
should be frozen, well - the outcome depends on what is done when the thread 
should be frozen. Obviously it doesn't hurt too much to let kernel threads 
run, or we'd see problems at the moment. For this reason, I consider this 
patch more to be a nicety than a necessity. Nice because it makes kernel 
threads behave in a manner consistent with what happens to userspace (hence 
defaulting to freezing, rather than not).

Regards,

Nigel
-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Nigel Cunningham]

Hi.

On Friday 27 January 2006 09:10, Rafael J. Wysocki wrote:
> Hi,
>
> On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> > Hi everyone.
> >
> > This set of patches represents the freezer upgrade patches from Suspend2.
> >
> > The key features of this changeset are:
> >
> > - Use of Christoph Lameter's todo list notifiers, which help with SMP
> >   cleanness.
> > - Splitting the freezing of kernel and userspace processes. Freezing
> >   currently suffers from a race because userspace processes can be
> >   submitting work for kernel threads, thereby stopping them from
> >   responding to freeze messages in a timely manner. The freezer can
> >   thus give up when it doesn't really need to. (This is not normally
> >   a problem only because load is not usually high).
>
> Could you please describe specific situation?

The simplest example would be:

dd if=/dev/hda of=/dev/null
echo disk > /sys/power/state

> > - The use of bdev freezing to ensure filesystems are properly frozen,
> >   thereby increasing the integrity of on-disk data in the case where
> >   a resume doesn't occur. This is also helpful in the case of Suspend2,
> >   where we don't atomically copy all memory, instead writing LRU pages
> >   separately.
>
> Is this also needed when we do atomically copy all memory?

I'm not thawing the bdevs until the end of resuming, so no.

Regards,

Nigel

-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Rafael J. Wysocki]

Hi,

On Friday, 27 January 2006 05:04, Nigel Cunningham wrote:
> On Friday 27 January 2006 09:10, Rafael J. Wysocki wrote:
> > On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> > > Hi everyone.
> > >
> > > This set of patches represents the freezer upgrade patches from Suspend2.
> > >
> > > The key features of this changeset are:
> > >
> > > - Use of Christoph Lameter's todo list notifiers, which help with SMP
> > >   cleanness.
> > > - Splitting the freezing of kernel and userspace processes. Freezing
> > >   currently suffers from a race because userspace processes can be
> > >   submitting work for kernel threads, thereby stopping them from
> > >   responding to freeze messages in a timely manner. The freezer can
> > >   thus give up when it doesn't really need to. (This is not normally
> > >   a problem only because load is not usually high).
> >
> > Could you please describe specific situation?
> 
> The simplest example would be:
> 
> dd if=/dev/hda of=/dev/null
> echo disk > /sys/power/state

Well, I don't think it's a usual kind of workload. :-)

Anyway, could you please give some details?  I mean how exactly your patch
helps in this particular case?

Greetings,
Rafael
 

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Nigel Cunningham]

Hi.

On Friday 27 January 2006 22:18, Rafael J. Wysocki wrote:
> Hi,
>
> On Friday, 27 January 2006 05:04, Nigel Cunningham wrote:
> > On Friday 27 January 2006 09:10, Rafael J. Wysocki wrote:
> > > On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> > > > Hi everyone.
> > > >
> > > > This set of patches represents the freezer upgrade patches from
> > > > Suspend2.
> > > >
> > > > The key features of this changeset are:
> > > >
> > > > - Use of Christoph Lameter's todo list notifiers, which help with SMP
> > > >   cleanness.
> > > > - Splitting the freezing of kernel and userspace processes. Freezing
> > > >   currently suffers from a race because userspace processes can be
> > > >   submitting work for kernel threads, thereby stopping them from
> > > >   responding to freeze messages in a timely manner. The freezer can
> > > >   thus give up when it doesn't really need to. (This is not normally
> > > >   a problem only because load is not usually high).
> > >
> > > Could you please describe specific situation?
> >
> > The simplest example would be:
> >
> > dd if=/dev/hda of=/dev/null
> > echo disk > /sys/power/state
>
> Well, I don't think it's a usual kind of workload. :-)

No, but I/O alone shouldn't have such effect.

> Anyway, could you please give some details?  I mean how exactly your patch
> helps in this particular case?

I thought I did :). Freezing userspace first means the dd thread gets stopped 
first. Once the dd thread is stopped, the kernel threads processing the I/O 
requests have a finite amount of work to do (instead of having new work being 
submitted all the time), and can thus complete that and then be frozen in a 
far more deterministic fashion.

Regarding the stats I promised to Pavel, I'm heading home from LCA today, so I 
probably won't get them prepared until Monday now - unless I get lazy and 
only do 10 attempts instead of 100 :)

Regards,

Nigel
-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Pavel Machek]

On So 28-01-06 05:20:28, Nigel Cunningham wrote:
> Hi.
> 
> On Friday 27 January 2006 22:18, Rafael J. Wysocki wrote:
> > Hi,
> >
> > On Friday, 27 January 2006 05:04, Nigel Cunningham wrote:
> > > On Friday 27 January 2006 09:10, Rafael J. Wysocki wrote:
> > > > On Thursday, 26 January 2006 04:45, Nigel Cunningham wrote:
> > > > > Hi everyone.
> > > > >
> > > > > This set of patches represents the freezer upgrade patches from
> > > > > Suspend2.
> > > > >
> > > > > The key features of this changeset are:
> > > > >
> > > > > - Use of Christoph Lameter's todo list notifiers, which help with SMP
> > > > >   cleanness.
> > > > > - Splitting the freezing of kernel and userspace processes. Freezing
> > > > >   currently suffers from a race because userspace processes can be
> > > > >   submitting work for kernel threads, thereby stopping them from
> > > > >   responding to freeze messages in a timely manner. The freezer can
> > > > >   thus give up when it doesn't really need to. (This is not normally
> > > > >   a problem only because load is not usually high).
> > > >
> > > > Could you please describe specific situation?
> > >
> > > The simplest example would be:
> > >
> > > dd if=/dev/hda of=/dev/null
> > > echo disk > /sys/power/state
> >
> > Well, I don't think it's a usual kind of workload. :-)
> 
> No, but I/O alone shouldn't have such effect.
> 
> > Anyway, could you please give some details?  I mean how exactly your patch
> > helps in this particular case?
> 
> I thought I did :). Freezing userspace first means the dd thread gets stopped 
> first. Once the dd thread is stopped, the kernel threads processing the I/O 
> requests have a finite amount of work to do (instead of having new work being 
> submitted all the time), and can thus complete that and then be frozen in a 
> far more deterministic fashion.
> 
> Regarding the stats I promised to Pavel, I'm heading home from LCA today, so I 
> probably won't get them prepared until Monday now - unless I get lazy and 
> only do 10 attempts instead of 100 :)

You could swapoff -a to make your cycle a lot faster.... Freezing is
still done in that case, IIRC.
								Pavel
-- 
Thanks, Sharp!

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 16848 bytes --]

Hi.

On Saturday 28 January 2006 09:22, Pavel Machek wrote:
> You could swapoff -a to make your cycle a lot faster.... Freezing is
> still done in that case, IIRC.

That was right. I just did the test:

I used

stress -d 5 --hdd-bytes 100M -i 5 -c 5

to test.

In both cases, I:

1) booted to init S on an amd64 laptop with 1GB ram
2) for swsusp, swapoff'd /dev/hda1 and for suspend2 did echo 1 > /proc/suspend2/freezer_test
3) started stress
4) then started the attempts at suspending without restarting stress between each iteration.

I usually left a couple of seconds between each iteration to allow stress to fill buffers etc again, but
was called away once during swsusp testing, leaving a much bigger period between 2 tests.

Kernel log output follows (sorry about the long lines):

Jan 30 15:33:08 localhost kernel: [  372.681816] Stopping tasks: ===================
Jan 30 15:33:08 localhost kernel: [  378.714028] Restarting tasks...<6> Strange, kjournald not stopped
Jan 30 15:33:08 localhost kernel: [  378.734593]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  378.755084]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  378.775148]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  378.794802]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  378.813947]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  387.171737]  done
Jan 30 15:33:08 localhost kernel: [  406.457958] Stopping tasks: ==========================|
Jan 30 15:33:08 localhost kernel: [  409.527817] Freeing memory...  ^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^Hdone (130644 pages freed)
Jan 30 15:33:08 localhost kernel: [  410.276750] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  410.690283] ACPI: PCI Interrupt 0000:00:14.0[A] -> GSI 17 (level, low) -> IRQ 169
Jan 30 15:33:08 localhost kernel: [  410.739835] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  410.758226] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 22 (level, low) -> IRQ 177
Jan 30 15:33:08 localhost kernel: [  410.776764]  pci_irq-0385 [03] pci_irq_derive        : Unable to derive IRQ for device 0000:00:1f.0
Jan 30 15:33:08 localhost kernel: [  410.795579] ACPI: PCI Interrupt 0000:00:1f.0[A]: no GSI
Jan 30 15:33:08 localhost kernel: [  411.472572] Restarting tasks... done
Jan 30 15:33:08 localhost kernel: [  489.731269] Stopping tasks: =======================
Jan 30 15:33:08 localhost kernel: [  495.767731] Restarting tasks...<6> Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  495.786297]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  500.733572]  done
Jan 30 15:33:08 localhost kernel: [  548.200154] Stopping tasks: =========================
Jan 30 15:33:08 localhost kernel: [  554.229009] Restarting tasks...<6> Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  559.584968]  done
Jan 30 15:33:08 localhost kernel: [  772.953163] Stopping tasks: ==========================|
Jan 30 15:33:08 localhost kernel: [  774.268679] Freeing memory...  ^H-^H\^H|^H/^H-^H\^H|^H/^H-^H\^H|^H/^Hdone (157371 pages freed)
Jan 30 15:33:08 localhost kernel: [  774.948384] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  775.352905] ACPI: PCI Interrupt 0000:00:14.0[A] -> GSI 17 (level, low) -> IRQ 169
Jan 30 15:33:08 localhost kernel: [  775.402457] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  775.421503] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 22 (level, low) -> IRQ 177
Jan 30 15:33:08 localhost kernel: [  775.441055]  pci_irq-0385 [03] pci_irq_derive        : Unable to derive IRQ for device 0000:00:1f.0
Jan 30 15:33:08 localhost kernel: [  775.461109] ACPI: PCI Interrupt 0000:00:1f.0[A]: no GSI
Jan 30 15:33:08 localhost kernel: [  776.140128] Restarting tasks... done
Jan 30 15:33:08 localhost kernel: [  793.729930] Stopping tasks: ================
Jan 30 15:33:08 localhost kernel: [  799.765453] Restarting tasks...<6> Strange, kjournald not stopped
Jan 30 15:33:08 localhost kernJan 30 15:33:08 localhost kernel: [  825.247334] Stopping tasks: =============================|
Jan 30 15:33:08 localhost kernel: [  825.279168] Freeing memory...  ^H-^H\^H|^Hdone (23358 pages freed)
Jan 30 15:33:08 localhost kernel: [  826.844573] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  827.409905] ACPI: PCI Interrupt 0000:00:14.0[A] -> GSI 17 (level, low) -> IRQ 169
Jan 30 15:33:08 localhost kernel: [  827.459457] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  827.476078] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 22 (level, low) -> IRQ 177
Jan 30 15:33:08 localhost kernel: [  827.492796]  pci_irq-0385 [03] pci_irq_derive        : Unable to derive IRQ for device 0000:00:1f.0
Jan 30 15:33:08 localhost kernel: [  827.509962] ACPI: PCI Interrupt 0000:00:1f.0[A]: no GSI
Jan 30 15:33:08 localhost kernel: [  828.182831] Restarting tasks... done
Jan 30 15:33:08 localhost kernel: [  870.265642] Stopping tasks: ================
Jan 30 15:33:08 localhost kernel: [  876.299090] Restarting tasks...<6> Strange, kjournald not stopped
Jan 30 15:33:08 localhost kernel: [  876.316040]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.332588]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.348638]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.364182]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.379489]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.394073]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.408172]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.422010]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.435853]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.449156]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.462449]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.475099]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  881.427744]  done
el: [  799.785374]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.805440]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.825298]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.844855]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.863529]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.881792]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  799.899884]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  799.917305]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  799.934153]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  799.950444]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  805.247267]  done
Jan 30 15:33:08 localhost kernel: [  825.247334] Stopping tasks: =============================|
Jan 30 15:33:08 localhost kernel: [  825.279168] Freeing memory...  ^H-^H\^H|^Hdone (23358 pages freed)
Jan 30 15:33:08 localhost kernel: [  826.844573] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  827.409905] ACPI: PCI Interrupt 0000:00:14.0[A] -> GSI 17 (level, low) -> IRQ 169
Jan 30 15:33:08 localhost kernel: [  827.459457] acpi_bus-0201 [02] bus_set_power         : Device is not power manageable
Jan 30 15:33:08 localhost kernel: [  827.476078] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 22 (level, low) -> IRQ 177
Jan 30 15:33:08 localhost kernel: [  827.492796]  pci_irq-0385 [03] pci_irq_derive        : Unable to derive IRQ for device 0000:00:1f.0
Jan 30 15:33:08 localhost kernel: [  827.509962] ACPI: PCI Interrupt 0000:00:1f.0[A]: no GSI
Jan 30 15:33:08 localhost kernel: [  828.182831] Restarting tasks... done
Jan 30 15:33:08 localhost kernel: [  870.265642] Stopping tasks: ================
Jan 30 15:33:08 localhost kernel: [  876.299090] Restarting tasks...<6> Strange, kjournald not stopped
Jan 30 15:33:08 localhost kernel: [  876.316040]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.332588]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.348638]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.364182]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.379489]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.394073]  Strange, stress not stopped
Jan 30 15:33:08 localhost kernel: [  876.408172]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.422010]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.435853]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.449156]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.462449]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  876.475099]  Strange, pdflush not stopped
Jan 30 15:33:08 localhost kernel: [  881.427744]  done


Then with the new version:
[   86.975030] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[   86.975035] suspend_userui: program not configured. suspend_userui disabled.
[   86.975223] Freezing processes
[  101.101759] Preparing Image.
[  101.960220] Suspend2 debugging info:
[  101.960223] - SUSPEND core   : 2.2.0.1
[  101.960224] - Kernel Version : 2.6.15
[  101.960225] - Compiler vers. : 4.0
[  101.960227] - Attempt number : 1
[  101.960228] - Parameters     : 0 128 0 0 0 0
[  101.960229] - Overall expected compression percentage: 0.
[  101.960231] - Swapwriter active.
[  101.960232]   Swap available for image: 295184 pages.
[  101.960234] - Filewriter inactive.
[  101.960235] - No I/O speed stats available.
[  113.041463] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  113.041470] suspend_userui: program not configured. suspend_userui disabled.
[  113.041557] Freezing processes
[  131.156259] Preparing Image.
[  132.038683] Suspend2 debugging info:
[  132.038685] - SUSPEND core   : 2.2.0.1
[  132.038687] - Kernel Version : 2.6.15
[  132.038688] - Compiler vers. : 4.0
[  132.038689] - Attempt number : 2
[  132.038690] - Parameters     : 0 128 0 0 0 0
[  132.038692] - Overall expected compression percentage: 0.
[  132.038693] - Swapwriter active.
[  132.038694]   Swap available for image: 295184 pages.
[  132.038696] - Filewriter inactive.
[  132.038697] - No I/O speed stats available.
[  145.228742] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  145.228747] suspend_userui: program not configured. suspend_userui disabled.
[  145.228974] Freezing processes
[  152.404097] Preparing Image.
[  153.366314] Suspend2 debugging info:
[  153.366316] - SUSPEND core   : 2.2.0.1
[  153.366317] - Kernel Version : 2.6.15
[  153.366319] - Compiler vers. : 4.0
[  153.366320] - Attempt number : 3
[  153.366321] - Parameters     : 0 128 0 0 0 0
[  153.366323] - Overall expected compression percentage: 0.
[  153.366324] - Swapwriter active.
[  153.366325]   Swap available for image: 295184 pages.
[  153.366327] - Filewriter inactive.
[  153.366328] - No I/O speed stats available.
[  163.088026] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  163.088032] suspend_userui: program not configured. suspend_userui disabled.
[  163.088217] Freezing processes
[  179.203517] Preparing Image.
[  180.083987] Suspend2 debugging info:
[  180.083989] - SUSPEND core   : 2.2.0.1
[  180.083990] - Kernel Version : 2.6.15
[  180.083992] - Compiler vers. : 4.0
[  180.083993] - Attempt number : 4
[  180.083994] - Parameters     : 0 128 0 0 0 0
[  180.083996] - Overall expected compression percentage: 0.
[  180.083997] - Swapwriter active.
[  180.083998]   Swap available for image: 295184 pages.
[  180.084000] - Filewriter inactive.
[  180.084001] - No I/O speed stats available.
[  190.528302] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  190.528308] suspend_userui: program not configured. suspend_userui disabled.
[  190.528461] Freezing processes
[  198.554222] Preparing Image.
[  199.473349] Suspend2 debugging info:
[  199.473351] - SUSPEND core   : 2.2.0.1
[  199.473353] - Kernel Version : 2.6.15
[  199.473354] - Compiler vers. : 4.0
[  199.473355] - Attempt number : 5
[  199.473357] - Parameters     : 0 128 0 0 0 0
[  199.473358] - Overall expected compression percentage: 0.
[  199.473360] - Swapwriter active.
[  199.473361]   Swap available for image: 295184 pages.
[  199.473362] - Filewriter inactive.
[  199.473364] - No I/O speed stats available.
[  208.377977] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  208.377984] suspend_userui: program not configured. suspend_userui disabled.
[  208.378079] Freezing processes
[  227.420522] Preparing Image.
[  228.395082] Suspend2 debugging info:
[  228.395084] - SUSPEND core   : 2.2.0.1
[  228.395085] - Kernel Version : 2.6.15
[  228.395087] - Compiler vers. : 4.0
[  228.395088] - Attempt number : 6
[  228.395089] - Parameters     : 0 128 0 0 0 0
[  228.395091] - Overall expected compression percentage: 0.
[  228.395092] - Swapwriter active.
[  228.395093]   Swap available for image: 295184 pages.
[  228.395095] - Filewriter inactive.
[  228.395097] - No I/O speed stats available.
[  238.455606] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  238.455611] suspend_userui: program not configured. suspend_userui disabled.
[  238.455802] Freezing processes
[  245.563055] Preparing Image.
[  246.461738] Suspend2 debugging info:
[  246.461741] - SUSPEND core   : 2.2.0.1
[  246.461742] - Kernel Version : 2.6.15
[  246.461743] - Compiler vers. : 4.0
[  246.461745] - Attempt number : 7
[  246.461746] - Parameters     : 0 128 0 0 0 0
[  246.461747] - Overall expected compression percentage: 0.
[  246.461749] - Swapwriter active.
[  246.461750]   Swap available for image: 295184 pages.
[  246.461752] - Filewriter inactive.
[  255.808530] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  255.808537] suspend_userui: program not configured. suspend_userui disabled.
[  255.808785] Freezing processes
[  275.877164] Preparing Image.
[  276.277097] Losing some ticks... checking if CPU frequency changed.
[  276.705838] Suspend2 debugging info:
[  276.705841] - SUSPEND core   : 2.2.0.1
[  276.705842] - Kernel Version : 2.6.15
[  276.705844] - Compiler vers. : 4.0
[  276.705845] - Attempt number : 8
[  276.705846] - Parameters     : 0 128 0 0 0 0
[  276.705848] - Overall expected compression percentage: 0.
[  276.705849] - Swapwriter active.
[  276.705850]   Swap available for image: 295184 pages.
[  276.705852] - Filewriter inactive.
[  276.705853] - No I/O speed stats available.
[  288.497553] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  288.497559] suspend_userui: program not configured. suspend_userui disabled.
[  288.497791] Freezing processes
[  290.355243] Preparing Image.
[  291.262225] Suspend2 debugging info:
[  291.262228] - SUSPEND core   : 2.2.0.1
[  291.262229] - Kernel Version : 2.6.15
[  291.262230] - Compiler vers. : 4.0
[  291.262232] - Attempt number : 9
[  291.262233] - Parameters     : 0 128 0 0 0 0
[  291.262234] - Overall expected compression percentage: 0.
[  291.262236] - Swapwriter active.
[  291.262237]   Swap available for image: 295184 pages.
[  291.262239] - Filewriter inactive.
[  291.262240] - No I/O speed stats available.
[  309.866461] Suspend2 2.2.0.1: Initiating a software suspend cycle.
[  309.866467] suspend_userui: program not configured. suspend_userui disabled.
[  309.866655] Freezing processes
[  322.885993] Preparing Image.
[  323.860115] Suspend2 debugging info:
[  323.860118] - SUSPEND core   : 2.2.0.1
[  323.860119] - Kernel Version : 2.6.15
[  323.860121] - Compiler vers. : 4.0
[  323.860122] - Attempt number : 10
[  323.860123] - Parameters     : 0 128 0 0 0 0
[  323.860125] - Overall expected compression percentage: 0.
[  323.860126] - Swapwriter active.
[  323.860128]   Swap available for image: 295184 pages.
[  323.860129] - Filewriter inactive.
[  323.860130] - No I/O speed stats available.



-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Stefan Seyfried]

On Fri, Jan 27, 2006 at 01:18:01PM +0100, Rafael J. Wysocki wrote:
> Hi,
> 
> On Friday, 27 January 2006 05:04, Nigel Cunningham wrote:

> > The simplest example would be:
> > 
> > dd if=/dev/hda of=/dev/null
> > echo disk > /sys/power/state
> 
> Well, I don't think it's a usual kind of workload. :-)

Compiling a kernel, having updatedb and mandb run in the background
and then trying to suspend while the compile still runs might do as well.

And this can happen, think of "battery-critical => suspend" setup.
This is exactly the case i do not like the suspend to fail, because there
might not be enough juice left to do a second try.
-- 
Stefan Seyfried                  \ "I didn't want to write for pay. I
QA / R&D Team Mobile Devices      \ wanted to be paid for what I write."
SUSE LINUX Products GmbH, Nürnberg \                    -- Leonard Cohen

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Pavel Machek]

On Po 30-01-06 15:54:49, Nigel Cunningham wrote:
> Hi.
> 
> On Saturday 28 January 2006 09:22, Pavel Machek wrote:
> > You could swapoff -a to make your cycle a lot faster.... Freezing is
> > still done in that case, IIRC.
> 
> That was right. I just did the test:
> 
> I used
> 
> stress -d 5 --hdd-bytes 100M -i 5 -c 5
> 
> to test.

Quite easy to reproduce, indeed. I'll try to find out if simple
solution exists. (Notice that we still have that famous mysqld bug. I
still have testcase somewhere).
								Pavel
-- 
Thanks, Sharp!

Re: [ 00/23] [Suspend2] Freezer Upgrade Patches

[Rafael J. Wysocki]

Hi,

On Monday 30 January 2006 08:53, Stefan Seyfried wrote:
> On Fri, Jan 27, 2006 at 01:18:01PM +0100, Rafael J. Wysocki wrote:
> > On Friday, 27 January 2006 05:04, Nigel Cunningham wrote:
> 
> > > The simplest example would be:
> > > 
> > > dd if=/dev/hda of=/dev/null
> > > echo disk > /sys/power/state
> > 
> > Well, I don't think it's a usual kind of workload. :-)
> 
> Compiling a kernel, having updatedb and mandb run in the background
> and then trying to suspend while the compile still runs might do as well.
> 
> And this can happen, think of "battery-critical => suspend" setup.
> This is exactly the case i do not like the suspend to fail, because there
> might not be enough juice left to do a second try.

Still IMO we don't need all of the patches in the series to prevent this from
happening and it's quite difficult to single out the relevant change.

Greetings,
Rafael

Re: [ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Rafael J. Wysocki]

Hi,

On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> 
> Disable usermode helper invocations when the freezer is on. This avoids
> deadlocks due to hotplug events occuring while processes are frozen.
> 
> Signed-off-by: Nigel Cunningham <nigel@suspend2.net>
> 
>  kernel/kmod.c |    4 ++++
>  1 files changed, 4 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/kmod.c b/kernel/kmod.c
> index 51a8920..12afa2c 100644
> --- a/kernel/kmod.c
> +++ b/kernel/kmod.c
> @@ -36,6 +36,7 @@
>  #include <linux/mount.h>
>  #include <linux/kernel.h>
>  #include <linux/init.h>
> +#include <linux/freezer.h>
>  #include <asm/uaccess.h>
>  
>  extern int max_threads;
> @@ -249,6 +250,9 @@ int call_usermodehelper_keys(char *path,
>  	if (!khelper_wq)
>  		return -EBUSY;
>  
> +	if (freezer_is_on())
> +		return 0;
> +
>  	if (path[0] == '\0')
>  		return 0;
>  

Disabling the usermode helper while freeze_processes() is executed seems to be
a good idea to me, but I think it should be done with a mutex or something
like that.

Greetings,
Rafael

Re: [ 15/23] [Suspend2] Helper for counting uninterruptible threads of a type.

[Rafael J. Wysocki]

Hi,

On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> 
> Add a helper which counts the number of patches of a type (all
> or userspace only) which are in TASK_UNINTERRUPTIBLE state.
> These tasks are signalled (just in case they leave that state at
> a later point), but we do not consider freezing to have failed
> if and when they do not enter the freezer.
> 
> Note that when they eventually leave TASK_UNINTERRUPTIBLE state,
> they will enter the refrigerator, but will immediately exit if
> we no longer want to freeze at that point.

I think we need to do something like this to prevent problems with
freezing under load.

Greetings,
Rafael

Re: [ 15/23] [Suspend2] Helper for counting uninterruptible threads of a type.

[Pavel Machek]

On Po 30-01-06 23:18:28, Rafael J. Wysocki wrote:
> Hi,
> 
> On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> > 
> > Add a helper which counts the number of patches of a type (all
> > or userspace only) which are in TASK_UNINTERRUPTIBLE state.
> > These tasks are signalled (just in case they leave that state at
> > a later point), but we do not consider freezing to have failed
> > if and when they do not enter the freezer.
> > 
> > Note that when they eventually leave TASK_UNINTERRUPTIBLE state,
> > they will enter the refrigerator, but will immediately exit if
> > we no longer want to freeze at that point.
> 
> I think we need to do something like this to prevent problems with
> freezing under load.

That is dangerous... task in UNINTERRUPTIBLE may hold some lock,
AFAICT.

No, there's some simple bug in refrigerator, and I/we need to fix
that. Signals work under load, so refrigerator should, too.

							Pavel
-- 
Thanks, Sharp!

Re: [ 15/23] [Suspend2] Helper for counting uninterruptible threads of a type.

[Rafael J. Wysocki]

Hi,

On Monday 30 January 2006 23:25, Pavel Machek wrote:
> On Po 30-01-06 23:18:28, Rafael J. Wysocki wrote:
> > On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> > > 
> > > Add a helper which counts the number of patches of a type (all
> > > or userspace only) which are in TASK_UNINTERRUPTIBLE state.
> > > These tasks are signalled (just in case they leave that state at
> > > a later point), but we do not consider freezing to have failed
> > > if and when they do not enter the freezer.
> > > 
> > > Note that when they eventually leave TASK_UNINTERRUPTIBLE state,
> > > they will enter the refrigerator, but will immediately exit if
> > > we no longer want to freeze at that point.
> > 
> > I think we need to do something like this to prevent problems with
> > freezing under load.
> 
> That is dangerous... task in UNINTERRUPTIBLE may hold some lock,
> AFAICT.

Yes, and we have discussed that already, but frankly I'm still unconvinced. ;-)

> No, there's some simple bug in refrigerator, and I/we need to fix
> that. Signals work under load, so refrigerator should, too.

I don't think there's a bug as such.  The refrigerator is just very simple
and apparently does not cover all possible cases.

I think the problems with freezing tasks are generally related to
uninterruptible processes waiting for events that never happen.

IMHO we can try to defer calling freeze() for kernel threads until all of the
user space processes are frozen.  If that doesn't help, we'll need to treat
uninterruptible tasks in a special way, I'm afraid.

Greetings,
Rafael

Re: [ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

Hi.

On Tuesday 31 January 2006 08:05, Rafael J. Wysocki wrote:
> Hi,
>
> On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> > Disable usermode helper invocations when the freezer is on. This avoids
> > deadlocks due to hotplug events occuring while processes are frozen.
> >
> > Signed-off-by: Nigel Cunningham <nigel@suspend2.net>
> >
> >  kernel/kmod.c |    4 ++++
> >  1 files changed, 4 insertions(+), 0 deletions(-)
> >
> > diff --git a/kernel/kmod.c b/kernel/kmod.c
> > index 51a8920..12afa2c 100644
> > --- a/kernel/kmod.c
> > +++ b/kernel/kmod.c
> > @@ -36,6 +36,7 @@
> >  #include <linux/mount.h>
> >  #include <linux/kernel.h>
> >  #include <linux/init.h>
> > +#include <linux/freezer.h>
> >  #include <asm/uaccess.h>
> >
> >  extern int max_threads;
> > @@ -249,6 +250,9 @@ int call_usermodehelper_keys(char *path,
> >  	if (!khelper_wq)
> >  		return -EBUSY;
> >
> > +	if (freezer_is_on())
> > +		return 0;
> > +
> >  	if (path[0] == '\0')
> >  		return 0;
>
> Disabling the usermode helper while freeze_processes() is executed seems to
> be a good idea to me, but I think it should be done with a mutex or
> something like that.

With the refrigerator code you guys are using at the moment, ouldn't that 
result in deadlocks when we later try to freeze the process in preparation 
for the atomic restore? (Or perhaps you don't freeze processes at that 
point?)

Regards,

Nigel

> Greetings,
> Rafael
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ 15/23] [Suspend2] Helper for counting uninterruptible threads of a type.

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 1986 bytes --]

Hi.

On Tuesday 31 January 2006 08:25, Pavel Machek wrote:
> On Po 30-01-06 23:18:28, Rafael J. Wysocki wrote:
> > Hi,
> >
> > On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> > > Add a helper which counts the number of patches of a type (all
> > > or userspace only) which are in TASK_UNINTERRUPTIBLE state.
> > > These tasks are signalled (just in case they leave that state at
> > > a later point), but we do not consider freezing to have failed
> > > if and when they do not enter the freezer.
> > >
> > > Note that when they eventually leave TASK_UNINTERRUPTIBLE state,
> > > they will enter the refrigerator, but will immediately exit if
> > > we no longer want to freeze at that point.
> >
> > I think we need to do something like this to prevent problems with
> > freezing under load.
>
> That is dangerous... task in UNINTERRUPTIBLE may hold some lock,
> AFAICT.
>
> No, there's some simple bug in refrigerator, and I/we need to fix
> that. Signals work under load, so refrigerator should, too.

I know you understand English, Pavel! Re-read what I wrote previously! The 
problem is a race. You're telling processes that process I/O to freeze at the 
same time as you're telling processes that submit I/O to freeze. If kjournald 
(eg) enters the refrigerator while dd is still running, dd is going to have a 
chance of getting stuck, waiting for some I/O to complete.

This also means your sys_sync is totally useless if anything is submitting 
I/O, since there will still be I/O being submitted after the sync is done. 
This is why (in the numbers I submitted yesterday), the current 
implementation took so much longer than the new one. The sys_sync really only 
introduces a big delay. It needs to be done after the things that generate 
I/O have been stopped.

Regards,

Nigel

-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[RFC/RFT] finally solve "swsusp fails with mysqld" problem

[Pavel Machek]

Place refrigerator hook at more clever place; avoids "system can't be
suspended while mysqld running" problem.

I'd like you to test it. It looks correct to me, and it is actually a
solution, not a workaround like my previous tries. It still does not
solve suspend while running stress tests.

Signed-off-by: Pavel Machek <pavel@suse.cz>

---
commit 01d049aacf36961d361cfd382fcbf746afcbb61b
tree 67e3d1fcb65cebfab058f9a699534b1689494dad
parent 92dfb5b4ac96e200138006901837056825c758d3
author <pavel@amd.ucw.cz> Tue, 31 Jan 2006 10:18:22 +0100
committer <pavel@amd.ucw.cz> Tue, 31 Jan 2006 10:18:22 +0100

 arch/i386/kernel/signal.c |    3 ---
 fs/jbd/journal.c          |    1 +
 kernel/signal.c           |    4 ++--
 3 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/arch/i386/kernel/signal.c b/arch/i386/kernel/signal.c
index adcd069..5ad8c65 100644
--- a/arch/i386/kernel/signal.c
+++ b/arch/i386/kernel/signal.c
@@ -615,9 +615,6 @@ int fastcall do_signal(struct pt_regs *r
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
-		goto no_signal;
-
 	if (!oldset)
 		oldset = &current->blocked;
 
diff --git a/fs/jbd/journal.c b/fs/jbd/journal.c
index e4b516a..20b5918 100644
--- a/fs/jbd/journal.c
+++ b/fs/jbd/journal.c
@@ -153,6 +153,7 @@ loop:
 	}
 
 	wake_up(&journal->j_wait_done_commit);
+	/* Race here? May someone already be waking at *next* commit? */
 	if (freezing(current)) {
 		/*
 		 * The simpler the better. Flushing journal isn't a
diff --git a/kernel/signal.c b/kernel/signal.c
index 717f1f3..6ef3c90 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -1922,6 +1922,8 @@ int get_signal_to_deliver(siginfo_t *inf
 	sigset_t *mask = &current->blocked;
 	int signr = 0;
 
+	try_to_freeze();
+
 relock:
 	spin_lock_irq(&current->sighand->siglock);
 	for (;;) {
@@ -2307,7 +2309,6 @@ sys_rt_sigtimedwait(const sigset_t __use
 
 			timeout = schedule_timeout_interruptible(timeout);
 
-			try_to_freeze();
 			spin_lock_irq(&current->sighand->siglock);
 			sig = dequeue_signal(current, &these, &info);
 			current->blocked = current->real_blocked;

diff --git a/arch/x86_64/kernel/signal.c b/arch/x86_64/kernel/signal.c
index 5876df1..f4dd2ca 100644
--- a/arch/x86_64/kernel/signal.c
+++ b/arch/x86_64/kernel/signal.c
@@ -443,9 +443,6 @@ int do_signal(struct pt_regs *regs, sigs
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
-		goto no_signal;
-
 	if (!oldset)
 		oldset = &current->blocked;
 

-- 
Thanks, Sharp!

Re: [ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Rafael J. Wysocki]

Hi,

On Tuesday 31 January 2006 04:24, Nigel Cunningham wrote:
> On Tuesday 31 January 2006 08:05, Rafael J. Wysocki wrote:
> > On Thursday 26 January 2006 04:45, Nigel Cunningham wrote:
> > > Disable usermode helper invocations when the freezer is on. This avoids
> > > deadlocks due to hotplug events occuring while processes are frozen.
> > >
> > > Signed-off-by: Nigel Cunningham <nigel@suspend2.net>
> > >
> > >  kernel/kmod.c |    4 ++++
> > >  1 files changed, 4 insertions(+), 0 deletions(-)
> > >
> > > diff --git a/kernel/kmod.c b/kernel/kmod.c
> > > index 51a8920..12afa2c 100644
> > > --- a/kernel/kmod.c
> > > +++ b/kernel/kmod.c
> > > @@ -36,6 +36,7 @@
> > >  #include <linux/mount.h>
> > >  #include <linux/kernel.h>
> > >  #include <linux/init.h>
> > > +#include <linux/freezer.h>
> > >  #include <asm/uaccess.h>
> > >
> > >  extern int max_threads;
> > > @@ -249,6 +250,9 @@ int call_usermodehelper_keys(char *path,
> > >  	if (!khelper_wq)
> > >  		return -EBUSY;
> > >
> > > +	if (freezer_is_on())
> > > +		return 0;
> > > +
> > >  	if (path[0] == '\0')
> > >  		return 0;
> >
> > Disabling the usermode helper while freeze_processes() is executed seems to
> > be a good idea to me, but I think it should be done with a mutex or
> > something like that.
> 
> With the refrigerator code you guys are using at the moment, ouldn't that 
> result in deadlocks when we later try to freeze the process in preparation 
> for the atomic restore? (Or perhaps you don't freeze processes at that 
> point?)

I'm not sure what you mean.  I said "mutex" because you seem to have a race
here (the freezer may be started right after the freezer_is_on() check).  IMO
the freezer should disable the invocations of new usermode helpers and
wait util all of the already running helpers are finished.  For this purpose
two variables would be needed and a lock.

Greetings,
Rafael

Re: [RFC/RFT] finally solve "swsusp fails with mysqld" problem

[Rafael J. Wysocki]

Hi,

On Tuesday 31 January 2006 10:27, Pavel Machek wrote:
> 
> Place refrigerator hook at more clever place; avoids "system can't be
> suspended while mysqld running" problem.
> 
> I'd like you to test it. It looks correct to me, and it is actually a
> solution, not a workaround like my previous tries. It still does not
> solve suspend while running stress tests.

Which kernel is it against?  It does not apply to the recent -mm ...

Rafael

Re: [RFC/RFT] finally solve "swsusp fails with mysqld" problem

[Pavel Machek]

Hi!

> > Place refrigerator hook at more clever place; avoids "system can't be
> > suspended while mysqld running" problem.
> > 
> > I'd like you to test it. It looks correct to me, and it is actually a
> > solution, not a workaround like my previous tries. It still does not
> > solve suspend while running stress tests.
> 
> Which kernel is it against?  It does not apply to the recent -mm ...

Against vanilla -- but it should be easy to make it work against -mm.

								Pavel
-- 
Thanks, Sharp!

Re: [RFC/RFT] finally solve "swsusp fails with mysqld" problem

[Rafael J. Wysocki]

Hi,

On Tuesday 31 January 2006 22:29, Pavel Machek wrote:
> > > Place refrigerator hook at more clever place; avoids "system can't be
> > > suspended while mysqld running" problem.
> > > 
> > > I'd like you to test it. It looks correct to me, and it is actually a
> > > solution, not a workaround like my previous tries. It still does not
> > > solve suspend while running stress tests.
> > 
> > Which kernel is it against?  It does not apply to the recent -mm ...
> 
> Against vanilla -- but it should be easy to make it work against -mm.

Yes, I've already done it.

BTW, the no_signal labels are no longer needed in signal.c for i386 and
x86_64.

Greetings,
Rafael

Re: [ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Nigel Cunningham]

[-- Attachment #1: Type: text/plain, Size: 2025 bytes --]

Hi.

On Tuesday 31 January 2006 20:58, Rafael J. Wysocki wrote:
> Hi,
> > > >
> > > > +	if (freezer_is_on())
> > > > +		return 0;
> > > > +
> > > >  	if (path[0] == '\0')
> > > >  		return 0;
> > >
> > > Disabling the usermode helper while freeze_processes() is executed
> > > seems to be a good idea to me, but I think it should be done with a
> > > mutex or something like that.
> >
> > With the refrigerator code you guys are using at the moment, ouldn't that
> > result in deadlocks when we later try to freeze the process in
> > preparation for the atomic restore? (Or perhaps you don't freeze
> > processes at that point?)
>
> I'm not sure what you mean.  I said "mutex" because you seem to have a race
> here (the freezer may be started right after the freezer_is_on() check). 
> IMO the freezer should disable the invocations of new usermode helpers and
> wait util all of the already running helpers are finished.  For this
> purpose two variables would be needed and a lock.

Sorry. Being a bit thick.

I wasn't worried about already-running usermode helpers (or about-to-run 
helpers either) because the spawned processes would either complete or be 
caught be the usual freezing code. My concern was more that new invocations 
of this path don't leave us with unfrozen processes hanging around. That 
said, I think you have a valid point about letting existing helpers run to 
completion. It does make me concerned though about the possibility of a 
long-lived helper (not that I know that there really are such things at the 
moment). Do you think that needs to be taken as a genuine concern? If so, I 
guess we then need to make freezing a four stage process:

1. Stop new usermodehelpers from starting & let existing ones run to 
completion.
2. Freeze userspace.
3. Freezer bdevs.
4. Freezer kernel space.

Regards,

Nigel
-- 
See our web page for Howtos, FAQs, the Wiki and mailing list info.
http://www.suspend2.net                IRC: #suspend2 on Freenode

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ 06/23] [Suspend2] Disable usermode helper invocations when the freezer is on.

[Rafael J. Wysocki]

Hi,

On Wednesday 01 February 2006 00:21, Nigel Cunningham wrote:
> On Tuesday 31 January 2006 20:58, Rafael J. Wysocki wrote:
> > Hi,
> > > > >
> > > > > +	if (freezer_is_on())
> > > > > +		return 0;
> > > > > +
> > > > >  	if (path[0] == '\0')
> > > > >  		return 0;
> > > >
> > > > Disabling the usermode helper while freeze_processes() is executed
> > > > seems to be a good idea to me, but I think it should be done with a
> > > > mutex or something like that.
> > >
> > > With the refrigerator code you guys are using at the moment, ouldn't that
> > > result in deadlocks when we later try to freeze the process in
> > > preparation for the atomic restore? (Or perhaps you don't freeze
> > > processes at that point?)
> >
> > I'm not sure what you mean.  I said "mutex" because you seem to have a race
> > here (the freezer may be started right after the freezer_is_on() check). 
> > IMO the freezer should disable the invocations of new usermode helpers and
> > wait util all of the already running helpers are finished.  For this
> > purpose two variables would be needed and a lock.
> 
> Sorry. Being a bit thick.
> 
> I wasn't worried about already-running usermode helpers (or about-to-run 
> helpers either) because the spawned processes would either complete or be 
> caught be the usual freezing code. My concern was more that new invocations 
> of this path don't leave us with unfrozen processes hanging around. That 
> said, I think you have a valid point about letting existing helpers run to 
> completion. It does make me concerned though about the possibility of a 
> long-lived helper (not that I know that there really are such things at the 
> moment). Do you think that needs to be taken as a genuine concern? If so, I 
> guess we then need to make freezing a four stage process:
> 
> 1. Stop new usermodehelpers from starting & let existing ones run to 
> completion.
> 2. Freeze userspace.
> 3. Freezer bdevs.
> 4. Freezer kernel space.

Well, I know a little about bdevs, but generally I think that's reasonable.

Greetings,
Rafael