[PATCH 0/2] clarify limitations of hibernation

[PATCH 0/2] clarify limitations of hibernation

[Luigi Semenzato]

These patches aim to make it clearer under which circumstances
hibernation will function as expected.  They include one documentation
change, and one change which logs more information on failure to
enter hibernation.

(These were sent yesterday to linux-kernel only by accident.)

Luigi Semenzato (2):
  Documentation: clarify limitations of hibernation
  pm: add more logging on hibernation failure

 Documentation/admin-guide/pm/sleep-states.rst | 18 +++++++++++++++++-
 kernel/power/snapshot.c                       | 18 ++++++++++++------
 2 files changed, 29 insertions(+), 7 deletions(-)

-- 
2.24.1.735.g03f4e72817-goog

[PATCH 1/2] Documentation: clarify limitations of hibernation

[Luigi Semenzato]

Entering hibernation (suspend-to-disk) will fail if the kernel
cannot allocate enough memory to create a snapshot of all pages
in use; i.e., if memory in use is over 1/2 of total RAM.  This
patch makes this limitation clearer in the documentation.  Without
it, users may assume that hibernation can replace suspend-to-RAM
when in fact its functionality is more limited.

Signed-off-by: Luigi Semenzato <semenzato@google.com>
---
 Documentation/admin-guide/pm/sleep-states.rst | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/pm/sleep-states.rst b/Documentation/admin-guide/pm/sleep-states.rst
index cd3a28cb81f4..fd0072eb8c03 100644
--- a/Documentation/admin-guide/pm/sleep-states.rst
+++ b/Documentation/admin-guide/pm/sleep-states.rst
@@ -112,7 +112,9 @@ Hibernation
 This state (also referred to as Suspend-to-Disk or STD) offers the greatest
 energy savings and can be used even in the absence of low-level platform support
 for system suspend.  However, it requires some low-level code for resuming the
-system to be present for the underlying CPU architecture.
+system to be present for the underlying CPU architecture.  Additionally, the
+current implementation can enter the hibernation state only when memory
+pressure is low (see "Limitations" below).
 
 Hibernation is significantly different from any of the system suspend variants.
 It takes three system state changes to put it into hibernation and two system
@@ -149,6 +151,20 @@ Hibernation is supported if the :c:macro:`CONFIG_HIBERNATION` kernel
 configuration option is set.  However, this option can only be set if support
 for the given CPU architecture includes the low-level code for system resume.
 
+Limitations of Hibernation
+==========================
+
+When entering hibernation, the kernel tries to allocate a chunk of memory large
+enough to contain a copy of all pages in use, to use it for the system
+snapshot.  If the allocation fails, the system cannot hibernate and the
+operation fails with ENOMEM.  This will happen, for instance, when the total
+amount of anonymous pages (process data) exceeds 1/2 of total RAM.
+
+One possible workaround (besides terminating enough processes) is to force
+excess anonymous pages out to swap before hibernating.  This can be achieved
+with memcgroups, by lowering memory usage limits with ``echo <new limit> >
+/dev/cgroup/memory/<group>/memory.mem.usage_in_bytes``.  However, the latter
+operation is not guaranteed to succeed.
 
 Basic ``sysfs`` Interfaces for System Suspend and Hibernation
 =============================================================
-- 
2.24.1.735.g03f4e72817-goog

[PATCH 2/2] pm: add more logging on hibernation failure

[Luigi Semenzato]

Hibernation fails when the kernel cannot allocate enough memory
to copy all pages in use.  This patch ensures that the failure
reason is clearly logged.

Signed-off-by: Luigi Semenzato <semenzato@google.com>
---
 kernel/power/snapshot.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 26b9168321e7..df498717a97e 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1705,16 +1705,20 @@ int hibernate_preallocate_memory(void)
 	ktime_t start, stop;
 	int error;
 
-	pr_info("Preallocating image memory... ");
+	pr_info("Preallocating hibernation image memory\n");
 	start = ktime_get();
 
 	error = memory_bm_create(&orig_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate original bitmap\n");
 		goto err_out;
+	}
 
 	error = memory_bm_create(&copy_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate copy bitmap\n");
 		goto err_out;
+	}
 
 	alloc_normal = 0;
 	alloc_highmem = 0;
@@ -1804,8 +1808,11 @@ int hibernate_preallocate_memory(void)
 		alloc -= pages;
 		pages += pages_highmem;
 		pages_highmem = preallocate_image_highmem(alloc);
-		if (pages_highmem < alloc)
+		if (pages_highmem < alloc) {
+			pr_err("Image allocation is %lu pages short\n",
+				alloc - pages_highmem);
 			goto err_out;
+		}
 		pages += pages_highmem;
 		/*
 		 * size is the desired number of saveable pages to leave in
@@ -1836,13 +1843,12 @@ int hibernate_preallocate_memory(void)
 
  out:
 	stop = ktime_get();
-	pr_cont("done (allocated %lu pages)\n", pages);
+	pr_info("Allocated %lu pages for hibernation shapshot\n", pages);
 	swsusp_show_speed(start, stop, pages, "Allocated");
 
 	return 0;
 
  err_out:
-	pr_cont("\n");
 	swsusp_free();
 	return -ENOMEM;
 }
-- 
2.24.1.735.g03f4e72817-goog

Re: [PATCH 1/2] Documentation: clarify limitations of hibernation

[Chris Down]

Luigi Semenzato writes:
>+One possible workaround (besides terminating enough processes) is to force
>+excess anonymous pages out to swap before hibernating.  This can be achieved
>+with memcgroups, by lowering memory usage limits with ``echo <new limit> >

s/memcgroups/memory cgroups/

I also wouldn't say "excess", it implies that the cold(er) pages are 
unnecessary, which may not be true.

>+/dev/cgroup/memory/<group>/memory.mem.usage_in_bytes``.  However, the latter
>+operation is not guaranteed to succeed.

This is cgroup v1, which is frozen and in the process of being deprecated. We 
should at least mention memory.high or similar.

Also, /dev/cgroup? /sys/fs/cgroup has been more or less standardised for quite 
a while now...

(I also wonder whether the need for this in general is something the user 
shouldn't have to care about and we should improve management of in-kernel, but 
I know very little about hibernation so I'll leave that alone.)

Re: [PATCH 1/2] Documentation: clarify limitations of hibernation

[Chris Down]

Oh, and:

Luigi Semenzato writes:
>+system to be present for the underlying CPU architecture.  Additionally, the
>+current implementation can enter the hibernation state only when memory
>+pressure is low (see "Limitations" below).

This doesn't seem to make much sense to me. You claim it only works when 
pressure is low, but then suggest people raise pressure to fix it.

I suppose you mean "only when memory *usage* is low"? I mean, if memory 
pressure is low, then it's likely to result in these cold pages sitting around 
in the first place.

Re: [PATCH 1/2] Documentation: clarify limitations of hibernation

[Luigi Semenzato]

Thank you Chris, those are excellent comments.

I am replacing "memory pressure" with "memory usage".  I had used
"pressure" because "usage" is vague and here we specifically refer to
non-reclaimable pages.  But it's true that "pressure" has a different,
specific meaning for the mm, so "usage" is better.  The example should
clarify the problem.

I am also removing the workaround.  I thought it may be useful, but as
you said, it's quite debatable that it belongs here.

On Fri, Dec 27, 2019 at 4:25 PM Chris Down <chris@chrisdown.name> wrote:
>
> Oh, and:
>
> Luigi Semenzato writes:
> >+system to be present for the underlying CPU architecture.  Additionally, the
> >+current implementation can enter the hibernation state only when memory
> >+pressure is low (see "Limitations" below).
>
> This doesn't seem to make much sense to me. You claim it only works when
> pressure is low, but then suggest people raise pressure to fix it.
>
> I suppose you mean "only when memory *usage* is low"? I mean, if memory
> pressure is low, then it's likely to result in these cold pages sitting around
> in the first place.

[PATCH 2/2] pm: add more logging on hibernation failure

[Luigi Semenzato]

Hibernation fails when the kernel cannot allocate enough memory
to copy all pages in use.  This patch ensures that the failure
reason is clearly logged.

Signed-off-by: Luigi Semenzato <semenzato@google.com>
---
 kernel/power/snapshot.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 26b9168321e7..df498717a97e 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1705,16 +1705,20 @@ int hibernate_preallocate_memory(void)
 	ktime_t start, stop;
 	int error;
 
-	pr_info("Preallocating image memory... ");
+	pr_info("Preallocating hibernation image memory\n");
 	start = ktime_get();
 
 	error = memory_bm_create(&orig_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate original bitmap\n");
 		goto err_out;
+	}
 
 	error = memory_bm_create(&copy_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate copy bitmap\n");
 		goto err_out;
+	}
 
 	alloc_normal = 0;
 	alloc_highmem = 0;
@@ -1804,8 +1808,11 @@ int hibernate_preallocate_memory(void)
 		alloc -= pages;
 		pages += pages_highmem;
 		pages_highmem = preallocate_image_highmem(alloc);
-		if (pages_highmem < alloc)
+		if (pages_highmem < alloc) {
+			pr_err("Image allocation is %lu pages short\n",
+				alloc - pages_highmem);
 			goto err_out;
+		}
 		pages += pages_highmem;
 		/*
 		 * size is the desired number of saveable pages to leave in
@@ -1836,13 +1843,12 @@ int hibernate_preallocate_memory(void)
 
  out:
 	stop = ktime_get();
-	pr_cont("done (allocated %lu pages)\n", pages);
+	pr_info("Allocated %lu pages for hibernation shapshot\n", pages);
 	swsusp_show_speed(start, stop, pages, "Allocated");
 
 	return 0;
 
  err_out:
-	pr_cont("\n");
 	swsusp_free();
 	return -ENOMEM;
 }
-- 
2.24.1.735.g03f4e72817-goog

[PATCH 2/2] pm: add more logging on hibernation failure

[Luigi Semenzato]

Hibernation fails when the kernel cannot allocate enough memory
to copy all pages in use.  This patch ensures that the failure
reason is clearly logged.

Signed-off-by: Luigi Semenzato <semenzato@google.com>
---
 kernel/power/snapshot.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 26b9168321e7..df498717a97e 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1705,16 +1705,20 @@ int hibernate_preallocate_memory(void)
 	ktime_t start, stop;
 	int error;
 
-	pr_info("Preallocating image memory... ");
+	pr_info("Preallocating hibernation image memory\n");
 	start = ktime_get();
 
 	error = memory_bm_create(&orig_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate original bitmap\n");
 		goto err_out;
+	}
 
 	error = memory_bm_create(&copy_bm, GFP_IMAGE, PG_ANY);
-	if (error)
+	if (error) {
+		pr_err("Cannot allocate copy bitmap\n");
 		goto err_out;
+	}
 
 	alloc_normal = 0;
 	alloc_highmem = 0;
@@ -1804,8 +1808,11 @@ int hibernate_preallocate_memory(void)
 		alloc -= pages;
 		pages += pages_highmem;
 		pages_highmem = preallocate_image_highmem(alloc);
-		if (pages_highmem < alloc)
+		if (pages_highmem < alloc) {
+			pr_err("Image allocation is %lu pages short\n",
+				alloc - pages_highmem);
 			goto err_out;
+		}
 		pages += pages_highmem;
 		/*
 		 * size is the desired number of saveable pages to leave in
@@ -1836,13 +1843,12 @@ int hibernate_preallocate_memory(void)
 
  out:
 	stop = ktime_get();
-	pr_cont("done (allocated %lu pages)\n", pages);
+	pr_info("Allocated %lu pages for hibernation shapshot\n", pages);
 	swsusp_show_speed(start, stop, pages, "Allocated");
 
 	return 0;
 
  err_out:
-	pr_cont("\n");
 	swsusp_free();
 	return -ENOMEM;
 }
-- 
2.24.1.735.g03f4e72817-goog