[PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[Bojan Smojver]

Hi Rafael,

One more version. Heeding Per's suggestion to optimise when
CONFIG_HIGHMEM is not configured.

---------------------------------------
Hibernation/thaw fixes/improvements:

1. Calculate the number of required free pages based on non-high memory
pages only, because that is where the buffers will come from.

2. Do not allocate memory for buffers from emergency pools, unless
absolutely required. Do not warn about and do not retry non-essential
failed allocations.

3. Do not check the amount of free pages left on every single page
write, but wait until one map is completely populated and then check.

4. Set maximum number of pages for read buffering consistently, instead
of inadvertently depending on the size of the sector type.

5. Fix copyright line, which I missed when I submitted the hibernation
threading patch.

6. Dispense with bit shifting arithmetic to improve readability.

Signed-off-by: Bojan Smojver <bojan@rexursive.com>
Signed-off-by: Per Olofsson <pelle@debian.org>
---
 kernel/power/swap.c |   76 +++++++++++++++++++++++++++++++++++++++------------
 1 files changed, 58 insertions(+), 18 deletions(-)

diff --git a/kernel/power/swap.c b/kernel/power/swap.c
index 8742fd0..8a1c293 100644
--- a/kernel/power/swap.c
+++ b/kernel/power/swap.c
@@ -6,7 +6,7 @@
  *
  * Copyright (C) 1998,2001-2005 Pavel Machek <pavel@ucw.cz>
  * Copyright (C) 2006 Rafael J. Wysocki <rjw@sisk.pl>
- * Copyright (C) 2010 Bojan Smojver <bojan@rexursive.com>
+ * Copyright (C) 2010-2012 Bojan Smojver <bojan@rexursive.com>
  *
  * This file is released under the GPLv2.
  *
@@ -51,6 +51,36 @@
 
 #define MAP_PAGE_ENTRIES	(PAGE_SIZE / sizeof(sector_t) - 1)
 
+/*
+ * Number of free pages that are not high.
+ */
+#ifdef CONFIG_HIGHMEM
+static unsigned long low_free_pages(void)
+{
+	struct zone *zone;
+	unsigned long free = 0;
+
+	for_each_populated_zone(zone)
+		if (!is_highmem(zone))
+			free += zone_page_state(zone, NR_FREE_PAGES);
+	return free;
+}
+#else
+static inline unsigned long low_free_pages(void)
+{
+	return nr_free_pages();
+}
+#endif
+
+/*
+ * Number of pages required to be kept free while writing the image. Always
+ * half of all available low pages before the writing starts.
+ */
+static inline unsigned long reqd_free_pages(void)
+{
+	return low_free_pages() / 2;
+}
+
 struct swap_map_page {
 	sector_t entries[MAP_PAGE_ENTRIES];
 	sector_t next_swap;
@@ -72,7 +102,7 @@ struct swap_map_handle {
 	sector_t cur_swap;
 	sector_t first_sector;
 	unsigned int k;
-	unsigned long nr_free_pages, written;
+	unsigned long reqd_free_pages;
 	u32 crc32;
 };
 
@@ -265,14 +295,17 @@ static int write_page(void *buf, sector_t offset, struct bio **bio_chain)
 		return -ENOSPC;
 
 	if (bio_chain) {
-		src = (void *)__get_free_page(__GFP_WAIT | __GFP_HIGH);
+		src = (void *)__get_free_page(__GFP_WAIT | __GFP_NOWARN |
+		                              __GFP_NORETRY);
 		if (src) {
 			copy_page(src, buf);
 		} else {
 			ret = hib_wait_on_bio_chain(bio_chain); /* Free pages */
 			if (ret)
 				return ret;
-			src = (void *)__get_free_page(__GFP_WAIT | __GFP_HIGH);
+			src = (void *)__get_free_page(__GFP_WAIT |
+			                              __GFP_NOWARN |
+			                              __GFP_NORETRY);
 			if (src) {
 				copy_page(src, buf);
 			} else {
@@ -316,8 +349,7 @@ static int get_swap_writer(struct swap_map_handle *handle)
 		goto err_rel;
 	}
 	handle->k = 0;
-	handle->nr_free_pages = nr_free_pages() >> 1;
-	handle->written = 0;
+	handle->reqd_free_pages = reqd_free_pages();
 	handle->first_sector = handle->cur_swap;
 	return 0;
 err_rel:
@@ -351,12 +383,17 @@ static int swap_write_page(struct swap_map_handle *handle, void *buf,
 		clear_page(handle->cur);
 		handle->cur_swap = offset;
 		handle->k = 0;
-	}
-	if (bio_chain && ++handle->written > handle->nr_free_pages) {
-		error = hib_wait_on_bio_chain(bio_chain);
-		if (error)
-			goto out;
-		handle->written = 0;
+
+		if (bio_chain && low_free_pages() <= handle->reqd_free_pages) {
+			error = hib_wait_on_bio_chain(bio_chain);
+			if (error)
+				goto out;
+			/*
+			 * Recalculate the number of required free pages, to
+			 * make sure we never take more than half.
+			 */
+			handle->reqd_free_pages = reqd_free_pages();
+		}
 	}
  out:
 	return error;
@@ -404,7 +441,7 @@ static int swap_writer_finish(struct swap_map_handle *handle,
 #define LZO_THREADS	3
 
 /* Maximum number of pages for read buffering. */
-#define LZO_READ_PAGES	(MAP_PAGE_ENTRIES * 8)
+#define LZO_READ_PAGES	8192
 
 
 /**
@@ -615,10 +652,10 @@ static int save_image_lzo(struct swap_map_handle *handle,
 	}
 
 	/*
-	 * Adjust number of free pages after all allocations have been done.
-	 * We don't want to run out of pages when writing.
+	 * Adjust the number of required free pages after all allocations have
+	 * been done. We don't want to run out of pages when writing.
 	 */
-	handle->nr_free_pages = nr_free_pages() >> 1;
+	handle->reqd_free_pages = reqd_free_pages();
 
 	/*
 	 * Start the CRC32 thread.
@@ -1129,14 +1166,17 @@ static int load_image_lzo(struct swap_map_handle *handle,
 
 	/*
 	 * Adjust number of pages for read buffering, in case we are short.
+	 * Never take more than half of all available low pages.
 	 */
-	read_pages = (nr_free_pages() - snapshot_get_image_size()) >> 1;
+	read_pages = (low_free_pages() - snapshot_get_image_size()) / 2;
 	read_pages = clamp_val(read_pages, LZO_CMP_PAGES, LZO_READ_PAGES);
 
 	for (i = 0; i < read_pages; i++) {
 		page[i] = (void *)__get_free_page(i < LZO_CMP_PAGES ?
 		                                  __GFP_WAIT | __GFP_HIGH :
-		                                  __GFP_WAIT);
+		                                  __GFP_WAIT | __GFP_NOWARN |
+		                                  __GFP_NORETRY);
+
 		if (!page[i]) {
 			if (i < LZO_CMP_PAGES) {
 				ring_size = i;
---------------------------------------

-- 
Bojan

Re: [PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[Bojan Smojver]

Bojan Smojver <bojan@rexursive.com> wrote: 

>-	read_pages = (nr_free_pages() - snapshot_get_image_size()) >> 1;
>+	read_pages = (low_free_pages() - snapshot_get_image_size()) / 2;

Actually, I am pretty sure this part of the patch is wrong. Image can be bigger than the size of non-high memory. So, this calculation may turn up entirely bogus numbers, given we are dealing with unsigned values. Sure, we clamp that a line later, so all is not lost, but still, it needs to be fixed.

Will have a better version for you tomorrow.

-- 
Bojan

Re: [PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[Per Olofsson]

2012-04-12 13:31, Bojan Smojver skrev:
> Bojan Smojver <bojan@rexursive.com> wrote:
> 
>> -	read_pages = (nr_free_pages() - snapshot_get_image_size()) >> 1; 
>> +	read_pages = (low_free_pages() - snapshot_get_image_size()) / 2;
> 
> Actually, I am pretty sure this part of the patch is wrong. Image can
> be bigger than the size of non-high memory. So, this calculation may
> turn up entirely bogus numbers, given we are dealing with unsigned
> values. Sure, we clamp that a line later, so all is not lost, but
> still, it needs to be fixed.

Indeed. I think what you want is:

read_pages = min(low_free_pages(),
                 nr_free_pages() - snapshot_get_image_size()) / 2;

-- 
Pelle

Re: [PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[Bojan Smojver]

On Thu, 2012-04-12 at 18:30 +0200, Per Olofsson wrote:
> Indeed. I think what you want is:
> 
> read_pages = min(low_free_pages(),
>                  nr_free_pages() - snapshot_get_image_size()) / 2; 

I was thinking more like this:
----------------------
unsigned long read_pages = 0;

[...]

if (low_free_pages() > snapshot_get_image_size())
       read_pages = (low_free_pages() - snapshot_get_image_size()) / 2;
read_pages = clamp_val(read_pages, LZO_MIN_RD_PAGES, LZO_MAX_RD_PAGES);
----------------------

Where LZO_MIN_RD_PAGES and LZO_MAX_RD_PAGES are set to 1024 and 8192,
respectively (this was picked empirically).

Because we don't really know how many highmem pages are in the image
(this is figured out by prepare_image() function, half way through
reading the image - so way after this calculation is done), we assume
the worst case scenario. And that is that there are no highmem pages in
the image.

Given that we cannot use pages from highmem for buffers anyway, the
above should be careful enough. Of course, there is still some
possibility of running out of pages, but the kernel is usually in a
pretty good shape memory-wise on image load, so we should be able to
squeeze a few MBs out of it, at least.

-- 
Bojan

Re: [PATCH v10]: Hibernation: fix the number of pages used for hibernate/thaw buffering

[Per Olofsson]

2012-04-12 23:30, Bojan Smojver skrev:
> I was thinking more like this:
> ----------------------
> unsigned long read_pages = 0;
> 
> [...]
> 
> if (low_free_pages() > snapshot_get_image_size())
>        read_pages = (low_free_pages() - snapshot_get_image_size()) / 2;
> read_pages = clamp_val(read_pages, LZO_MIN_RD_PAGES, LZO_MAX_RD_PAGES);
> ----------------------
> 
> Where LZO_MIN_RD_PAGES and LZO_MAX_RD_PAGES are set to 1024 and 8192,
> respectively (this was picked empirically).
> 
> Because we don't really know how many highmem pages are in the image
> (this is figured out by prepare_image() function, half way through
> reading the image - so way after this calculation is done), we assume
> the worst case scenario. And that is that there are no highmem pages in
> the image.
> 
> Given that we cannot use pages from highmem for buffers anyway, the
> above should be careful enough. Of course, there is still some
> possibility of running out of pages, but the kernel is usually in a
> pretty good shape memory-wise on image load, so we should be able to
> squeeze a few MBs out of it, at least.

OK, I see your point now.

Yet another reason to switch to 64-bit then I guess :-)

-- 
Pelle