[PATCH 0/6] adaptive readahead update

[PATCH 0/6] adaptive readahead update

[Fengguang Wu]

Andrew,

Here're more readahead updates. They go like:

--- broken-out/series	2007-01-05 13:13:19.000000000 +0800
+++ patches/series	2007-01-05 22:11:37.000000000 +0800
@@ -985,13 +985,17 @@
 readahead-sysctl-parameters.patch
 readahead-sysctl-parameters-use-ctl_unnumbered.patch
 readahead-sysctl-parameters-fix.patch
+readahead-sysctl-parameters-set-readahead_hit_rate-1.patch
 readahead-min-max-sizes.patch
+readahead-min-max-sizes-remove-get_readahead_bounds.patch
 readahead-state-based-method-aging-accounting.patch
 readahead-state-based-method-routines.patch
 readahead-state-based-method.patch
 readahead-context-based-method.patch
 readahead-context-based-method-locking-fix.patch
 readahead-context-based-method-locking-fix-2.patch
+readahead-context-based-method-update-ra_min.patch
+readahead-context-based-method-remove-readahead_ratio.patch
 readahead-initial-method-guiding-sizes.patch
 readahead-initial-method-thrashing-guard-size.patch
 readahead-initial-method-user-recommended-size.patch
@@ -1001,9 +1005,10 @@
 readahead-call-scheme.patch
 readahead-call-scheme-ifdef-fix.patch
 readahead-call-scheme-build-fix.patch
+readahead-call-scheme-remove-get_readahead_bounds.patch
-readahead-laptop-mode.patch
 readahead-loop-case.patch
 readahead-nfsd-case.patch
+readahead-nfsd-case-remove-ra_min.patch
 readahead-nfsd-case-fix.patch
 readahead-nfsd-case-fix-uninitialized-ra_min-ra_max.patch
 readahead-turn-on-by-default.patch


Summary of changes
==================

- change default value of readahead_hit_rate to 1
- remove defered readahead for laptop mode
- remove get_readahead_bounds() and update ra_min of the context method
- remove use of readahead_ratio in the context method


readahead-laptop-mode.patch
===========================

Please remove it for now.

It defers readahead when the laptop is spinned down, which may cause unpleasant
delays if the user is watching movie and the media player do not cache enough
data in userland.


readahead-nfsd-case-remove-ra_min.patch
=======================================

Please fold it immediately to avoid a compile error when bisecting.


readahead-insert-cond_resched-calls.patch
readahead-remove-size-limit-on-read_ahead_kb.patch                                                                      
readahead-remove-size-limit-of-max_sectors_kb-on-read_ahead_kb.patch                                                    
====================================================================

It may be better to push them to mainline _before_ the other patches.

They solve some issues on large readahead size.
The main users are currently some laptop mode users.

Regards,
Fengguang Wu
--

[PATCH 1/6] readahead: sysctl parameters: set readahead_hit_rate=1

[Fengguang Wu]

[-- Attachment #1: readahead-sysctl-parameters-set-readahead_hit_rate-1.patch --]
[-- Type: text/plain, Size: 804 bytes --]

Set default readahead_hit_rate to 1 for the majority users.

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>
---
 Documentation/sysctl/vm.txt |    2 +-
 mm/readahead.c              |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

--- linux.orig/Documentation/sysctl/vm.txt
+++ linux/Documentation/sysctl/vm.txt
@@ -261,4 +261,4 @@ Possible values can be:
 
 The larger value, the more capabilities, with more possible overheads.
 
-The default value is 0.
+The default value is 1.
--- linux.orig/mm/readahead.c
+++ linux/mm/readahead.c
@@ -42,7 +42,7 @@ int readahead_ratio = 50;
 EXPORT_SYMBOL_GPL(readahead_ratio);
 
 /* Readahead as long as cache hit ratio keeps above 1/##. */
-int readahead_hit_rate = 0;
+int readahead_hit_rate = 1;
 #endif /* CONFIG_ADAPTIVE_READAHEAD */
 
 /*

--

[PATCH 2/6] readahead: min/max sizes: remove get_readahead_bounds()

[Fengguang Wu]

[-- Attachment #1: readahead-min-max-sizes-remove-get_readahead_bounds.patch --]
[-- Type: text/plain, Size: 1283 bytes --]

Remove get_readahead_bounds():
- ra_max: we already have get_max_readahead() for it
- ra_min: is only used by context based readahead, and will be moved there
          and set to a more reasonable value

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>

---
 mm/readahead.c |   24 ------------------------
 1 file changed, 24 deletions(-)

--- linux-2.6.20-rc3-mm1.orig/mm/readahead.c
+++ linux-2.6.20-rc3-mm1/mm/readahead.c
@@ -780,30 +780,6 @@ out:
 	return nr_pages;
 }
 
-/*
- * ra_min is mainly determined by the size of cache memory. Reasonable?
- *
- * Table of concrete numbers for 4KB page size:
- *   inactive + free (MB):    4   8   16   32   64  128  256  512 1024
- *            ra_min (KB):   16  16   16   16   20   24   32   48   64
- */
-static inline void get_readahead_bounds(struct file_ra_state *ra,
-					unsigned long *ra_min,
-					unsigned long *ra_max)
-{
-	unsigned long active;
-	unsigned long inactive;
-	unsigned long free;
-
-	__get_zone_counts(&active, &inactive, &free, NODE_DATA(numa_node_id()));
-
-	free += inactive;
-	*ra_max = min(min(ra->ra_pages, 0xFFFFUL), free / 2);
-	*ra_min = min(min(MIN_RA_PAGES + (free >> 14),
-			  DIV_ROUND_UP(64*1024, PAGE_CACHE_SIZE)),
-			  *ra_max / 8);
-}
-
 #endif /* CONFIG_ADAPTIVE_READAHEAD */
 
 /*

--

[PATCH 3/6] readahead: context based method: update ra_min

[Fengguang Wu]

[-- Attachment #1: readahead-context-based-method-update-ra_min.patch --]
[-- Type: text/plain, Size: 3342 bytes --]

Classify the 4 cases into 2 classes, and assign proper ra_min for them.
Also update comments correspondly.

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>

---
 mm/readahead.c |   49 +++++++++++++++++++++++++++++++----------------
 1 file changed, 33 insertions(+), 16 deletions(-)

--- linux-2.6.20-rc3-mm1.orig/mm/readahead.c
+++ linux-2.6.20-rc3-mm1/mm/readahead.c
@@ -1285,17 +1285,26 @@ static int
 try_context_based_readahead(struct address_space *mapping,
 			struct file_ra_state *ra,
 			struct page *page, pgoff_t offset,
-			unsigned long ra_min, unsigned long ra_max)
+			unsigned long req_size, unsigned long ra_max)
 {
 	pgoff_t start;
+	unsigned long ra_min;
 	unsigned long ra_size;
 	unsigned long la_size;
 
-	/* Check if there is a segment of history pages, and its end index.
+	/*
+	 * Check if there is a segment of history pages, and its end index.
 	 * Based on which we decide whether and where to start read-ahead.
-	 *
-	 * Case 1: we have a current page.
-	 * 	   Search forward for a nearby hole.
+	 */
+
+	/*
+	 * Select a reasonable large initial size for sequential reads.
+	 */
+	ra_min = min(req_size * 4, mapping->backing_dev_info->ra_pages0);
+
+	/*
+	 * Case s1: we have a current page.
+	 * =======> Search forward for a nearby hole.
 	 */
 	read_lock_irq(&mapping->tree_lock);
 	if (page) {
@@ -1308,8 +1317,9 @@ try_context_based_readahead(struct addre
 		return -1;
 	}
 
-	/* Case 2: current page is missing; previous page is present.
-	 *         Just do read-ahead from the current index on.
+	/*
+	 * Case s2: current page is missing; previous page is present.
+	 * =======> Just do read-ahead from the current index on.
 	 * There's clear sign of sequential reading. It can be
 	 * 	a) seek => read => this read
 	 * 	b) cache hit read(s) => this read
@@ -1321,26 +1331,33 @@ try_context_based_readahead(struct addre
 		goto has_history_pages;
 	}
 
-	/* Case 2x: the same context info as 2.
-	 * It can be the early stage of semi-sequential reads(interleaved/nfsd),
-	 * or an ugly random one.  So be conservative.
+	/*
+	 * Not an obvious sequential read:
+	 * select a conservative initial size, plus user prefered agressiveness.
+	 */
+	ra_min = min(req_size, MIN_RA_PAGES) +
+		 readahead_hit_rate * 8192 / PAGE_CACHE_SIZE;
+
+	/*
+	 * Case r1: the same context info as s2, but not that obvious.
+	 * =======> The same action as s2, but be conservative.
+	 * It can be the early stage of intermixed sequential reads,
+	 * or an ugly random one.
 	 */
 	if (readahead_hit_rate && __probe_page(mapping, offset - 1)) {
 		start = offset;
-		if (ra_min > 2 * readahead_hit_rate)
-		    ra_min = 2 * readahead_hit_rate;
 		goto has_history_pages;
 	}
 
-	/* Case 3: no current/previous pages;
-	 *         sparse read-ahead is enabled: ok, be aggressive.
-	 *         Check if there's any adjecent history pages.
+	/*
+	 * Case r2: no current/previous pages; sparse read-ahead is enabled.
+	 * =======> Do sparse read-ahead if there are adjecent history pages.
 	 */
 	if (readahead_hit_rate > 1) {
 		start = radix_tree_scan_data_backward(&mapping->page_tree,
 							       offset, ra_min);
 		if (start != ULONG_MAX && offset - start < ra_min) {
-			ra_min += offset - start;
+			ra_min *= 2;
 			offset = ++start; /* pretend the request starts here */
 			goto has_history_pages;
 		}

--

[PATCH 4/6] readahead: context based method: remove readahead_ratio

[Fengguang Wu]

[-- Attachment #1: readahead-context-based-method-remove-readahead_ratio.patch --]
[-- Type: text/plain, Size: 1104 bytes --]

The context based readahead is pretty conservative by nature,
so do not apply readahead_ratio here.

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>

---
 mm/readahead.c |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

--- linux-2.6.20-rc3-mm1.orig/mm/readahead.c
+++ linux-2.6.20-rc3-mm1/mm/readahead.c
@@ -1236,9 +1236,7 @@ static unsigned long count_history_pages
 	 * Check the far pages coarsely.
 	 * The enlarged count will contribute to the look-ahead size.
 	 */
-	lookback = ra_max * (LOOKAHEAD_RATIO + 1) *
-						100 / (readahead_ratio | 1);
-
+	lookback = ra_max * LOOKAHEAD_RATIO;
 	for (count += ra_max; count < lookback; count += ra_max)
 		if (!__probe_page(mapping, offset - count))
 			break;
@@ -1384,7 +1382,8 @@ has_history_pages:
 		adjust_rala_aggressive(ra_max, &ra_size, &la_size);
 		ra_set_class(ra, RA_CLASS_CONTEXT_AGGRESSIVE);
 	} else {
-		ra_size = max(ra_min, ra_size * readahead_ratio / 100);
+		if (ra_size < ra_min)
+		    ra_size = ra_min;
 		if (!adjust_rala(ra_max, &ra_size, &la_size))
 			return -1;
 		ra_set_class(ra, RA_CLASS_CONTEXT);

--

[PATCH 5/6] readahead: call scheme: remove get_readahead_bounds()

[Fengguang Wu]

[-- Attachment #1: readahead-call-scheme-remove-get_readahead_bounds.patch --]
[-- Type: text/plain, Size: 1070 bytes --]

Remove the one and only get_readahead_bounds() call.

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>

---
 mm/readahead.c |    5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

--- linux-2.6.20-rc3-mm1.orig/mm/readahead.c
+++ linux-2.6.20-rc3-mm1/mm/readahead.c
@@ -1572,7 +1572,6 @@ page_cache_readahead_adaptive(struct add
 				pgoff_t offset, unsigned long req_size)
 {
 	unsigned long ra_size;
-	unsigned long ra_min;
 	unsigned long ra_max;
 	int ret;
 
@@ -1593,7 +1592,7 @@ page_cache_readahead_adaptive(struct add
 	else if (offset)
 		ra_account(ra, RA_EVENT_CACHE_MISS, req_size);
 
-	get_readahead_bounds(ra, &ra_min, &ra_max);
+	ra_max = get_max_readahead(ra);
 
 	/* read-ahead disabled? */
 	if (unlikely(!ra_max || !readahead_ratio)) {
@@ -1633,7 +1632,7 @@ page_cache_readahead_adaptive(struct add
 	 * Context based sequential read-ahead.
 	 */
 	ret = try_context_based_readahead(mapping, ra, page,
-							offset, ra_min, ra_max);
+						offset, req_size, ra_max);
 	if (ret > 0)
 		return ra_submit(ra, mapping, filp);
 	if (ret < 0)

--

[PATCH 6/6] readahead: nfsd case: remove ra_min

[Fengguang Wu]

[-- Attachment #1: readahead-nfsd-case-remove-ra_min.patch --]
[-- Type: text/plain, Size: 692 bytes --]

ra_min => req_size.
Now it is for try_context_based_readahead() to compute ra_min from req_size.

Please fold it into readahead-nfsd-case.patch to avoid a compiling error.

Signed-off-by: Fengguang Wu <wfg@mail.ustc.edu.cn>
---
 mm/readahead.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- linux-2.6.20-rc3-mm1.orig/mm/readahead.c
+++ linux-2.6.20-rc3-mm1/mm/readahead.c
@@ -1673,7 +1673,7 @@ readit:
 		   !probe_page(mapping, ra_index)) {
 			ra->prev_page = ra_index - 1;
 			ret = try_context_based_readahead(mapping, ra, NULL,
-						 ra_index, ra_min, ra_max);
+						 ra_index, req_size, ra_max);
 			if (ret > 0)
 				ra_size += ra_submit(ra, mapping, filp);
 		}

--