Isolated(anon) and Isolated(file)

Isolated(anon) and Isolated(file)

[Hugh Dickins]

Hi KOSAKI-san,

May I question the addition of Isolated(anon) and Isolated(file)
lines to /proc/meminfo?  I get irritated by all such "0 kB" lines!

I see their appropriateness and usefulness in the Alt-Sysrq-M-style
info which accompanies an OOM; and I see that those statistics help
you to identify and fix bugs of having too many pages isolated.

But IMHO they're too transient to be appropriate in /proc/meminfo:
by the time the "cat /proc/meminfo" is done, the situation is very
different (or should be once the bugs are fixed).

Almost all its numbers are transient, of course, but these seem
so much so that I think /proc/meminfo is better off without them
(compressing more info into fewer lines).

Perhaps I'm in the minority: if others care, what do they think?

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Minchan Kim]

On Sun, 13 Sep 2009 20:42:38 +0100 (BST)
Hugh Dickins <hugh.dickins@tiscali.co.uk> wrote:

> Hi KOSAKI-san,
> 
> May I question the addition of Isolated(anon) and Isolated(file)
> lines to /proc/meminfo?  I get irritated by all such "0 kB" lines!
> 
> I see their appropriateness and usefulness in the Alt-Sysrq-M-style
> info which accompanies an OOM; and I see that those statistics help
> you to identify and fix bugs of having too many pages isolated.

Right. 
 
> But IMHO they're too transient to be appropriate in /proc/meminfo:
> by the time the "cat /proc/meminfo" is done, the situation is very
> different (or should be once the bugs are fixed).

I agree. 
 
> Almost all its numbers are transient, of course, but these seem
> so much so that I think /proc/meminfo is better off without them
> (compressing more info into fewer lines).
> 
> Perhaps I'm in the minority: if others care, what do they think?

At that time, we need isolated page count per zone. 
So we added it in zone_stat_item. 

As you know, most of zone_stat_item are fields of meminfo. 
So, I supported it as part of meminfo without serious thinking.
 
Now I agree with your opinion. 

It's very transient so it is valuable when OOM or Sysrq happens. 
If you get irritated by it, we can remove things related to meminfo 
but keep isolated count, then when we meets OOM, we can show it. 

Let's listen to others.

> Hugh


-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Wu Fengguang]

On Mon, Sep 14, 2009 at 03:42:38AM +0800, Hugh Dickins wrote:
> Hi KOSAKI-san,
> 
> May I question the addition of Isolated(anon) and Isolated(file)
> lines to /proc/meminfo?  I get irritated by all such "0 kB" lines!
> 
> I see their appropriateness and usefulness in the Alt-Sysrq-M-style
> info which accompanies an OOM; and I see that those statistics help
> you to identify and fix bugs of having too many pages isolated.
> 
> But IMHO they're too transient to be appropriate in /proc/meminfo:
> by the time the "cat /proc/meminfo" is done, the situation is very
> different (or should be once the bugs are fixed).
> 
> Almost all its numbers are transient, of course, but these seem
> so much so that I think /proc/meminfo is better off without them
> (compressing more info into fewer lines).
> 
> Perhaps I'm in the minority: if others care, what do they think?

Hugh, I tend to agree with you.

Typically one will have difficulty running the "cat /proc/meminfo"
command when isolated numbers are large, because that means so many
processes running that 'cat' cannot be scheduled for very long time.

If anywhere, /proc/vmstat (which has more "advanced" numbers) or
/sys/devices/system/node/node0/meminfo (which is less visited by
human?) may be more suitable place for the isolated numbers.

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[KOSAKI Motohiro]

Hi

> Hi KOSAKI-san,
> 
> May I question the addition of Isolated(anon) and Isolated(file)
> lines to /proc/meminfo?  I get irritated by all such "0 kB" lines!
> 
> I see their appropriateness and usefulness in the Alt-Sysrq-M-style
> info which accompanies an OOM; and I see that those statistics help
> you to identify and fix bugs of having too many pages isolated.
> 
> But IMHO they're too transient to be appropriate in /proc/meminfo:
> by the time the "cat /proc/meminfo" is done, the situation is very
> different (or should be once the bugs are fixed).
> 
> Almost all its numbers are transient, of course, but these seem
> so much so that I think /proc/meminfo is better off without them
> (compressing more info into fewer lines).
> 
> Perhaps I'm in the minority: if others care, what do they think?

I think Alt-Sysrq-M isn't useful in this case. because, if heavy memory
pressure occur, the administrator can't input "echo > /proc/sysrq-trigger"
to his terminal.
In the otherhand, many system get /proc/meminfo per every second. then,
the administrator can see last got statistics.

However, I halfly agree with you. Isolated field is transient value.
In almost case, it display 0kB. it is a bit annoy.

Fortunately, now /proc/vmstat and /sys/device/system/node/meminfo also
can display isolated value.
(As far as I rememberd, it was implemented by Wu's request)
We can use it. IOW, we can remove isolated field from /proc/meminfo.


How about following patch?


========== CUT HERE ===============================
From 7aa6fa2b76ff5d063b8bfa4a3af38c39b9396fd5 Mon Sep 17 00:00:00 2001
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Date: Tue, 15 Sep 2009 10:16:51 +0900
Subject: [PATCH] Kill Isolated field in /proc/meminfo

Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
It is only increased at heavy memory pressure case.

So, if the system haven't get memory pressure, this field isn't useful.
And now, we have two alternative way, /sys/device/system/node/node{n}/meminfo
and /prov/vmstat. Then, it can be removed.

Reported-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 fs/proc/meminfo.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index 7d46c2e..c7bff4f 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -65,8 +65,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 		"Active(file):   %8lu kB\n"
 		"Inactive(file): %8lu kB\n"
 		"Unevictable:    %8lu kB\n"
-		"Isolated(anon): %8lu kB\n"
-		"Isolated(file): %8lu kB\n"
 		"Mlocked:        %8lu kB\n"
 #ifdef CONFIG_HIGHMEM
 		"HighTotal:      %8lu kB\n"
@@ -116,8 +114,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 		K(pages[LRU_ACTIVE_FILE]),
 		K(pages[LRU_INACTIVE_FILE]),
 		K(pages[LRU_UNEVICTABLE]),
-		K(global_page_state(NR_ISOLATED_ANON)),
-		K(global_page_state(NR_ISOLATED_FILE)),
 		K(global_page_state(NR_MLOCK)),
 #ifdef CONFIG_HIGHMEM
 		K(i.totalhigh),
-- 
1.6.2.5




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Minchan Kim]

On Tue, Sep 15, 2009 at 11:56 AM, KOSAKI Motohiro
<kosaki.motohiro@jp.fujitsu.com> wrote:
> Hi
>
>> Hi KOSAKI-san,
>>
>> May I question the addition of Isolated(anon) and Isolated(file)
>> lines to /proc/meminfo?  I get irritated by all such "0 kB" lines!
>>
>> I see their appropriateness and usefulness in the Alt-Sysrq-M-style
>> info which accompanies an OOM; and I see that those statistics help
>> you to identify and fix bugs of having too many pages isolated.
>>
>> But IMHO they're too transient to be appropriate in /proc/meminfo:
>> by the time the "cat /proc/meminfo" is done, the situation is very
>> different (or should be once the bugs are fixed).
>>
>> Almost all its numbers are transient, of course, but these seem
>> so much so that I think /proc/meminfo is better off without them
>> (compressing more info into fewer lines).
>>
>> Perhaps I'm in the minority: if others care, what do they think?
>
> I think Alt-Sysrq-M isn't useful in this case. because, if heavy memory
> pressure occur, the administrator can't input "echo > /proc/sysrq-trigger"
> to his terminal.
> In the otherhand, many system get /proc/meminfo per every second. then,
> the administrator can see last got statistics.
>
> However, I halfly agree with you. Isolated field is transient value.
> In almost case, it display 0kB. it is a bit annoy.
>
> Fortunately, now /proc/vmstat and /sys/device/system/node/meminfo also
> can display isolated value.
> (As far as I rememberd, it was implemented by Wu's request)
> We can use it. IOW, we can remove isolated field from /proc/meminfo.
>
>
> How about following patch?
>
>
> ========== CUT HERE ===============================
> From 7aa6fa2b76ff5d063b8bfa4a3af38c39b9396fd5 Mon Sep 17 00:00:00 2001
> From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Date: Tue, 15 Sep 2009 10:16:51 +0900
> Subject: [PATCH] Kill Isolated field in /proc/meminfo
>
> Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> It is only increased at heavy memory pressure case.
>
> So, if the system haven't get memory pressure, this field isn't useful.
> And now, we have two alternative way, /sys/device/system/node/node{n}/meminfo
> and /prov/vmstat. Then, it can be removed.
>
> Reported-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Reviewed-by: Minchan Kim <minchan.kim@gmail.com>


-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Wu Fengguang]

On Tue, Sep 15, 2009 at 10:56:27AM +0800, KOSAKI Motohiro wrote:
> From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Date: Tue, 15 Sep 2009 10:16:51 +0900
> Subject: [PATCH] Kill Isolated field in /proc/meminfo
> 
> Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> It is only increased at heavy memory pressure case.
> 
> So, if the system haven't get memory pressure, this field isn't useful.
> And now, we have two alternative way, /sys/device/system/node/node{n}/meminfo
> and /prov/vmstat. Then, it can be removed.
> 
> Reported-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Acked-by: Wu Fengguang <fengguang.wu@intel.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Hugh Dickins]

On Tue, 15 Sep 2009, KOSAKI Motohiro wrote:
> From 7aa6fa2b76ff5d063b8bfa4a3af38c39b9396fd5 Mon Sep 17 00:00:00 2001
> From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Date: Tue, 15 Sep 2009 10:16:51 +0900
> Subject: [PATCH] Kill Isolated field in /proc/meminfo
> 
> Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> It is only increased at heavy memory pressure case.
> 
> So, if the system haven't get memory pressure, this field isn't useful.
> And now, we have two alternative way, /sys/device/system/node/node{n}/meminfo
> and /prov/vmstat. Then, it can be removed.
> 
> Reported-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Acked-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>

I should be overjoyed that you agree to hide the Isolateds from my sight:
thank you.  But in fact I'm a little depressed, now you've reminded me of
almost-the-same-but-annoyingly-different /sys/devices/unmemorable/meminfo.

Oh well, since I never see it (I'd need some nodes), I guess I don't
even need to turn a blind eye to it; and it already contains other
stuff I objected to in /proc/meminfo.

I still think your Isolateds make most sense in the OOM display;
and yes, they are there in /proc/vmstat, that's good too.

> ---
>  fs/proc/meminfo.c |    4 ----
>  1 files changed, 0 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
> index 7d46c2e..c7bff4f 100644
> --- a/fs/proc/meminfo.c
> +++ b/fs/proc/meminfo.c
> @@ -65,8 +65,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>  		"Active(file):   %8lu kB\n"
>  		"Inactive(file): %8lu kB\n"
>  		"Unevictable:    %8lu kB\n"
> -		"Isolated(anon): %8lu kB\n"
> -		"Isolated(file): %8lu kB\n"
>  		"Mlocked:        %8lu kB\n"
>  #ifdef CONFIG_HIGHMEM
>  		"HighTotal:      %8lu kB\n"
> @@ -116,8 +114,6 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
>  		K(pages[LRU_ACTIVE_FILE]),
>  		K(pages[LRU_INACTIVE_FILE]),
>  		K(pages[LRU_UNEVICTABLE]),
> -		K(global_page_state(NR_ISOLATED_ANON)),
> -		K(global_page_state(NR_ISOLATED_FILE)),
>  		K(global_page_state(NR_MLOCK)),
>  #ifdef CONFIG_HIGHMEM
>  		K(i.totalhigh),
> -- 
> 1.6.2.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[KOSAKI Motohiro]

> On Tue, 15 Sep 2009, KOSAKI Motohiro wrote:
> > From 7aa6fa2b76ff5d063b8bfa4a3af38c39b9396fd5 Mon Sep 17 00:00:00 2001
> > From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> > Date: Tue, 15 Sep 2009 10:16:51 +0900
> > Subject: [PATCH] Kill Isolated field in /proc/meminfo
> > 
> > Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> > It is only increased at heavy memory pressure case.
> > 
> > So, if the system haven't get memory pressure, this field isn't useful.
> > And now, we have two alternative way, /sys/device/system/node/node{n}/meminfo
> > and /prov/vmstat. Then, it can be removed.
> > 
> > Reported-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> > Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> 
> Acked-by: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> 
> I should be overjoyed that you agree to hide the Isolateds from my sight:
> thank you.  But in fact I'm a little depressed, now you've reminded me of
> almost-the-same-but-annoyingly-different /sys/devices/unmemorable/meminfo.
> 
> Oh well, since I never see it (I'd need some nodes), I guess I don't
> even need to turn a blind eye to it; and it already contains other
> stuff I objected to in /proc/meminfo.
> 
> I still think your Isolateds make most sense in the OOM display;
> and yes, they are there in /proc/vmstat, that's good too.

Hmm..
You touch different another problem. /proc/vmstat don't have sufficient
capability. Recently average #-of-cpus and numa rapidly become common.
Then, many administrator decide to run two or more each unrelated workload
on one machine and they are separated by cpuset.

However, /proc/vmstat don't have per-numa nor per-cpuset capability. then,
We lost a way of getting vm statistics.

<btw>
Another several user want per-memcgroup statistics. another user want another.
Several month ago, Ingo proposed object based tracing framework and /proc/vmstat
replace with it. I like his idea. but flexibility statiscis decrease
system performnce a bit. My brain haven't got the answer.
</btw>


Anyway, we need to decide drop or not before merge the patch. unfortunately
current is merge window.
Then, I decide to drop /sys/devices/node/meminfo too at once. Perhaps I'll
resubmit the same patch after (Of cource, perhaps not), but We know field
adding is feature enhancement but field removing is regression.

Andrew, very sorry, could you please pick up following patch and it merge
my last patch? the patch description also be rewritten.


========================================================
From 094c314ba851171d8201f4446783341ea0d22141 Mon Sep 17 00:00:00 2001
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Date: Wed, 16 Sep 2009 09:22:44 +0900
Subject: [PATCH] Kill Isolated field in /proc/meminfo fix

Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
It is only increased at heavy memory pressure case.

So, if the system haven't get memory pressure, this field isn't useful.
And now, we have an alternative way, (i.e. /prov/vmstat).
Then, it can be removed.

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 drivers/base/node.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/drivers/base/node.c b/drivers/base/node.c
index f50621b..1fe5536 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -73,8 +73,6 @@ static ssize_t node_read_meminfo(struct sys_device * dev,
 		       "Node %d Active(file):   %8lu kB\n"
 		       "Node %d Inactive(file): %8lu kB\n"
 		       "Node %d Unevictable:    %8lu kB\n"
-		       "Node %d Isolated(anon): %8lu kB\n"
-		       "Node %d Isolated(file): %8lu kB\n"
 		       "Node %d Mlocked:        %8lu kB\n"
 #ifdef CONFIG_HIGHMEM
 		       "Node %d HighTotal:      %8lu kB\n"
@@ -108,8 +106,6 @@ static ssize_t node_read_meminfo(struct sys_device * dev,
 		       nid, K(node_page_state(nid, NR_ACTIVE_FILE)),
 		       nid, K(node_page_state(nid, NR_INACTIVE_FILE)),
 		       nid, K(node_page_state(nid, NR_UNEVICTABLE)),
-		       nid, K(node_page_state(nid, NR_ISOLATED_ANON)),
-		       nid, K(node_page_state(nid, NR_ISOLATED_FILE)),
 		       nid, K(node_page_state(nid, NR_MLOCK)),
 #ifdef CONFIG_HIGHMEM
 		       nid, K(i.totalhigh),
-- 
1.6.2.5



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Andrew Morton]

On Wed, 16 Sep 2009 11:09:54 +0900 (JST) KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:

> Subject: [PATCH] Kill Isolated field in /proc/meminfo fix
> 
> Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> It is only increased at heavy memory pressure case.

Have we made up our minds yet?

Below is what remains.  Please check that the changelog is still
accurate and complete.  If not, please send along a new one?



From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

If the system is running a heavy load of processes then concurrent reclaim
can isolate a large number of pages from the LRU. /proc/meminfo and the
output generated for an OOM do not show how many pages were isolated.

This has been observed during process fork bomb testing (mstctl11 in LTP).

This patch shows the information about isolated pages.

Reproduced via:

-----------------------
% ./hackbench 140 process 1000
   => OOM occur

active_anon:146 inactive_anon:0 isolated_anon:49245
 active_file:79 inactive_file:18 isolated_file:113
 unevictable:0 dirty:0 writeback:0 unstable:0 buffer:39
 free:370 slab_reclaimable:309 slab_unreclaimable:5492
 mapped:53 shmem:15 pagetables:28140 bounce:0

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Wu Fengguang <fengguang.wu@intel.com>
Reviewed-by: Minchan Kim <minchan.kim@gmail.com>
Cc: Hugh Dickins <hugh.dickins@tiscali.co.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/mmzone.h |    2 ++
 mm/migrate.c           |   11 +++++++++++
 mm/page_alloc.c        |   12 +++++++++---
 mm/vmscan.c            |   12 +++++++++++-
 mm/vmstat.c            |    2 ++
 5 files changed, 35 insertions(+), 4 deletions(-)

diff -puN drivers/base/node.c~mm-vmstat-add-isolate-pages drivers/base/node.c
diff -puN fs/proc/meminfo.c~mm-vmstat-add-isolate-pages fs/proc/meminfo.c
diff -puN include/linux/mmzone.h~mm-vmstat-add-isolate-pages include/linux/mmzone.h
--- a/include/linux/mmzone.h~mm-vmstat-add-isolate-pages
+++ a/include/linux/mmzone.h
@@ -100,6 +100,8 @@ enum zone_stat_item {
 	NR_BOUNCE,
 	NR_VMSCAN_WRITE,
 	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
+	NR_ISOLATED_ANON,	/* Temporary isolated pages from anon lru */
+	NR_ISOLATED_FILE,	/* Temporary isolated pages from file lru */
 	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
 #ifdef CONFIG_NUMA
 	NUMA_HIT,		/* allocated in intended node */
diff -puN mm/migrate.c~mm-vmstat-add-isolate-pages mm/migrate.c
--- a/mm/migrate.c~mm-vmstat-add-isolate-pages
+++ a/mm/migrate.c
@@ -67,6 +67,8 @@ int putback_lru_pages(struct list_head *
 
 	list_for_each_entry_safe(page, page2, l, lru) {
 		list_del(&page->lru);
+		dec_zone_page_state(page, NR_ISOLATED_ANON +
+				    !!page_is_file_cache(page));
 		putback_lru_page(page);
 		count++;
 	}
@@ -698,6 +700,8 @@ unlock:
  		 * restored.
  		 */
  		list_del(&page->lru);
+		dec_zone_page_state(page, NR_ISOLATED_ANON +
+				    !!page_is_file_cache(page));
 		putback_lru_page(page);
 	}
 
@@ -742,6 +746,13 @@ int migrate_pages(struct list_head *from
 	struct page *page2;
 	int swapwrite = current->flags & PF_SWAPWRITE;
 	int rc;
+	unsigned long flags;
+
+	local_irq_save(flags);
+	list_for_each_entry(page, from, lru)
+		__inc_zone_page_state(page, NR_ISOLATED_ANON +
+				      !!page_is_file_cache(page));
+	local_irq_restore(flags);
 
 	if (!swapwrite)
 		current->flags |= PF_SWAPWRITE;
diff -puN mm/page_alloc.c~mm-vmstat-add-isolate-pages mm/page_alloc.c
--- a/mm/page_alloc.c~mm-vmstat-add-isolate-pages
+++ a/mm/page_alloc.c
@@ -2152,16 +2152,18 @@ void show_free_areas(void)
 		}
 	}
 
-	printk("Active_anon:%lu active_file:%lu inactive_anon:%lu\n"
-		" inactive_file:%lu"
+	printk("active_anon:%lu inactive_anon:%lu isolated_anon:%lu\n"
+		" active_file:%lu inactive_file:%lu isolated_file:%lu\n"
 		" unevictable:%lu"
 		" dirty:%lu writeback:%lu unstable:%lu buffer:%lu\n"
 		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
 		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n",
 		global_page_state(NR_ACTIVE_ANON),
-		global_page_state(NR_ACTIVE_FILE),
 		global_page_state(NR_INACTIVE_ANON),
+		global_page_state(NR_ISOLATED_ANON),
+		global_page_state(NR_ACTIVE_FILE),
 		global_page_state(NR_INACTIVE_FILE),
+		global_page_state(NR_ISOLATED_FILE),
 		global_page_state(NR_UNEVICTABLE),
 		global_page_state(NR_FILE_DIRTY),
 		global_page_state(NR_WRITEBACK),
@@ -2189,6 +2191,8 @@ void show_free_areas(void)
 			" active_file:%lukB"
 			" inactive_file:%lukB"
 			" unevictable:%lukB"
+			" isolated(anon):%lukB"
+			" isolated(file):%lukB"
 			" present:%lukB"
 			" mlocked:%lukB"
 			" dirty:%lukB"
@@ -2215,6 +2219,8 @@ void show_free_areas(void)
 			K(zone_page_state(zone, NR_ACTIVE_FILE)),
 			K(zone_page_state(zone, NR_INACTIVE_FILE)),
 			K(zone_page_state(zone, NR_UNEVICTABLE)),
+			K(zone_page_state(zone, NR_ISOLATED_ANON)),
+			K(zone_page_state(zone, NR_ISOLATED_FILE)),
 			K(zone->present_pages),
 			K(zone_page_state(zone, NR_MLOCK)),
 			K(zone_page_state(zone, NR_FILE_DIRTY)),
diff -puN mm/vmscan.c~mm-vmstat-add-isolate-pages mm/vmscan.c
--- a/mm/vmscan.c~mm-vmstat-add-isolate-pages
+++ a/mm/vmscan.c
@@ -1072,6 +1072,8 @@ static unsigned long shrink_inactive_lis
 		unsigned long nr_active;
 		unsigned int count[NR_LRU_LISTS] = { 0, };
 		int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
+		unsigned long nr_anon;
+		unsigned long nr_file;
 
 		nr_taken = sc->isolate_pages(sc->swap_cluster_max,
 			     &page_list, &nr_scan, sc->order, mode,
@@ -1102,6 +1104,10 @@ static unsigned long shrink_inactive_lis
 		__mod_zone_page_state(zone, NR_INACTIVE_ANON,
 						-count[LRU_INACTIVE_ANON]);
 
+		nr_anon = count[LRU_ACTIVE_ANON] + count[LRU_INACTIVE_ANON];
+		nr_file = count[LRU_ACTIVE_FILE] + count[LRU_INACTIVE_FILE];
+		__mod_zone_page_state(zone, NR_ISOLATED_ANON, nr_anon);
+		__mod_zone_page_state(zone, NR_ISOLATED_FILE, nr_file);
 
 		reclaim_stat->recent_scanned[0] += count[LRU_INACTIVE_ANON];
 		reclaim_stat->recent_scanned[0] += count[LRU_ACTIVE_ANON];
@@ -1169,6 +1175,9 @@ static unsigned long shrink_inactive_lis
 				spin_lock_irq(&zone->lru_lock);
 			}
 		}
+		__mod_zone_page_state(zone, NR_ISOLATED_ANON, -nr_anon);
+		__mod_zone_page_state(zone, NR_ISOLATED_FILE, -nr_file);
+
   	} while (nr_scanned < max_scan);
 
 done:
@@ -1279,6 +1288,7 @@ static void shrink_active_list(unsigned 
 		__mod_zone_page_state(zone, NR_ACTIVE_FILE, -nr_taken);
 	else
 		__mod_zone_page_state(zone, NR_ACTIVE_ANON, -nr_taken);
+	__mod_zone_page_state(zone, NR_ISOLATED_ANON + file, nr_taken);
 	spin_unlock_irq(&zone->lru_lock);
 
 	while (!list_empty(&l_hold)) {
@@ -1329,7 +1339,7 @@ static void shrink_active_list(unsigned 
 						LRU_ACTIVE + file * LRU_FILE);
 	move_active_pages_to_lru(zone, &l_inactive,
 						LRU_BASE   + file * LRU_FILE);
-
+	__mod_zone_page_state(zone, NR_ISOLATED_ANON + file, -nr_taken);
 	spin_unlock_irq(&zone->lru_lock);
 }
 
diff -puN mm/vmstat.c~mm-vmstat-add-isolate-pages mm/vmstat.c
--- a/mm/vmstat.c~mm-vmstat-add-isolate-pages
+++ a/mm/vmstat.c
@@ -644,6 +644,8 @@ static const char * const vmstat_text[] 
 	"nr_bounce",
 	"nr_vmscan_write",
 	"nr_writeback_temp",
+	"nr_isolated_anon",
+	"nr_isolated_file",
 	"nr_shmem",
 #ifdef CONFIG_NUMA
 	"numa_hit",
_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[KOSAKI Motohiro]

> On Wed, 16 Sep 2009 11:09:54 +0900 (JST) KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> 
> > Subject: [PATCH] Kill Isolated field in /proc/meminfo fix
> > 
> > Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> > It is only increased at heavy memory pressure case.
> 
> Have we made up our minds yet?
> 
> Below is what remains.  Please check that the changelog is still
> accurate and complete.  If not, please send along a new one?

Oh, great. I think that's correct.
Thanks!




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Rik van Riel]

Andrew Morton wrote:
> On Wed, 16 Sep 2009 11:09:54 +0900 (JST) KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> 
>> Subject: [PATCH] Kill Isolated field in /proc/meminfo fix
>>
>> Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
>> It is only increased at heavy memory pressure case.
> 
> Have we made up our minds yet?
> 
> Below is what remains.  Please check that the changelog is still
> accurate and complete.  If not, please send along a new one?

Looks good to me.  The isolated stats are printed at OOM
time and in /proc/vmstat - just where they belong, IMHO.

-- 
All rights reversed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Isolated(anon) and Isolated(file)

[Wu Fengguang]

On Wed, Sep 16, 2009 at 10:19:57AM +0800, Andrew Morton wrote:
> On Wed, 16 Sep 2009 11:09:54 +0900 (JST) KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> 
> > Subject: [PATCH] Kill Isolated field in /proc/meminfo fix
> > 
> > Hugh Dickins pointed out Isolated field dislpay 0kB at almost time.
> > It is only increased at heavy memory pressure case.
> 
> Have we made up our minds yet?
> 
> Below is what remains.  Please check that the changelog is still
> accurate and complete.  If not, please send along a new one?
> 
> 
> 
> From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> 
> If the system is running a heavy load of processes then concurrent reclaim
> can isolate a large number of pages from the LRU. /proc/meminfo and the

Better to change /proc/meminfo to /proc/vmstat.

Otherwise looks good to me.

Thanks,
Fengguang

> output generated for an OOM do not show how many pages were isolated.
> 
> This has been observed during process fork bomb testing (mstctl11 in LTP).
> 
> This patch shows the information about isolated pages.
> 
> Reproduced via:
> 
> -----------------------
> % ./hackbench 140 process 1000
>    => OOM occur
> 
> active_anon:146 inactive_anon:0 isolated_anon:49245
>  active_file:79 inactive_file:18 isolated_file:113
>  unevictable:0 dirty:0 writeback:0 unstable:0 buffer:39
>  free:370 slab_reclaimable:309 slab_unreclaimable:5492
>  mapped:53 shmem:15 pagetables:28140 bounce:0
> 
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Acked-by: Rik van Riel <riel@redhat.com>
> Acked-by: Wu Fengguang <fengguang.wu@intel.com>
> Reviewed-by: Minchan Kim <minchan.kim@gmail.com>
> Cc: Hugh Dickins <hugh.dickins@tiscali.co.uk>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  include/linux/mmzone.h |    2 ++
>  mm/migrate.c           |   11 +++++++++++
>  mm/page_alloc.c        |   12 +++++++++---
>  mm/vmscan.c            |   12 +++++++++++-
>  mm/vmstat.c            |    2 ++
>  5 files changed, 35 insertions(+), 4 deletions(-)
> 
> diff -puN drivers/base/node.c~mm-vmstat-add-isolate-pages drivers/base/node.c
> diff -puN fs/proc/meminfo.c~mm-vmstat-add-isolate-pages fs/proc/meminfo.c
> diff -puN include/linux/mmzone.h~mm-vmstat-add-isolate-pages include/linux/mmzone.h
> --- a/include/linux/mmzone.h~mm-vmstat-add-isolate-pages
> +++ a/include/linux/mmzone.h
> @@ -100,6 +100,8 @@ enum zone_stat_item {
>  	NR_BOUNCE,
>  	NR_VMSCAN_WRITE,
>  	NR_WRITEBACK_TEMP,	/* Writeback using temporary buffers */
> +	NR_ISOLATED_ANON,	/* Temporary isolated pages from anon lru */
> +	NR_ISOLATED_FILE,	/* Temporary isolated pages from file lru */
>  	NR_SHMEM,		/* shmem pages (included tmpfs/GEM pages) */
>  #ifdef CONFIG_NUMA
>  	NUMA_HIT,		/* allocated in intended node */
> diff -puN mm/migrate.c~mm-vmstat-add-isolate-pages mm/migrate.c
> --- a/mm/migrate.c~mm-vmstat-add-isolate-pages
> +++ a/mm/migrate.c
> @@ -67,6 +67,8 @@ int putback_lru_pages(struct list_head *
>  
>  	list_for_each_entry_safe(page, page2, l, lru) {
>  		list_del(&page->lru);
> +		dec_zone_page_state(page, NR_ISOLATED_ANON +
> +				    !!page_is_file_cache(page));
>  		putback_lru_page(page);
>  		count++;
>  	}
> @@ -698,6 +700,8 @@ unlock:
>   		 * restored.
>   		 */
>   		list_del(&page->lru);
> +		dec_zone_page_state(page, NR_ISOLATED_ANON +
> +				    !!page_is_file_cache(page));
>  		putback_lru_page(page);
>  	}
>  
> @@ -742,6 +746,13 @@ int migrate_pages(struct list_head *from
>  	struct page *page2;
>  	int swapwrite = current->flags & PF_SWAPWRITE;
>  	int rc;
> +	unsigned long flags;
> +
> +	local_irq_save(flags);
> +	list_for_each_entry(page, from, lru)
> +		__inc_zone_page_state(page, NR_ISOLATED_ANON +
> +				      !!page_is_file_cache(page));
> +	local_irq_restore(flags);
>  
>  	if (!swapwrite)
>  		current->flags |= PF_SWAPWRITE;
> diff -puN mm/page_alloc.c~mm-vmstat-add-isolate-pages mm/page_alloc.c
> --- a/mm/page_alloc.c~mm-vmstat-add-isolate-pages
> +++ a/mm/page_alloc.c
> @@ -2152,16 +2152,18 @@ void show_free_areas(void)
>  		}
>  	}
>  
> -	printk("Active_anon:%lu active_file:%lu inactive_anon:%lu\n"
> -		" inactive_file:%lu"
> +	printk("active_anon:%lu inactive_anon:%lu isolated_anon:%lu\n"
> +		" active_file:%lu inactive_file:%lu isolated_file:%lu\n"
>  		" unevictable:%lu"
>  		" dirty:%lu writeback:%lu unstable:%lu buffer:%lu\n"
>  		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
>  		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n",
>  		global_page_state(NR_ACTIVE_ANON),
> -		global_page_state(NR_ACTIVE_FILE),
>  		global_page_state(NR_INACTIVE_ANON),
> +		global_page_state(NR_ISOLATED_ANON),
> +		global_page_state(NR_ACTIVE_FILE),
>  		global_page_state(NR_INACTIVE_FILE),
> +		global_page_state(NR_ISOLATED_FILE),
>  		global_page_state(NR_UNEVICTABLE),
>  		global_page_state(NR_FILE_DIRTY),
>  		global_page_state(NR_WRITEBACK),
> @@ -2189,6 +2191,8 @@ void show_free_areas(void)
>  			" active_file:%lukB"
>  			" inactive_file:%lukB"
>  			" unevictable:%lukB"
> +			" isolated(anon):%lukB"
> +			" isolated(file):%lukB"
>  			" present:%lukB"
>  			" mlocked:%lukB"
>  			" dirty:%lukB"
> @@ -2215,6 +2219,8 @@ void show_free_areas(void)
>  			K(zone_page_state(zone, NR_ACTIVE_FILE)),
>  			K(zone_page_state(zone, NR_INACTIVE_FILE)),
>  			K(zone_page_state(zone, NR_UNEVICTABLE)),
> +			K(zone_page_state(zone, NR_ISOLATED_ANON)),
> +			K(zone_page_state(zone, NR_ISOLATED_FILE)),
>  			K(zone->present_pages),
>  			K(zone_page_state(zone, NR_MLOCK)),
>  			K(zone_page_state(zone, NR_FILE_DIRTY)),
> diff -puN mm/vmscan.c~mm-vmstat-add-isolate-pages mm/vmscan.c
> --- a/mm/vmscan.c~mm-vmstat-add-isolate-pages
> +++ a/mm/vmscan.c
> @@ -1072,6 +1072,8 @@ static unsigned long shrink_inactive_lis
>  		unsigned long nr_active;
>  		unsigned int count[NR_LRU_LISTS] = { 0, };
>  		int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
> +		unsigned long nr_anon;
> +		unsigned long nr_file;
>  
>  		nr_taken = sc->isolate_pages(sc->swap_cluster_max,
>  			     &page_list, &nr_scan, sc->order, mode,
> @@ -1102,6 +1104,10 @@ static unsigned long shrink_inactive_lis
>  		__mod_zone_page_state(zone, NR_INACTIVE_ANON,
>  						-count[LRU_INACTIVE_ANON]);
>  
> +		nr_anon = count[LRU_ACTIVE_ANON] + count[LRU_INACTIVE_ANON];
> +		nr_file = count[LRU_ACTIVE_FILE] + count[LRU_INACTIVE_FILE];
> +		__mod_zone_page_state(zone, NR_ISOLATED_ANON, nr_anon);
> +		__mod_zone_page_state(zone, NR_ISOLATED_FILE, nr_file);
>  
>  		reclaim_stat->recent_scanned[0] += count[LRU_INACTIVE_ANON];
>  		reclaim_stat->recent_scanned[0] += count[LRU_ACTIVE_ANON];
> @@ -1169,6 +1175,9 @@ static unsigned long shrink_inactive_lis
>  				spin_lock_irq(&zone->lru_lock);
>  			}
>  		}
> +		__mod_zone_page_state(zone, NR_ISOLATED_ANON, -nr_anon);
> +		__mod_zone_page_state(zone, NR_ISOLATED_FILE, -nr_file);
> +
>    	} while (nr_scanned < max_scan);
>  
>  done:
> @@ -1279,6 +1288,7 @@ static void shrink_active_list(unsigned 
>  		__mod_zone_page_state(zone, NR_ACTIVE_FILE, -nr_taken);
>  	else
>  		__mod_zone_page_state(zone, NR_ACTIVE_ANON, -nr_taken);
> +	__mod_zone_page_state(zone, NR_ISOLATED_ANON + file, nr_taken);
>  	spin_unlock_irq(&zone->lru_lock);
>  
>  	while (!list_empty(&l_hold)) {
> @@ -1329,7 +1339,7 @@ static void shrink_active_list(unsigned 
>  						LRU_ACTIVE + file * LRU_FILE);
>  	move_active_pages_to_lru(zone, &l_inactive,
>  						LRU_BASE   + file * LRU_FILE);
> -
> +	__mod_zone_page_state(zone, NR_ISOLATED_ANON + file, -nr_taken);
>  	spin_unlock_irq(&zone->lru_lock);
>  }
>  
> diff -puN mm/vmstat.c~mm-vmstat-add-isolate-pages mm/vmstat.c
> --- a/mm/vmstat.c~mm-vmstat-add-isolate-pages
> +++ a/mm/vmstat.c
> @@ -644,6 +644,8 @@ static const char * const vmstat_text[] 
>  	"nr_bounce",
>  	"nr_vmscan_write",
>  	"nr_writeback_temp",
> +	"nr_isolated_anon",
> +	"nr_isolated_file",
>  	"nr_shmem",
>  #ifdef CONFIG_NUMA
>  	"numa_hit",
> _

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>