All of lore.kernel.org
 help / color / mirror / Atom feed
* [patch] drop_caches: add some documentation and info message
@ 2014-02-07 17:40 ` Johannes Weiner
  0 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 17:40 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Dave Hansen, Michal Hocko, KOSAKI Motohiro, KAMEZAWA Hiroyuki,
	Rik van Riel, linux-mm, linux-kernel

From: Dave Hansen <dave@linux.vnet.ibm.com>

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, and add a little KERN_NOTICE.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c            |  4 ++++
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..36278c610a5f 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): dropped kernel caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..3579d391e950 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -5,6 +5,7 @@
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/fs.h>
+#include <linux/ratelimit.h>
 #include <linux/writeback.h>
 #include <linux/sysctl.h>
 #include <linux/gfp.h>
@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 	if (ret)
 		return ret;
 	if (write) {
+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
+				   current->comm, task_pid_nr(current),
+				   sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
-- 
1.8.5.3


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [patch] drop_caches: add some documentation and info message
@ 2014-02-07 17:40 ` Johannes Weiner
  0 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 17:40 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Dave Hansen, Michal Hocko, KOSAKI Motohiro, KAMEZAWA Hiroyuki,
	Rik van Riel, linux-mm, linux-kernel

From: Dave Hansen <dave@linux.vnet.ibm.com>

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, and add a little KERN_NOTICE.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c            |  4 ++++
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..36278c610a5f 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): dropped kernel caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..3579d391e950 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -5,6 +5,7 @@
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/fs.h>
+#include <linux/ratelimit.h>
 #include <linux/writeback.h>
 #include <linux/sysctl.h>
 #include <linux/gfp.h>
@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 	if (ret)
 		return ret;
 	if (write) {
+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
+				   current->comm, task_pid_nr(current),
+				   sysctl_drop_caches);
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
-- 
1.8.5.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 17:40 ` Johannes Weiner
@ 2014-02-07 17:55   ` Rik van Riel
  -1 siblings, 0 replies; 22+ messages in thread
From: Rik van Riel @ 2014-02-07 17:55 UTC (permalink / raw)
  To: Johannes Weiner, Andrew Morton
  Cc: Dave Hansen, Michal Hocko, KOSAKI Motohiro, KAMEZAWA Hiroyuki,
	linux-mm, linux-kernel

On 02/07/2014 12:40 PM, Johannes Weiner wrote:

> @@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>   	if (ret)
>   		return ret;
>   	if (write) {
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>   		if (sysctl_drop_caches & 1)
>   			iterate_supers(drop_pagecache_sb, NULL);
>   		if (sysctl_drop_caches & 2)
>

Would it be better to print this after the operation
has completed?

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-07 17:55   ` Rik van Riel
  0 siblings, 0 replies; 22+ messages in thread
From: Rik van Riel @ 2014-02-07 17:55 UTC (permalink / raw)
  To: Johannes Weiner, Andrew Morton
  Cc: Dave Hansen, Michal Hocko, KOSAKI Motohiro, KAMEZAWA Hiroyuki,
	linux-mm, linux-kernel

On 02/07/2014 12:40 PM, Johannes Weiner wrote:

> @@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>   	if (ret)
>   		return ret;
>   	if (write) {
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>   		if (sysctl_drop_caches & 1)
>   			iterate_supers(drop_pagecache_sb, NULL);
>   		if (sysctl_drop_caches & 2)
>

Would it be better to print this after the operation
has completed?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 17:55   ` Rik van Riel
@ 2014-02-07 18:13     ` Johannes Weiner
  -1 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 18:13 UTC (permalink / raw)
  To: Rik van Riel
  Cc: Andrew Morton, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 12:55:37PM -0500, Rik van Riel wrote:
> On 02/07/2014 12:40 PM, Johannes Weiner wrote:
> 
> >@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> >  	if (ret)
> >  		return ret;
> >  	if (write) {
> >+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> >+				   current->comm, task_pid_nr(current),
> >+				   sysctl_drop_caches);
> >  		if (sysctl_drop_caches & 1)
> >  			iterate_supers(drop_pagecache_sb, NULL);
> >  		if (sysctl_drop_caches & 2)
> >
> 
> Would it be better to print this after the operation
> has completed?

It would make more sense grammatically :-) Either way is fine with me,
updated below to inform after the fact.

---
From: Dave Hansen <dave@linux.vnet.ibm.com>
Date: Fri, 12 Oct 2012 14:30:54 +0200
Subject: [patch] drop_caches: add some documentation and info message

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, and add a little KERN_NOTICE.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c            |  4 ++++
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..36278c610a5f 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): dropped kernel caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..02ae3386e08f 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -5,6 +5,7 @@
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/fs.h>
+#include <linux/ratelimit.h>
 #include <linux/writeback.h>
 #include <linux/sysctl.h>
 #include <linux/gfp.h>
@@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
 			drop_slab();
+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
+				   current->comm, task_pid_nr(current),
+				   sysctl_drop_caches);
 	}
 	return 0;
 }
-- 
1.8.5.3


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-07 18:13     ` Johannes Weiner
  0 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 18:13 UTC (permalink / raw)
  To: Rik van Riel
  Cc: Andrew Morton, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 12:55:37PM -0500, Rik van Riel wrote:
> On 02/07/2014 12:40 PM, Johannes Weiner wrote:
> 
> >@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> >  	if (ret)
> >  		return ret;
> >  	if (write) {
> >+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> >+				   current->comm, task_pid_nr(current),
> >+				   sysctl_drop_caches);
> >  		if (sysctl_drop_caches & 1)
> >  			iterate_supers(drop_pagecache_sb, NULL);
> >  		if (sysctl_drop_caches & 2)
> >
> 
> Would it be better to print this after the operation
> has completed?

It would make more sense grammatically :-) Either way is fine with me,
updated below to inform after the fact.

---
From: Dave Hansen <dave@linux.vnet.ibm.com>
Date: Fri, 12 Oct 2012 14:30:54 +0200
Subject: [patch] drop_caches: add some documentation and info message

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, and add a little KERN_NOTICE.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c            |  4 ++++
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..36278c610a5f 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): dropped kernel caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..02ae3386e08f 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -5,6 +5,7 @@
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/fs.h>
+#include <linux/ratelimit.h>
 #include <linux/writeback.h>
 #include <linux/sysctl.h>
 #include <linux/gfp.h>
@@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
 			drop_slab();
+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
+				   current->comm, task_pid_nr(current),
+				   sysctl_drop_caches);
 	}
 	return 0;
 }
-- 
1.8.5.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 18:13     ` Johannes Weiner
@ 2014-02-07 19:29       ` Rik van Riel
  -1 siblings, 0 replies; 22+ messages in thread
From: Rik van Riel @ 2014-02-07 19:29 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Andrew Morton, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On 02/07/2014 01:13 PM, Johannes Weiner wrote:

>> Would it be better to print this after the operation
>> has completed?
>
> It would make more sense grammatically :-) Either way is fine with me,
> updated below to inform after the fact.

Well, in principle the operation could take an arbitrarily
long time, so there are some minor concerns besides
grammatical correctness, too :)

Thanks for updating the patch.

> ---
> From: Dave Hansen <dave@linux.vnet.ibm.com>
> Date: Fri, 12 Oct 2012 14:30:54 +0200
> Subject: [patch] drop_caches: add some documentation and info message

> Dropping caches is a very drastic and disruptive operation that is
> good for debugging and running tests, but if it creates bug reports
> from production use, kernel developers should be aware of its use.
>
> Add a bit more documentation about it, and add a little KERN_NOTICE.
>
> [akpm@linux-foundation.org: checkpatch fixes]
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
> Signed-off-by: Michal Hocko <mhocko@suse.cz>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>

Reviewed-by: Rik van Riel <riel@redhat.com>


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-07 19:29       ` Rik van Riel
  0 siblings, 0 replies; 22+ messages in thread
From: Rik van Riel @ 2014-02-07 19:29 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Andrew Morton, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On 02/07/2014 01:13 PM, Johannes Weiner wrote:

>> Would it be better to print this after the operation
>> has completed?
>
> It would make more sense grammatically :-) Either way is fine with me,
> updated below to inform after the fact.

Well, in principle the operation could take an arbitrarily
long time, so there are some minor concerns besides
grammatical correctness, too :)

Thanks for updating the patch.

> ---
> From: Dave Hansen <dave@linux.vnet.ibm.com>
> Date: Fri, 12 Oct 2012 14:30:54 +0200
> Subject: [patch] drop_caches: add some documentation and info message

> Dropping caches is a very drastic and disruptive operation that is
> good for debugging and running tests, but if it creates bug reports
> from production use, kernel developers should be aware of its use.
>
> Add a bit more documentation about it, and add a little KERN_NOTICE.
>
> [akpm@linux-foundation.org: checkpatch fixes]
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
> Signed-off-by: Michal Hocko <mhocko@suse.cz>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>

Reviewed-by: Rik van Riel <riel@redhat.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 18:13     ` Johannes Weiner
@ 2014-02-07 20:31       ` Andrew Morton
  -1 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-07 20:31 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>  	}
>  	return 0;
>  }

My concern with this is that there may be people whose
other-party-provided software uses drop_caches.  Their machines will
now sit there emitting log messages and there's nothing they can do
about it, apart from whining at their vendors.


We could do something like this?

--- a/fs/drop_caches.c~drop_caches-add-some-documentation-and-info-message-fix
+++ a/fs/drop_caches.c
@@ -60,13 +60,17 @@ int drop_caches_sysctl_handler(ctl_table
 	if (ret)
 		return ret;
 	if (write) {
+		static int stfu;
+
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
 			drop_slab();
-		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
-				   current->comm, task_pid_nr(current),
-				   sysctl_drop_caches);
+		stfu |= sysctl_drop_caches & 4;
+		if (!stfu)
+			pr_info_ratelimited("%s (%d): dropped kernel caches: %d\n",
+					   current->comm, task_pid_nr(current),
+					   sysctl_drop_caches);
 	}
 	return 0;
 }
_

(note switch to pr_info_ratelimited)

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-07 20:31       ` Andrew Morton
  0 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-07 20:31 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>  	}
>  	return 0;
>  }

My concern with this is that there may be people whose
other-party-provided software uses drop_caches.  Their machines will
now sit there emitting log messages and there's nothing they can do
about it, apart from whining at their vendors.


We could do something like this?

--- a/fs/drop_caches.c~drop_caches-add-some-documentation-and-info-message-fix
+++ a/fs/drop_caches.c
@@ -60,13 +60,17 @@ int drop_caches_sysctl_handler(ctl_table
 	if (ret)
 		return ret;
 	if (write) {
+		static int stfu;
+
 		if (sysctl_drop_caches & 1)
 			iterate_supers(drop_pagecache_sb, NULL);
 		if (sysctl_drop_caches & 2)
 			drop_slab();
-		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
-				   current->comm, task_pid_nr(current),
-				   sysctl_drop_caches);
+		stfu |= sysctl_drop_caches & 4;
+		if (!stfu)
+			pr_info_ratelimited("%s (%d): dropped kernel caches: %d\n",
+					   current->comm, task_pid_nr(current),
+					   sysctl_drop_caches);
 	}
 	return 0;
 }
_

(note switch to pr_info_ratelimited)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 20:31       ` Andrew Morton
@ 2014-02-07 21:26         ` Johannes Weiner
  -1 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 21:26 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> 
> > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> >  			iterate_supers(drop_pagecache_sb, NULL);
> >  		if (sysctl_drop_caches & 2)
> >  			drop_slab();
> > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > +				   current->comm, task_pid_nr(current),
> > +				   sysctl_drop_caches);
> >  	}
> >  	return 0;
> >  }
> 
> My concern with this is that there may be people whose
> other-party-provided software uses drop_caches.  Their machines will
> now sit there emitting log messages and there's nothing they can do
> about it, apart from whining at their vendors.

Ironically, we have a customer that is complaining that we currently
do not log these events, and they want to know who in their stack is
being idiotic.

> We could do something like this?

They can already change the log level.  The below will suppress
valuable debugging information in a way that still results in
inconspicuous looking syslog excerpts, which somewhat undermines the
original motivation for this change.

So I'm not fond of it, but I'd rather have this patch with it than no
patch at all.  As long as the message is printed per default.

> --- a/fs/drop_caches.c~drop_caches-add-some-documentation-and-info-message-fix
> +++ a/fs/drop_caches.c
> @@ -60,13 +60,17 @@ int drop_caches_sysctl_handler(ctl_table
>  	if (ret)
>  		return ret;
>  	if (write) {
> +		static int stfu;
> +
>  		if (sysctl_drop_caches & 1)
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> -		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> -				   current->comm, task_pid_nr(current),
> -				   sysctl_drop_caches);
> +		stfu |= sysctl_drop_caches & 4;
> +		if (!stfu)
> +			pr_info_ratelimited("%s (%d): dropped kernel caches: %d\n",
> +					   current->comm, task_pid_nr(current),
> +					   sysctl_drop_caches);
>  	}
>  	return 0;
>  }
> _
> 
> (note switch to pr_info_ratelimited)

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-07 21:26         ` Johannes Weiner
  0 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-07 21:26 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> 
> > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> >  			iterate_supers(drop_pagecache_sb, NULL);
> >  		if (sysctl_drop_caches & 2)
> >  			drop_slab();
> > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > +				   current->comm, task_pid_nr(current),
> > +				   sysctl_drop_caches);
> >  	}
> >  	return 0;
> >  }
> 
> My concern with this is that there may be people whose
> other-party-provided software uses drop_caches.  Their machines will
> now sit there emitting log messages and there's nothing they can do
> about it, apart from whining at their vendors.

Ironically, we have a customer that is complaining that we currently
do not log these events, and they want to know who in their stack is
being idiotic.

> We could do something like this?

They can already change the log level.  The below will suppress
valuable debugging information in a way that still results in
inconspicuous looking syslog excerpts, which somewhat undermines the
original motivation for this change.

So I'm not fond of it, but I'd rather have this patch with it than no
patch at all.  As long as the message is printed per default.

> --- a/fs/drop_caches.c~drop_caches-add-some-documentation-and-info-message-fix
> +++ a/fs/drop_caches.c
> @@ -60,13 +60,17 @@ int drop_caches_sysctl_handler(ctl_table
>  	if (ret)
>  		return ret;
>  	if (write) {
> +		static int stfu;
> +
>  		if (sysctl_drop_caches & 1)
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> -		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> -				   current->comm, task_pid_nr(current),
> -				   sysctl_drop_caches);
> +		stfu |= sysctl_drop_caches & 4;
> +		if (!stfu)
> +			pr_info_ratelimited("%s (%d): dropped kernel caches: %d\n",
> +					   current->comm, task_pid_nr(current),
> +					   sysctl_drop_caches);
>  	}
>  	return 0;
>  }
> _
> 
> (note switch to pr_info_ratelimited)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 18:13     ` Johannes Weiner
@ 2014-02-08 11:55       ` Rafael Aquini
  -1 siblings, 0 replies; 22+ messages in thread
From: Rafael Aquini @ 2014-02-08 11:55 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Andrew Morton, Dave Hansen, Michal Hocko,
	KOSAKI Motohiro, KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 01:13:32PM -0500, Johannes Weiner wrote:
> On Fri, Feb 07, 2014 at 12:55:37PM -0500, Rik van Riel wrote:
> > On 02/07/2014 12:40 PM, Johannes Weiner wrote:
> > 
> > >@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > >  	if (ret)
> > >  		return ret;
> > >  	if (write) {
> > >+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > >+				   current->comm, task_pid_nr(current),
> > >+				   sysctl_drop_caches);
> > >  		if (sysctl_drop_caches & 1)
> > >  			iterate_supers(drop_pagecache_sb, NULL);
> > >  		if (sysctl_drop_caches & 2)
> > >
> > 
> > Would it be better to print this after the operation
> > has completed?
> 
> It would make more sense grammatically :-) Either way is fine with me,
> updated below to inform after the fact.
> 
> ---
> From: Dave Hansen <dave@linux.vnet.ibm.com>
> Date: Fri, 12 Oct 2012 14:30:54 +0200
> Subject: [patch] drop_caches: add some documentation and info message
> 
> There is plenty of anecdotal evidence and a load of blog posts
> suggesting that using "drop_caches" periodically keeps your system
> running in "tip top shape".  Perhaps adding some kernel documentation
> will increase the amount of accurate data on its use.
> 
> If we are not shrinking caches effectively, then we have real bugs.
> Using drop_caches will simply mask the bugs and make them harder to
> find, but certainly does not fix them, nor is it an appropriate
> "workaround" to limit the size of the caches.  On the contrary, there
> have been bug reports on issues that turned out to be misguided use of
> cache dropping.
> 
> Dropping caches is a very drastic and disruptive operation that is
> good for debugging and running tests, but if it creates bug reports
> from production use, kernel developers should be aware of its use.
> 
> Add a bit more documentation about it, and add a little KERN_NOTICE.
> 
> [akpm@linux-foundation.org: checkpatch fixes]
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
> Signed-off-by: Michal Hocko <mhocko@suse.cz>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> ---
>  Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
>  fs/drop_caches.c            |  4 ++++
>  2 files changed, 31 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
> index d614a9b6a280..36278c610a5f 100644
> --- a/Documentation/sysctl/vm.txt
> +++ b/Documentation/sysctl/vm.txt
> @@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
>  
>  drop_caches
>  
> -Writing to this will cause the kernel to drop clean caches, dentries and
> -inodes from memory, causing that memory to become free.
> +Writing to this will cause the kernel to drop clean caches, as well as
> +reclaimable slab objects like dentries and inodes.  Once dropped, their
> +memory becomes free.
>  
>  To free pagecache:
>  	echo 1 > /proc/sys/vm/drop_caches
> -To free dentries and inodes:
> +To free reclaimable slab objects (includes dentries and inodes):
>  	echo 2 > /proc/sys/vm/drop_caches
> -To free pagecache, dentries and inodes:
> +To free slab objects and pagecache:
>  	echo 3 > /proc/sys/vm/drop_caches
>  
> -As this is a non-destructive operation and dirty objects are not freeable, the
> -user should run `sync' first.
> +This is a non-destructive operation and will not free any dirty objects.
> +To increase the number of objects freed by this operation, the user may run
> +`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
> +number of dirty objects on the system and create more candidates to be
> +dropped.
> +
> +This file is not a means to control the growth of the various kernel caches
> +(inodes, dentries, pagecache, etc...)  These objects are automatically
> +reclaimed by the kernel when memory is needed elsewhere on the system.
> +
> +Use of this file can cause performance problems.  Since it discards cached
> +objects, it may cost a significant amount of I/O and CPU to recreate the
> +dropped objects, especially if they were under heavy use.  Because of this,
> +use outside of a testing or debugging environment is not recommended.
> +
> +You may see informational messages in your kernel log when this file is
> +used:
> +
> +	cat (1234): dropped kernel caches: 3
> +
> +These are informational only.  They do not mean that anything is wrong
> +with your system.
>  
>  ==============================================================
>  
> diff --git a/fs/drop_caches.c b/fs/drop_caches.c
> index 9fd702f5bfb2..02ae3386e08f 100644
> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -5,6 +5,7 @@
>  #include <linux/kernel.h>
>  #include <linux/mm.h>
>  #include <linux/fs.h>
> +#include <linux/ratelimit.h>
>  #include <linux/writeback.h>
>  #include <linux/sysctl.h>
>  #include <linux/gfp.h>
> @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",

Just a nitpick here: 
 s/printk_ratelimited(KERN_INFO ...)/pr_info_ratelimited(...)


Acked-by: Rafael Aquini <aquini@redhat.com>


> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>  	}
>  	return 0;
>  }
> -- 
> 1.8.5.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-08 11:55       ` Rafael Aquini
  0 siblings, 0 replies; 22+ messages in thread
From: Rafael Aquini @ 2014-02-08 11:55 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Andrew Morton, Dave Hansen, Michal Hocko,
	KOSAKI Motohiro, KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, Feb 07, 2014 at 01:13:32PM -0500, Johannes Weiner wrote:
> On Fri, Feb 07, 2014 at 12:55:37PM -0500, Rik van Riel wrote:
> > On 02/07/2014 12:40 PM, Johannes Weiner wrote:
> > 
> > >@@ -59,6 +60,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > >  	if (ret)
> > >  		return ret;
> > >  	if (write) {
> > >+		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > >+				   current->comm, task_pid_nr(current),
> > >+				   sysctl_drop_caches);
> > >  		if (sysctl_drop_caches & 1)
> > >  			iterate_supers(drop_pagecache_sb, NULL);
> > >  		if (sysctl_drop_caches & 2)
> > >
> > 
> > Would it be better to print this after the operation
> > has completed?
> 
> It would make more sense grammatically :-) Either way is fine with me,
> updated below to inform after the fact.
> 
> ---
> From: Dave Hansen <dave@linux.vnet.ibm.com>
> Date: Fri, 12 Oct 2012 14:30:54 +0200
> Subject: [patch] drop_caches: add some documentation and info message
> 
> There is plenty of anecdotal evidence and a load of blog posts
> suggesting that using "drop_caches" periodically keeps your system
> running in "tip top shape".  Perhaps adding some kernel documentation
> will increase the amount of accurate data on its use.
> 
> If we are not shrinking caches effectively, then we have real bugs.
> Using drop_caches will simply mask the bugs and make them harder to
> find, but certainly does not fix them, nor is it an appropriate
> "workaround" to limit the size of the caches.  On the contrary, there
> have been bug reports on issues that turned out to be misguided use of
> cache dropping.
> 
> Dropping caches is a very drastic and disruptive operation that is
> good for debugging and running tests, but if it creates bug reports
> from production use, kernel developers should be aware of its use.
> 
> Add a bit more documentation about it, and add a little KERN_NOTICE.
> 
> [akpm@linux-foundation.org: checkpatch fixes]
> Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
> Signed-off-by: Michal Hocko <mhocko@suse.cz>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
> ---
>  Documentation/sysctl/vm.txt | 33 +++++++++++++++++++++++++++------
>  fs/drop_caches.c            |  4 ++++
>  2 files changed, 31 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
> index d614a9b6a280..36278c610a5f 100644
> --- a/Documentation/sysctl/vm.txt
> +++ b/Documentation/sysctl/vm.txt
> @@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
>  
>  drop_caches
>  
> -Writing to this will cause the kernel to drop clean caches, dentries and
> -inodes from memory, causing that memory to become free.
> +Writing to this will cause the kernel to drop clean caches, as well as
> +reclaimable slab objects like dentries and inodes.  Once dropped, their
> +memory becomes free.
>  
>  To free pagecache:
>  	echo 1 > /proc/sys/vm/drop_caches
> -To free dentries and inodes:
> +To free reclaimable slab objects (includes dentries and inodes):
>  	echo 2 > /proc/sys/vm/drop_caches
> -To free pagecache, dentries and inodes:
> +To free slab objects and pagecache:
>  	echo 3 > /proc/sys/vm/drop_caches
>  
> -As this is a non-destructive operation and dirty objects are not freeable, the
> -user should run `sync' first.
> +This is a non-destructive operation and will not free any dirty objects.
> +To increase the number of objects freed by this operation, the user may run
> +`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
> +number of dirty objects on the system and create more candidates to be
> +dropped.
> +
> +This file is not a means to control the growth of the various kernel caches
> +(inodes, dentries, pagecache, etc...)  These objects are automatically
> +reclaimed by the kernel when memory is needed elsewhere on the system.
> +
> +Use of this file can cause performance problems.  Since it discards cached
> +objects, it may cost a significant amount of I/O and CPU to recreate the
> +dropped objects, especially if they were under heavy use.  Because of this,
> +use outside of a testing or debugging environment is not recommended.
> +
> +You may see informational messages in your kernel log when this file is
> +used:
> +
> +	cat (1234): dropped kernel caches: 3
> +
> +These are informational only.  They do not mean that anything is wrong
> +with your system.
>  
>  ==============================================================
>  
> diff --git a/fs/drop_caches.c b/fs/drop_caches.c
> index 9fd702f5bfb2..02ae3386e08f 100644
> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -5,6 +5,7 @@
>  #include <linux/kernel.h>
>  #include <linux/mm.h>
>  #include <linux/fs.h>
> +#include <linux/ratelimit.h>
>  #include <linux/writeback.h>
>  #include <linux/sysctl.h>
>  #include <linux/gfp.h>
> @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  			iterate_supers(drop_pagecache_sb, NULL);
>  		if (sysctl_drop_caches & 2)
>  			drop_slab();
> +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",

Just a nitpick here: 
 s/printk_ratelimited(KERN_INFO ...)/pr_info_ratelimited(...)


Acked-by: Rafael Aquini <aquini@redhat.com>


> +				   current->comm, task_pid_nr(current),
> +				   sysctl_drop_caches);
>  	}
>  	return 0;
>  }
> -- 
> 1.8.5.3
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-07 21:26         ` Johannes Weiner
@ 2014-02-10 20:51           ` Andrew Morton
  -1 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-10 20:51 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> > 
> > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > >  			iterate_supers(drop_pagecache_sb, NULL);
> > >  		if (sysctl_drop_caches & 2)
> > >  			drop_slab();
> > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > > +				   current->comm, task_pid_nr(current),
> > > +				   sysctl_drop_caches);
> > >  	}
> > >  	return 0;
> > >  }
> > 
> > My concern with this is that there may be people whose
> > other-party-provided software uses drop_caches.  Their machines will
> > now sit there emitting log messages and there's nothing they can do
> > about it, apart from whining at their vendors.
> 
> Ironically, we have a customer that is complaining that we currently
> do not log these events, and they want to know who in their stack is
> being idiotic.

Right.  But if we release a kernel which goes blah on every write to
drop_caches, that customer has logs full of blahs which they are
now totally uninterested in.

> > We could do something like this?
> 
> They can already change the log level.

Suppressing unrelated things...

>  The below will suppress
> valuable debugging information in a way that still results in
> inconspicuous looking syslog excerpts, which somewhat undermines the
> original motivation for this change.

Yes, somewhat.  It is a compromise. You can see my concern here?



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-10 20:51           ` Andrew Morton
  0 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-10 20:51 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> > 
> > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > >  			iterate_supers(drop_pagecache_sb, NULL);
> > >  		if (sysctl_drop_caches & 2)
> > >  			drop_slab();
> > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > > +				   current->comm, task_pid_nr(current),
> > > +				   sysctl_drop_caches);
> > >  	}
> > >  	return 0;
> > >  }
> > 
> > My concern with this is that there may be people whose
> > other-party-provided software uses drop_caches.  Their machines will
> > now sit there emitting log messages and there's nothing they can do
> > about it, apart from whining at their vendors.
> 
> Ironically, we have a customer that is complaining that we currently
> do not log these events, and they want to know who in their stack is
> being idiotic.

Right.  But if we release a kernel which goes blah on every write to
drop_caches, that customer has logs full of blahs which they are
now totally uninterested in.

> > We could do something like this?
> 
> They can already change the log level.

Suppressing unrelated things...

>  The below will suppress
> valuable debugging information in a way that still results in
> inconspicuous looking syslog excerpts, which somewhat undermines the
> original motivation for this change.

Yes, somewhat.  It is a compromise. You can see my concern here?


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* RE: [patch] drop_caches: add some documentation and info message
  2014-02-10 20:51           ` Andrew Morton
@ 2014-02-10 21:11             ` Motohiro Kosaki
  -1 siblings, 0 replies; 22+ messages in thread
From: Motohiro Kosaki @ 2014-02-10 21:11 UTC (permalink / raw)
  To: Andrew Morton, Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, Motohiro Kosaki JP,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel



> -----Original Message-----
> From: Andrew Morton [mailto:akpm@linux-foundation.org]
> Sent: Tuesday, February 11, 2014 5:51 AM
> To: Johannes Weiner
> Cc: Rik van Riel; Dave Hansen; Michal Hocko; Motohiro Kosaki JP; KAMEZAWA
> Hiroyuki; linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [patch] drop_caches: add some documentation and info
> message
> 
> On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org>
> wrote:
> 
> > On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner
> <hannes@cmpxchg.org> wrote:
> > >
> > > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table,
> int write,
> > > >  			iterate_supers(drop_pagecache_sb, NULL);
> > > >  		if (sysctl_drop_caches & 2)
> > > >  			drop_slab();
> > > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel
> caches: %d\n",
> > > > +				   current->comm, task_pid_nr(current),
> > > > +				   sysctl_drop_caches);
> > > >  	}
> > > >  	return 0;
> > > >  }
> > >
> > > My concern with this is that there may be people whose
> > > other-party-provided software uses drop_caches.  Their machines will
> > > now sit there emitting log messages and there's nothing they can do
> > > about it, apart from whining at their vendors.
> >
> > Ironically, we have a customer that is complaining that we currently
> > do not log these events, and they want to know who in their stack is
> > being idiotic.
> 
> Right.  But if we release a kernel which goes blah on every write to
> drop_caches, that customer has logs full of blahs which they are now totally
> uninterested in.

Please let me know if I misunderstand something. This patch uses KERN_INFO.
Then, any log shouldn't be emitted by default.

Moreover, if someone change syslog log level to INFO, they are going to see
much prenty annoying and too much logs even if we reject this patch.





^ permalink raw reply	[flat|nested] 22+ messages in thread

* RE: [patch] drop_caches: add some documentation and info message
@ 2014-02-10 21:11             ` Motohiro Kosaki
  0 siblings, 0 replies; 22+ messages in thread
From: Motohiro Kosaki @ 2014-02-10 21:11 UTC (permalink / raw)
  To: Andrew Morton, Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, Motohiro Kosaki JP,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel



> -----Original Message-----
> From: Andrew Morton [mailto:akpm@linux-foundation.org]
> Sent: Tuesday, February 11, 2014 5:51 AM
> To: Johannes Weiner
> Cc: Rik van Riel; Dave Hansen; Michal Hocko; Motohiro Kosaki JP; KAMEZAWA
> Hiroyuki; linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [patch] drop_caches: add some documentation and info
> message
> 
> On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org>
> wrote:
> 
> > On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner
> <hannes@cmpxchg.org> wrote:
> > >
> > > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table,
> int write,
> > > >  			iterate_supers(drop_pagecache_sb, NULL);
> > > >  		if (sysctl_drop_caches & 2)
> > > >  			drop_slab();
> > > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel
> caches: %d\n",
> > > > +				   current->comm, task_pid_nr(current),
> > > > +				   sysctl_drop_caches);
> > > >  	}
> > > >  	return 0;
> > > >  }
> > >
> > > My concern with this is that there may be people whose
> > > other-party-provided software uses drop_caches.  Their machines will
> > > now sit there emitting log messages and there's nothing they can do
> > > about it, apart from whining at their vendors.
> >
> > Ironically, we have a customer that is complaining that we currently
> > do not log these events, and they want to know who in their stack is
> > being idiotic.
> 
> Right.  But if we release a kernel which goes blah on every write to
> drop_caches, that customer has logs full of blahs which they are now totally
> uninterested in.

Please let me know if I misunderstand something. This patch uses KERN_INFO.
Then, any log shouldn't be emitted by default.

Moreover, if someone change syslog log level to INFO, they are going to see
much prenty annoying and too much logs even if we reject this patch.




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-10 20:51           ` Andrew Morton
@ 2014-02-10 21:54             ` Johannes Weiner
  -1 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-10 21:54 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Mon, Feb 10, 2014 at 12:51:02PM -0800, Andrew Morton wrote:
> On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> 
> > On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> > > 
> > > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > > >  			iterate_supers(drop_pagecache_sb, NULL);
> > > >  		if (sysctl_drop_caches & 2)
> > > >  			drop_slab();
> > > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > > > +				   current->comm, task_pid_nr(current),
> > > > +				   sysctl_drop_caches);
> > > >  	}
> > > >  	return 0;
> > > >  }
> > > 
> > > My concern with this is that there may be people whose
> > > other-party-provided software uses drop_caches.  Their machines will
> > > now sit there emitting log messages and there's nothing they can do
> > > about it, apart from whining at their vendors.
> > 
> > Ironically, we have a customer that is complaining that we currently
> > do not log these events, and they want to know who in their stack is
> > being idiotic.
> 
> Right.  But if we release a kernel which goes blah on every write to
> drop_caches, that customer has logs full of blahs which they are
> now totally uninterested in.
> 
> > > We could do something like this?
> > 
> > They can already change the log level.
> 
> Suppressing unrelated things...
> 
> >  The below will suppress
> > valuable debugging information in a way that still results in
> > inconspicuous looking syslog excerpts, which somewhat undermines the
> > original motivation for this change.
> 
> Yes, somewhat.  It is a compromise. You can see my concern here?

How about this: we allow disabling the log message, but print the line
of the disabling call so it's clear who dunnit.  To make sure valuable
info is not missing in bug reports, add counters for the two events in
/proc/vmstat.

Does that sound acceptable?

---
From: Dave Hansen <dave@linux.vnet.ibm.com>
Date: Fri, 12 Oct 2012 14:30:54 +0200
Subject: [patch] drop_caches: add some documentation and info message

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, a syslog message to track down
abusers, and vmstat drop counters to help analyze problem reports.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt   | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c              | 16 ++++++++++++++--
 include/linux/vm_event_item.h |  1 +
 kernel/sysctl.c               |  4 ++--
 mm/vmstat.c                   |  3 +++
 5 files changed, 47 insertions(+), 10 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..e7e544bc4ead 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): drop_caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.  To disable them, echo 4 (bit 3) into drop_caches.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..9280202e488c 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -59,10 +59,22 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 	if (ret)
 		return ret;
 	if (write) {
-		if (sysctl_drop_caches & 1)
+		static int stfu;
+
+		if (sysctl_drop_caches & 1) {
 			iterate_supers(drop_pagecache_sb, NULL);
-		if (sysctl_drop_caches & 2)
+			count_vm_event(DROP_PAGECACHE);
+		}
+		if (sysctl_drop_caches & 2) {
 			drop_slab();
+			count_vm_event(DROP_SLAB);
+		}
+		if (!stfu) {
+			pr_info("%s (%d): drop_caches: %d\n",
+				current->comm, task_pid_nr(current),
+				sysctl_drop_caches);
+		}
+		stfu |= sysctl_drop_caches & 4;
 	}
 	return 0;
 }
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index c557c6d096de..99a7d0e71f8c 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -37,6 +37,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
 		PGINODESTEAL, SLABS_SCANNED, KSWAPD_INODESTEAL,
 		KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
 		PAGEOUTRUN, ALLOCSTALL, PGROTATED,
+		DROP_PAGECACHE, DROP_SLAB,
 #ifdef CONFIG_NUMA_BALANCING
 		NUMA_PTE_UPDATES,
 		NUMA_HUGE_PTE_UPDATES,
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 49e13e1f8fe6..4b0e0857b4b8 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -126,7 +126,7 @@ static int __maybe_unused neg_one = -1;
 static int zero;
 static int __maybe_unused one = 1;
 static int __maybe_unused two = 2;
-static int __maybe_unused three = 3;
+static int __maybe_unused four = 4;
 static unsigned long one_ul = 1;
 static int one_hundred = 100;
 #ifdef CONFIG_PRINTK
@@ -1283,7 +1283,7 @@ static struct ctl_table vm_table[] = {
 		.mode		= 0644,
 		.proc_handler	= drop_caches_sysctl_handler,
 		.extra1		= &one,
-		.extra2		= &three,
+		.extra2		= &four,
 	},
 #ifdef CONFIG_COMPACTION
 	{
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 72496140ac08..7887588ce8f8 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -810,6 +810,9 @@ const char * const vmstat_text[] = {
 
 	"pgrotated",
 
+	"drop_pagecache",
+	"drop_slab",
+
 #ifdef CONFIG_NUMA_BALANCING
 	"numa_pte_updates",
 	"numa_huge_pte_updates",
-- 
1.8.5.3



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-10 21:54             ` Johannes Weiner
  0 siblings, 0 replies; 22+ messages in thread
From: Johannes Weiner @ 2014-02-10 21:54 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Mon, Feb 10, 2014 at 12:51:02PM -0800, Andrew Morton wrote:
> On Fri, 7 Feb 2014 16:26:01 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> 
> > On Fri, Feb 07, 2014 at 12:31:29PM -0800, Andrew Morton wrote:
> > > On Fri, 7 Feb 2014 13:13:32 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:
> > > 
> > > > @@ -63,6 +64,9 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
> > > >  			iterate_supers(drop_pagecache_sb, NULL);
> > > >  		if (sysctl_drop_caches & 2)
> > > >  			drop_slab();
> > > > +		printk_ratelimited(KERN_INFO "%s (%d): dropped kernel caches: %d\n",
> > > > +				   current->comm, task_pid_nr(current),
> > > > +				   sysctl_drop_caches);
> > > >  	}
> > > >  	return 0;
> > > >  }
> > > 
> > > My concern with this is that there may be people whose
> > > other-party-provided software uses drop_caches.  Their machines will
> > > now sit there emitting log messages and there's nothing they can do
> > > about it, apart from whining at their vendors.
> > 
> > Ironically, we have a customer that is complaining that we currently
> > do not log these events, and they want to know who in their stack is
> > being idiotic.
> 
> Right.  But if we release a kernel which goes blah on every write to
> drop_caches, that customer has logs full of blahs which they are
> now totally uninterested in.
> 
> > > We could do something like this?
> > 
> > They can already change the log level.
> 
> Suppressing unrelated things...
> 
> >  The below will suppress
> > valuable debugging information in a way that still results in
> > inconspicuous looking syslog excerpts, which somewhat undermines the
> > original motivation for this change.
> 
> Yes, somewhat.  It is a compromise. You can see my concern here?

How about this: we allow disabling the log message, but print the line
of the disabling call so it's clear who dunnit.  To make sure valuable
info is not missing in bug reports, add counters for the two events in
/proc/vmstat.

Does that sound acceptable?

---
From: Dave Hansen <dave@linux.vnet.ibm.com>
Date: Fri, 12 Oct 2012 14:30:54 +0200
Subject: [patch] drop_caches: add some documentation and info message

There is plenty of anecdotal evidence and a load of blog posts
suggesting that using "drop_caches" periodically keeps your system
running in "tip top shape".  Perhaps adding some kernel documentation
will increase the amount of accurate data on its use.

If we are not shrinking caches effectively, then we have real bugs.
Using drop_caches will simply mask the bugs and make them harder to
find, but certainly does not fix them, nor is it an appropriate
"workaround" to limit the size of the caches.  On the contrary, there
have been bug reports on issues that turned out to be misguided use of
cache dropping.

Dropping caches is a very drastic and disruptive operation that is
good for debugging and running tests, but if it creates bug reports
from production use, kernel developers should be aware of its use.

Add a bit more documentation about it, a syslog message to track down
abusers, and vmstat drop counters to help analyze problem reports.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 Documentation/sysctl/vm.txt   | 33 +++++++++++++++++++++++++++------
 fs/drop_caches.c              | 16 ++++++++++++++--
 include/linux/vm_event_item.h |  1 +
 kernel/sysctl.c               |  4 ++--
 mm/vmstat.c                   |  3 +++
 5 files changed, 47 insertions(+), 10 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index d614a9b6a280..e7e544bc4ead 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -175,18 +175,39 @@ Setting this to zero disables periodic writeback altogether.
 
 drop_caches
 
-Writing to this will cause the kernel to drop clean caches, dentries and
-inodes from memory, causing that memory to become free.
+Writing to this will cause the kernel to drop clean caches, as well as
+reclaimable slab objects like dentries and inodes.  Once dropped, their
+memory becomes free.
 
 To free pagecache:
 	echo 1 > /proc/sys/vm/drop_caches
-To free dentries and inodes:
+To free reclaimable slab objects (includes dentries and inodes):
 	echo 2 > /proc/sys/vm/drop_caches
-To free pagecache, dentries and inodes:
+To free slab objects and pagecache:
 	echo 3 > /proc/sys/vm/drop_caches
 
-As this is a non-destructive operation and dirty objects are not freeable, the
-user should run `sync' first.
+This is a non-destructive operation and will not free any dirty objects.
+To increase the number of objects freed by this operation, the user may run
+`sync' prior to writing to /proc/sys/vm/drop_caches.  This will minimize the
+number of dirty objects on the system and create more candidates to be
+dropped.
+
+This file is not a means to control the growth of the various kernel caches
+(inodes, dentries, pagecache, etc...)  These objects are automatically
+reclaimed by the kernel when memory is needed elsewhere on the system.
+
+Use of this file can cause performance problems.  Since it discards cached
+objects, it may cost a significant amount of I/O and CPU to recreate the
+dropped objects, especially if they were under heavy use.  Because of this,
+use outside of a testing or debugging environment is not recommended.
+
+You may see informational messages in your kernel log when this file is
+used:
+
+	cat (1234): drop_caches: 3
+
+These are informational only.  They do not mean that anything is wrong
+with your system.  To disable them, echo 4 (bit 3) into drop_caches.
 
 ==============================================================
 
diff --git a/fs/drop_caches.c b/fs/drop_caches.c
index 9fd702f5bfb2..9280202e488c 100644
--- a/fs/drop_caches.c
+++ b/fs/drop_caches.c
@@ -59,10 +59,22 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
 	if (ret)
 		return ret;
 	if (write) {
-		if (sysctl_drop_caches & 1)
+		static int stfu;
+
+		if (sysctl_drop_caches & 1) {
 			iterate_supers(drop_pagecache_sb, NULL);
-		if (sysctl_drop_caches & 2)
+			count_vm_event(DROP_PAGECACHE);
+		}
+		if (sysctl_drop_caches & 2) {
 			drop_slab();
+			count_vm_event(DROP_SLAB);
+		}
+		if (!stfu) {
+			pr_info("%s (%d): drop_caches: %d\n",
+				current->comm, task_pid_nr(current),
+				sysctl_drop_caches);
+		}
+		stfu |= sysctl_drop_caches & 4;
 	}
 	return 0;
 }
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index c557c6d096de..99a7d0e71f8c 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -37,6 +37,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
 		PGINODESTEAL, SLABS_SCANNED, KSWAPD_INODESTEAL,
 		KSWAPD_LOW_WMARK_HIT_QUICKLY, KSWAPD_HIGH_WMARK_HIT_QUICKLY,
 		PAGEOUTRUN, ALLOCSTALL, PGROTATED,
+		DROP_PAGECACHE, DROP_SLAB,
 #ifdef CONFIG_NUMA_BALANCING
 		NUMA_PTE_UPDATES,
 		NUMA_HUGE_PTE_UPDATES,
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 49e13e1f8fe6..4b0e0857b4b8 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -126,7 +126,7 @@ static int __maybe_unused neg_one = -1;
 static int zero;
 static int __maybe_unused one = 1;
 static int __maybe_unused two = 2;
-static int __maybe_unused three = 3;
+static int __maybe_unused four = 4;
 static unsigned long one_ul = 1;
 static int one_hundred = 100;
 #ifdef CONFIG_PRINTK
@@ -1283,7 +1283,7 @@ static struct ctl_table vm_table[] = {
 		.mode		= 0644,
 		.proc_handler	= drop_caches_sysctl_handler,
 		.extra1		= &one,
-		.extra2		= &three,
+		.extra2		= &four,
 	},
 #ifdef CONFIG_COMPACTION
 	{
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 72496140ac08..7887588ce8f8 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -810,6 +810,9 @@ const char * const vmstat_text[] = {
 
 	"pgrotated",
 
+	"drop_pagecache",
+	"drop_slab",
+
 #ifdef CONFIG_NUMA_BALANCING
 	"numa_pte_updates",
 	"numa_huge_pte_updates",
-- 
1.8.5.3


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
  2014-02-10 21:54             ` Johannes Weiner
@ 2014-02-11 21:44               ` Andrew Morton
  -1 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-11 21:44 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Mon, 10 Feb 2014 16:54:16 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> How about this: we allow disabling the log message, but print the line
> of the disabling call so it's clear who dunnit.  To make sure valuable
> info is not missing in bug reports, add counters for the two events in
> /proc/vmstat.
> 
> Does that sound acceptable?

Yes, I really don't know what's the right thing to do here or where the
best tradeoff point is situated.  Let's start off this way and see what
happens I guess.

> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -59,10 +59,22 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  	if (ret)
>  		return ret;
>  	if (write) {
> -		if (sysctl_drop_caches & 1)
> +		static int stfu;

That identifier wasn't serious, but I kinda like it.

> +
> +		if (sysctl_drop_caches & 1) {
>  			iterate_supers(drop_pagecache_sb, NULL);
> -		if (sysctl_drop_caches & 2)
> +			count_vm_event(DROP_PAGECACHE);
> +		}
> +		if (sysctl_drop_caches & 2) {
>  			drop_slab();
> +			count_vm_event(DROP_SLAB);
> +		}
> +		if (!stfu) {
> +			pr_info("%s (%d): drop_caches: %d\n",
> +				current->comm, task_pid_nr(current),
> +				sysctl_drop_caches);
> +		}
> +		stfu |= sysctl_drop_caches & 4;
>  	}
>  	return 0;
>  }


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [patch] drop_caches: add some documentation and info message
@ 2014-02-11 21:44               ` Andrew Morton
  0 siblings, 0 replies; 22+ messages in thread
From: Andrew Morton @ 2014-02-11 21:44 UTC (permalink / raw)
  To: Johannes Weiner
  Cc: Rik van Riel, Dave Hansen, Michal Hocko, KOSAKI Motohiro,
	KAMEZAWA Hiroyuki, linux-mm, linux-kernel

On Mon, 10 Feb 2014 16:54:16 -0500 Johannes Weiner <hannes@cmpxchg.org> wrote:

> How about this: we allow disabling the log message, but print the line
> of the disabling call so it's clear who dunnit.  To make sure valuable
> info is not missing in bug reports, add counters for the two events in
> /proc/vmstat.
> 
> Does that sound acceptable?

Yes, I really don't know what's the right thing to do here or where the
best tradeoff point is situated.  Let's start off this way and see what
happens I guess.

> --- a/fs/drop_caches.c
> +++ b/fs/drop_caches.c
> @@ -59,10 +59,22 @@ int drop_caches_sysctl_handler(ctl_table *table, int write,
>  	if (ret)
>  		return ret;
>  	if (write) {
> -		if (sysctl_drop_caches & 1)
> +		static int stfu;

That identifier wasn't serious, but I kinda like it.

> +
> +		if (sysctl_drop_caches & 1) {
>  			iterate_supers(drop_pagecache_sb, NULL);
> -		if (sysctl_drop_caches & 2)
> +			count_vm_event(DROP_PAGECACHE);
> +		}
> +		if (sysctl_drop_caches & 2) {
>  			drop_slab();
> +			count_vm_event(DROP_SLAB);
> +		}
> +		if (!stfu) {
> +			pr_info("%s (%d): drop_caches: %d\n",
> +				current->comm, task_pid_nr(current),
> +				sysctl_drop_caches);
> +		}
> +		stfu |= sysctl_drop_caches & 4;
>  	}
>  	return 0;
>  }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2014-02-11 21:45 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-07 17:40 [patch] drop_caches: add some documentation and info message Johannes Weiner
2014-02-07 17:40 ` Johannes Weiner
2014-02-07 17:55 ` Rik van Riel
2014-02-07 17:55   ` Rik van Riel
2014-02-07 18:13   ` Johannes Weiner
2014-02-07 18:13     ` Johannes Weiner
2014-02-07 19:29     ` Rik van Riel
2014-02-07 19:29       ` Rik van Riel
2014-02-07 20:31     ` Andrew Morton
2014-02-07 20:31       ` Andrew Morton
2014-02-07 21:26       ` Johannes Weiner
2014-02-07 21:26         ` Johannes Weiner
2014-02-10 20:51         ` Andrew Morton
2014-02-10 20:51           ` Andrew Morton
2014-02-10 21:11           ` Motohiro Kosaki
2014-02-10 21:11             ` Motohiro Kosaki
2014-02-10 21:54           ` Johannes Weiner
2014-02-10 21:54             ` Johannes Weiner
2014-02-11 21:44             ` Andrew Morton
2014-02-11 21:44               ` Andrew Morton
2014-02-08 11:55     ` Rafael Aquini
2014-02-08 11:55       ` Rafael Aquini

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.