how to turn off, or to clear read cache?

how to turn off, or to clear read cache?

[Sergey Spiridonov]

Hi,

I need to make some performance tests. I need to switch off or to clear 
read cache, so that consequent reading of the same file will take the 
same amount of time.

Is there an easy way to do it, without rebuilding the kernel?

Please, put me on CC since I'm not subscriber of linux-kernel
-- 
Best regards, Sergey Spiridonov

Re: how to turn off, or to clear read cache?

[Måns Rullgård]

Sergey Spiridonov <spiridonov@gamic.com> writes:

> I need to make some performance tests. I need to switch off or to
> clear read cache, so that consequent reading of the same file will
> take the same amount of time.
>
> Is there an easy way to do it, without rebuilding the kernel?

I never found one, so I made this little patch.  It adds a mount
option to the ext3 filesystem that makes it drop the cache when a file
is closed.  Diff is against 2.4.21.  It's just a quick hack that
accomplishes what I needed.  I'm sure there's a better, more generic,
way.

Index: include/linux/ext3_fs.h
===================================================================
RCS file: /home/cvs/linux-2.4/include/linux/ext3_fs.h,v
retrieving revision 1.8
diff -u -r1.8 ext3_fs.h
--- include/linux/ext3_fs.h	1 Apr 2003 21:09:23 -0000	1.8
+++ include/linux/ext3_fs.h	17 Jun 2003 09:10:56 -0000
@@ -339,6 +339,7 @@
   #define EXT3_MOUNT_WRITEBACK_DATA	0x0C00	/* No data ordering */
 #define EXT3_MOUNT_UPDATE_JOURNAL	0x1000	/* Update the journal format */
 #define EXT3_MOUNT_NO_UID32		0x2000  /* Disable 32-bit UIDs */
+#define EXT3_MOUNT_NOCACHE		0x4000  /* Free cached blocks on close */
 
 /* Compatibility, for having both ext2_fs.h and ext3_fs.h included at once */
 #ifndef _LINUX_EXT2_FS_H
Index: fs/ext3/file.c
===================================================================
RCS file: /home/cvs/linux-2.4/fs/ext3/file.c,v
retrieving revision 1.4
diff -u -r1.4 file.c
--- fs/ext3/file.c	28 Aug 2002 21:11:16 -0000	1.4
+++ fs/ext3/file.c	17 Jun 2003 09:10:57 -0000
@@ -35,6 +35,10 @@
 {
 	if (filp->f_mode & FMODE_WRITE)
 		ext3_discard_prealloc (inode);
+	if (inode->i_sb->u.ext3_sb.s_mount_opt & EXT3_MOUNT_NOCACHE){
+		write_inode_now(inode, 1);
+		invalidate_inode_pages(inode);
+	}
 	return 0;
 }
 
Index: fs/ext3/super.c
===================================================================
RCS file: /home/cvs/linux-2.4/fs/ext3/super.c,v
retrieving revision 1.10
diff -u -r1.10 super.c
--- fs/ext3/super.c	28 Apr 2003 21:14:51 -0000	1.10
+++ fs/ext3/super.c	17 Jun 2003 09:10:59 -0000
@@ -654,6 +654,8 @@
 			if (want_numeric(value, "commit", &v))
 				return 0;
 			sbi->s_commit_interval = (HZ * v);
+		} else if (!strcmp (this_char, "nocache")){
+			set_opt (*mount_options, NOCACHE);
 		} else {
 			printk (KERN_ERR 
 				"EXT3-fs: Unrecognized mount option %s\n",


-- 
Måns Rullgård
mru@users.sf.net

Re: how to turn off, or to clear read cache?

[Denis Vlasenko]

On 20 August 2003 14:52, Sergey Spiridonov wrote:
> Hi,
> 
> I need to make some performance tests. I need to switch off or to clear 
> read cache, so that consequent reading of the same file will take the 
> same amount of time.

umount/mount cycle will do it, as well as intentional OOMing the box
(from non-root account please;)
--
vda

Re: how to turn off, or to clear read cache?

[Sergey Spiridonov]

Denis Vlasenko wrote:
> umount/mount cycle will do it, as well as intentional OOMing the box
> (from non-root account please;)

OOMing doesn't help also, since kernel starts to swap and I have 
performance degradation after. Swithching off the swap is dangerous in 
conjunction with OOMing.

-- 
Best regards, Sergey Spiridonov

Re: how to turn off, or to clear read cache?

[Jamie Lokier]

Denis Vlasenko wrote:
> On 20 August 2003 14:52, Sergey Spiridonov wrote:
> > I need to make some performance tests. I need to switch off or to clear 
> > read cache, so that consequent reading of the same file will take the 
> > same amount of time.
> 
> umount/mount cycle will do it, as well as intentional OOMing the box
> (from non-root account please;)

umount/mount is impractical when you're testing performance on a big
filesystem, which is always being used (e.g. /home).

I'm fairly sure that -o remount used to have this side effect, in fact
I used it quite a lot for that purpose, so it should be possible to
make it work again, or add a -o flushcache option.

-- Jamie

Re: how to turn off, or to clear read cache?

[Mike Fedyk]

On Wed, Aug 20, 2003 at 11:46:28PM +0100, Jamie Lokier wrote:
> Denis Vlasenko wrote:
> > On 20 August 2003 14:52, Sergey Spiridonov wrote:
> > > I need to make some performance tests. I need to switch off or to clear 
> > > read cache, so that consequent reading of the same file will take the 
> > > same amount of time.
> > 
> > umount/mount cycle will do it, as well as intentional OOMing the box
> > (from non-root account please;)
> 
> umount/mount is impractical when you're testing performance on a big
> filesystem, which is always being used (e.g. /home).
> 
> I'm fairly sure that -o remount used to have this side effect, in fact
> I used it quite a lot for that purpose, so it should be possible to
> make it work again, or add a -o flushcache option.

Won't that hold a lot of locks, and possibly pause the system until the
flushing is complete?  Hopefully, in 2.6 this doesn't call lock_kernel...

Re: how to turn off, or to clear read cache?

[Ihar 'Philips' Filipau]

Sergey Spiridonov wrote:
> Denis Vlasenko wrote:
> 
>> umount/mount cycle will do it, as well as intentional OOMing the box
>> (from non-root account please;)
> 
> 
> OOMing doesn't help also, since kernel starts to swap and I have 
> performance degradation after. Swithching off the swap is dangerous in 
> conjunction with OOMing.
> 

   indirectly, patch mentioned in this mail can help:

http://marc.theaimsgroup.com/?l=linux-kernel&m=106142721228497&w=2

   it provides a way to limit read cache - so it is easier to flush it.

P.S. will love to see something like this in 2.6. And for 2.4 too.

Re: how to turn off, or to clear read cache?

[Måns Rullgård]

Jeff Garzik <jgarzik@pobox.com> writes:

>> > Will it clear the cache?
>> 
>> It will probably clear some cache to make room for cache from hda.
>> 
>> perl -e '@f[0..100000000]=0'
>> 
>> will do it faster.
>
> Using fillmem will do it better :)

[39 lines of C code]

If you already have that piece of code, of course.

-- 
Måns Rullgård            You can write faster programs in C, but
mru@users.sf.net         you can write programs faster in Perl. 

Re: how to turn off, or to clear read cache?

[Jeff Garzik]

On Wed, Aug 20, 2003 at 06:57:15PM +0200, M?ns Rullg?rd wrote:
> Luciano Miguel Ferreira Rocha <luciano@lsd.di.uminho.pt> writes:
> > Will it clear the cache?
> 
> It will probably clear some cache to make room for cache from hda.
> 
> perl -e '@f[0..100000000]=0'
> 
> will do it faster.

Using fillmem will do it better :)

	Jeff




/* fillmem.c usage: "fillmem <number-of-megabytes>" */

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <string.h>

#define MEGS 140	/* default; override on command line */
#define MEG (1024 * 1024)

int main (int argc, char *argv[])
{
	void **data;
	int i, r;
	size_t megs = MEGS;

	if ((argc >= 2) && (atoi(argv[1]) > 0))
		megs = atoi(argv[1]);

	data = malloc (megs * sizeof (void*));
	if (!data) abort();

	memset (data, 0, megs * sizeof (void*));

	srand(time(NULL));

	for (i = 0; i < megs; i++) {
		data[i] = malloc(MEG);
		memset (data[i], i, MEG);
		printf("malloc/memset %03d/%03lu\n", i+1, megs);
	}
	for (i = megs - 1; i >= 0; i--) {
		r = rand() % 200;
		memset (data[i], r, MEG);
		printf("memset #2 %03d/%03lu = %d\n", i+1, megs, r);
	}
	printf("done\n");
	return 0;
}

Re: how to turn off, or to clear read cache?

[Måns Rullgård]

Luciano Miguel Ferreira Rocha <luciano@lsd.di.uminho.pt> writes:

>> >>> I need to make some performance tests. I need to switch off or to
>> >>> clear read cache, so that consequent reading of the same file will
>> >>> take the same amount of time.
>> >>>
>> >>>Is there an easy way to do it, without rebuilding the kernel?
>> >> Unmount and remount the filesystem.
>> >
>> >
>> > Would
>> >
>> > # mount -o remount
>> >
>> > do the job?
>> 
>> no
>
> What about dd if=/dev/hda bs=8M count=$(awk '/MemTotal/ { printf "%d", $2/4096 }' /proc/meminfo) ?
>
> Will it clear the cache?

It will probably clear some cache to make room for cache from hda.

perl -e '@f[0..100000000]=0'

will do it faster.

-- 
Måns Rullgård
mru@users.sf.net

Re: how to turn off, or to clear read cache?

[Luciano Miguel Ferreira Rocha]

On Wed, Aug 20, 2003 at 03:31:32PM +0200, Måns Rullgård wrote:
> Sergey Spiridonov <spiridonov@gamic.com> writes:
> 
> >>> I need to make some performance tests. I need to switch off or to
> >>> clear read cache, so that consequent reading of the same file will
> >>> take the same amount of time.
> >>>
> >>>Is there an easy way to do it, without rebuilding the kernel?
> >> Unmount and remount the filesystem.
> >
> >
> > Would
> >
> > # mount -o remount
> >
> > do the job?
> 
> no

What about dd if=/dev/hda bs=8M count=$(awk '/MemTotal/ { printf "%d", $2/4096 }' /proc/meminfo) ?

Will it clear the cache?

Regards,
Luciano Rocha

Re: how to turn off, or to clear read cache?

[Måns Rullgård]

Sergey Spiridonov <spiridonov@gamic.com> writes:

>>> I need to make some performance tests. I need to switch off or to
>>> clear read cache, so that consequent reading of the same file will
>>> take the same amount of time.
>>>
>>>Is there an easy way to do it, without rebuilding the kernel?
>> Unmount and remount the filesystem.
>
>
> Would
>
> # mount -o remount
>
> do the job?

no

-- 
Måns Rullgård
mru@users.sf.net

Re: how to turn off, or to clear read cache?

[Sergey Spiridonov]

John Bradford wrote:
>>I need to make some performance tests. I need to switch off or to clear 
>>read cache, so that consequent reading of the same file will take the 
>>same amount of time.
>>
>>Is there an easy way to do it, without rebuilding the kernel?
> 
> 
> Unmount and remount the filesystem.


Would

# mount -o remount

do the job?

I'm not able to unmount and remount.

-- 
Best regards, Sergey Spiridonov

Re: how to turn off, or to clear read cache?

[John Bradford]

> I need to make some performance tests. I need to switch off or to clear 
> read cache, so that consequent reading of the same file will take the 
> same amount of time.
>
> Is there an easy way to do it, without rebuilding the kernel?

Unmount and remount the filesystem.

John.