Re: Large buffer cache in EXT4

From: Martin Steigerwald <Martin@lichtvoll.de>
To: Subranshu Patel <spatel.ml@gmail.com>
Cc: linux-ext4@vger.kernel.org
Subject: Re: Large buffer cache in EXT4
Date: Sun, 17 Feb 2013 11:25:39 +0100	[thread overview]
Message-ID: <201302171125.40116.Martin@lichtvoll.de> (raw)
In-Reply-To: <CAEUQcegw=gq9ndo=Zd4xC2gJDZrRzNpS2hAZW9x5Lh=KL_YDHQ@mail.gmail.com>

Am Sonntag, 17. Februar 2013 schrieb Subranshu Patel:
> I created 2 filesystem on my system (RHEL 6.3, kernel version 2.6.32)
> - XFS and EXT4 and mounted them.
> 
> On both the filesystem I executed the mdtest tool(opensource tool) for
> 64 concurrent process. Each process performed the following:
> - Create large number of directories
> - Remove all the directories
> 
> During this time I monitored the memory usage of the system using sar
> command. I checked the 3 components - kbmemused, kbbuffers and
> kbcached
> 
> kbmemused - Amount of used memory in kilobytes. This does not take
> into account memory used by the kernel itself.
> kbbuffers - buffer cache
> kbcached - page cache
> 
> While the kbmemused and kbcached component was almost similar in EXT4
> and XFS (XFS being a little higher), the kbbuffer showed a totally
> different trend.
> 
> For EXT4, kbbuffers was:
> 390999KB for dir creation
> 364803KB for dir removal
> For XFS, kbbuffers was:
> 
> 1701KB for dir creation
> 2738KB for dir removal
> 
> In kernel 2.6, both buffer cache and page cache are merged. The page
> cache caches pages of files. The buffer cache caches disk blocks which
> consists of mainly metadata (not file data).
> 
> Why is the buffer cache large in case of EXT4 and what is stored in
> the buffer cache?

What is stored in the buffer cache? An interesting question. I also wondered 
about it.

I always thought filesystem metadata that is to be written to the disk. As 
opposed to dirty pages which are counted in Dirty: in /proc/meminfo.

Then on being asked in a Linux Performance Analyse and Tuning training I 
held where I had some little Linux kernel hackers in there, it seemed to me, 
they found out, that it is a disk block buffer by looking at the source. And 
indeed on doing dd if=/dev/zero of=/dev/somedevice bs=1M or so the buffer 
count raises considerably.

What I never really understand was what is the clear distinction between 
dirty pages and disk block buffers. Why isn´t anything that is about to be 
written to disk in one cache?

Can anybody enlighten me?

PS: buffers=0 with BTRFS also.

Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html