From: "Metin Döşlü" <metindoslu@gmail.com>
To: Jaegeuk Hanse <jaegeuk.hanse@gmail.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>, Jan Kara <jack@suse.cz>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: Problem in Page Cache Replacement
Date: Wed, 21 Nov 2012 12:07:22 +0200 [thread overview]
Message-ID: <CAJOrxZBpefqtkXr+XTxEZ6qy-6SCwQJ11makD=Lg_M4itY5Ang@mail.gmail.com> (raw)
In-Reply-To: <50ACA634.5000007@gmail.com>
On Wed, Nov 21, 2012 at 12:00 PM, Jaegeuk Hanse <jaegeuk.hanse@gmail.com> wrote:
>
> On 11/21/2012 05:58 PM, metin d wrote:
>
> Hi Fengguang,
>
> I run tests and attached the results. The line below I guess shows the data-1 page caches.
>
> 0x000000080000006c 6584051 25718 __RU_lA___________________P________ referenced,uptodate,lru,active,private
>
>
> I thinks this is just one state of page cache pages.
But why these page caches are in this state as opposed to other page
caches. From the results I conclude that:
data-1 pages are in state : referenced,uptodate,lru,active,private
data-2 pages are in state : referenced,uptodate,lru,mappedtodisk
>
>
>
>
> Metin
>
>
> ----- Original Message -----
> From: Jaegeuk Hanse <jaegeuk.hanse@gmail.com>
> To: Fengguang Wu <fengguang.wu@intel.com>
> Cc: metin d <metdos@yahoo.com>; Jan Kara <jack@suse.cz>; "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>; "linux-mm@kvack.org" <linux-mm@kvack.org>
> Sent: Wednesday, November 21, 2012 11:42 AM
> Subject: Re: Problem in Page Cache Replacement
>
> On 11/21/2012 05:02 PM, Fengguang Wu wrote:
> > On Wed, Nov 21, 2012 at 04:34:40PM +0800, Jaegeuk Hanse wrote:
> >> Cc Fengguang Wu.
> >>
> >> On 11/21/2012 04:13 PM, metin d wrote:
> >>>> Curious. Added linux-mm list to CC to catch more attention. If you run
> >>>> echo 1 >/proc/sys/vm/drop_caches does it evict data-1 pages from memory?
> >>> I'm guessing it'd evict the entries, but am wondering if we could run any more diagnostics before trying this.
> >>>
> >>> We regularly use a setup where we have two databases; one gets used frequently and the other one about once a month. It seems like the memory manager keeps unused pages in memory at the expense of frequently used database's performance.
> >>> My understanding was that under memory pressure from heavily
> >>> accessed pages, unused pages would eventually get evicted. Is there
> >>> anything else we can try on this host to understand why this is
> >>> happening?
> > We may debug it this way.
> >
> > 1) run 'fadvise data-2 0 0 dontneed' to drop data-2 cached pages
> > (please double check via /proc/vmstat whether it does the expected work)
> >
> > 2) run 'page-types -r' with root, to view the page status for the
> > remaining pages of data-1
> >
> > The fadvise tool comes from Andrew Morton's ext3-tools. (source code attached)
> > Please compile them with options "-Dlinux -I. -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE"
> >
> > page-types can be found in the kernel source tree tools/vm/page-types.c
> >
> > Sorry that sounds a bit twisted.. I do have a patch to directly dump
> > page cache status of a user specified file, however it's not
> > upstreamed yet.
>
> Hi Fengguang,
>
> Thanks for you detail steps, I think metin can have a try.
>
> flags page-count MB symbolic-flags long-symbolic-flags
> 0x0000000000000000 607699 2373
> ___________________________________
> 0x0000000100000000 343227 1340
> _______________________r___________ reserved
>
> But I have some questions of the print of page-type:
>
> Is 2373MB here mean total memory in used include page cache? I don't
> think so.
> Which kind of pages will be marked reserved?
> Which line of long-symbolic-flags is for page cache?
>
> Regards,
> Jaegeuk
>
> >
> > Thanks,
> > Fengguang
> >
> >>> On Tue 20-11-12 09:42:42, metin d wrote:
> >>>> I have two PostgreSQL databases named data-1 and data-2 that sit on the
> >>>> same machine. Both databases keep 40 GB of data, and the total memory
> >>>> available on the machine is 68GB.
> >>>>
> >>>> I started data-1 and data-2, and ran several queries to go over all their
> >>>> data. Then, I shut down data-1 and kept issuing queries against data-2.
> >>>> For some reason, the OS still holds on to large parts of data-1's pages
> >>>> in its page cache, and reserves about 35 GB of RAM to data-2's files. As
> >>>> a result, my queries on data-2 keep hitting disk.
> >>>>
> >>>> I'm checking page cache usage with fincore. When I run a table scan query
> >>>> against data-2, I see that data-2's pages get evicted and put back into
> >>>> the cache in a round-robin manner. Nothing happens to data-1's pages,
> >>>> although they haven't been touched for days.
> >>>>
> >>>> Does anybody know why data-1's pages aren't evicted from the page cache?
> >>>> I'm open to all kind of suggestions you think it might relate to problem.
> >>> Curious. Added linux-mm list to CC to catch more attention. If you run
> >>> echo 1 >/proc/sys/vm/drop_caches
> >>> does it evict data-1 pages from memory?
> >>>
> >>>> This is an EC2 m2.4xlarge instance on Amazon with 68 GB of RAM and no
> >>>> swap space. The kernel version is:
> >>>>
> >>>> $ uname -r
> >>>> 3.2.28-45.62.amzn1.x86_64
> >>>> Edit:
> >>>>
> >>>> and it seems that I use one NUMA instance, if you think that it can a problem.
> >>>>
> >>>> $ numactl --hardware
> >>>> available: 1 nodes (0)
> >>>> node 0 cpus: 0 1 2 3 4 5 6 7
> >>>> node 0 size: 70007 MB
> >>>> node 0 free: 360 MB
> >>>> node distances:
> >>>> node 0
> >>>> 0: 10
>
>
--
Metin Döşlü
next prev parent reply other threads:[~2012-11-21 10:07 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-20 17:42 Problem in Page Cache Replacement metin d
2012-11-20 18:25 ` Jan Kara
2012-11-21 8:03 ` metin d
2012-11-21 8:13 ` metin d
2012-11-21 8:34 ` Jaegeuk Hanse
2012-11-21 9:02 ` Fengguang Wu
2012-11-21 9:10 ` Fengguang Wu
2012-11-21 9:42 ` Jaegeuk Hanse
2012-11-21 10:00 ` metin d
[not found] ` <1353491880.11679.YahooMailNeo@web141102.mail.bf1.yahoo.com>
[not found] ` <50ACA634.5000007@gmail.com>
2012-11-21 10:07 ` Metin Döşlü [this message]
2012-11-22 15:41 ` Fengguang Wu
2012-11-22 15:53 ` Fengguang Wu
2012-11-23 2:10 ` Jaegeuk Hanse
2012-11-25 20:08 ` Rik van Riel
2012-11-24 15:06 ` Metin Döşlü
2012-11-22 15:26 ` Fengguang Wu
2012-11-23 1:32 ` Jaegeuk Hanse
2012-11-23 2:25 ` Fengguang Wu
2012-11-21 21:34 ` Johannes Weiner
2012-11-21 22:01 ` metin d
2012-11-22 0:48 ` Jaegeuk Hanse
2012-11-22 1:09 ` Johannes Weiner
2012-11-22 9:37 ` metin d
2012-11-22 13:16 ` Jaegeuk Hanse
2012-11-22 16:17 ` Johannes Weiner
2012-11-23 2:14 ` Jaegeuk Hanse
2012-11-23 1:58 ` Jaegeuk Hanse
2012-11-23 8:08 ` metin d
2012-11-23 8:17 ` Jaegeuk Hanse
2012-11-23 8:25 ` metin d
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJOrxZBpefqtkXr+XTxEZ6qy-6SCwQJ11makD=Lg_M4itY5Ang@mail.gmail.com' \
--to=metindoslu@gmail.com \
--cc=fengguang.wu@intel.com \
--cc=jack@suse.cz \
--cc=jaegeuk.hanse@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).