From: Zhi Yong Wu <zwu.kernel@gmail.com>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org,
linuxram@linux.vnet.ibm.com, viro@zeniv.linux.org.uk,
dave@jikos.cz, tytso@mit.edu, cmm@us.ibm.com,
Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Subject: Re: [RFC v3 00/13] vfs: hot data tracking
Date: Fri, 19 Oct 2012 16:29:50 +0800 [thread overview]
Message-ID: <CAEH94LiWaaivc0-Xiq+WpS1cuHYV5jqnKMY756ZL8BW_LEWirw@mail.gmail.com> (raw)
In-Reply-To: <20121015204217.GB2739@dastard>
On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner <david@fromorbit.com> wrote:
> On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.kernel@gmail.com wrote:
>> From: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>>
>> NOTE:
>>
>> The patchset is currently post out mainly to make sure
>> it is going in the correct direction and hope to get some
>> helpful comments from other guys.
>> For more infomation, please check hot_tracking.txt in Documentation
>>
>> TODO List:
>
> 1) Fix OOM issues - the hot inode tracking caches grow very large
> and don't get trimmed under memory pressure. From slabtop, after
> creating roughly 24 million single byte files(*) on a machine with
> 8GB RAM:
>
> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
> 23859510 23859476 99% 0.12K 795317 30 3181268K hot_range_item
> 23859441 23859439 99% 0.16K 1037367 23 4149468K hot_inode_item
> 572530 572530 100% 0.55K 81790 7 327160K radix_tree_node
> 241706 241406 99% 0.22K 14218 17 56872K xfs_ili
> 241206 241204 99% 1.06K 80402 3 321608K xfs_inode
>
> The inode tracking is trying to track all 24 million inodes even
> though they have been written only once, and there are only 240,000
> inodes in the cache at this point in time. That was the last update
> that slabtop got, so it is indicative of the impending OOM situation
> that occurred.
>
>> Changelog from v2:
>> 1.) Converted to Radix trees, not RB-tree [Zhiyong, Dave Chinner]
>> 2.) Added memory shrinker [Dave Chinner]
>
> I haven't looked at the shrinker, but clearly it is not working,
HI, Dave,
Some guys suggest that when inode slab cache is shrinked, the
hot_inode[range]_item slab is accordingly also shrinked, this will
make hot tracking don't need to register its own shrinker. Do you
think of it?
If you don't like above idea. Do you have any good suggestion on how
to remove hot_inode_item and hot_range_item?
> otherwise the above OOM situation would not be occurring.
>
> Cheers,
>
> Dave.
>
> (*) Tested on an empty 17TB XFS filesystem with:
>
> $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc
> meta-data=/dev/vdc isize=256 agcount=17, agsize=268435455 blks
> = sectsz=512 attr=2, projid32bit=0
> data = bsize=4096 blocks=4563402735, imaxpct=5
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal log bsize=4096 blocks=131072, version=2
> = sectsz=512 sunit=1 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
> $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch
> $ sudo chmod 777 /mnt/scratch
> $ fs_mark -D 10000 -S0 -n 100000 -s 1 -L 63 -d \
> /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \
> /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \
> /mnt/scratch/6 -d /mnt/scratch/7
> .....
> 0 21600000 1 16679.3 12552262
> 0 22400000 1 15412.4 12588587
> 0 23200000 1 16367.6 14199322
> 0 24000000 1 15680.4 15741205
> <hangs here w/ OOM>
>
> --
> Dave Chinner
> david@fromorbit.com
--
Regards,
Zhi Yong Wu
next prev parent reply other threads:[~2012-10-19 8:29 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-10 10:07 [RFC v3 00/13] vfs: hot data tracking zwu.kernel
2012-10-10 10:07 ` [RFC v3 01/13] btrfs: add one new mount option '-o hot_track' zwu.kernel
[not found] ` <5075632c.03cc440a.1b33.7805SMTPIN_ADDED@mx.google.com>
2012-10-10 12:21 ` Zhi Yong Wu
2012-10-10 12:21 ` Zhi Yong Wu
2012-10-10 13:11 ` Lukáš Czerner
2012-10-10 13:16 ` Zhi Yong Wu
2012-10-10 16:28 ` David Sterba
2012-10-11 13:41 ` Zhi Yong Wu
2012-10-11 14:35 ` Zhi Yong Wu
2012-10-11 14:41 ` David Sterba
2012-10-11 14:46 ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 02/13] vfs: introduce private radix tree structures zwu.kernel
2012-10-10 15:34 ` David Sterba
2012-10-11 13:35 ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 03/13] vfs: Initialize and free main data structures zwu.kernel
2012-10-10 10:07 ` [RFC v3 04/13] vfs: add function for collecting raw access info zwu.kernel
2012-10-10 10:07 ` [RFC v3 05/13] vfs: add two map arrays zwu.kernel
2012-10-10 10:07 ` [RFC v3 06/13] vfs: add hooks to enable hot data tracking zwu.kernel
2012-10-10 10:07 ` [RFC v3 07/13] vfs: add function for updating map arrays zwu.kernel
2012-10-10 10:07 ` [RFC v3 08/13] vfs: add aging function for old map info zwu.kernel
2012-10-10 10:07 ` [RFC v3 09/13] vfs: add one wq to update map info periodically zwu.kernel
2012-10-16 0:27 ` Dave Chinner
2012-10-17 6:34 ` Zhi Yong Wu
2012-10-18 2:25 ` Zheng Liu
2012-10-18 2:26 ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 10/13] vfs: register one memory shrinker zwu.kernel
2012-10-10 10:07 ` [RFC v3 11/13] vfs: add 3 new ioctl interfaces zwu.kernel
2012-10-15 7:48 ` Dave Chinner
2012-10-15 7:57 ` Zhi Yong Wu
2012-10-16 3:17 ` Dave Chinner
2012-10-16 4:18 ` Zhi Yong Wu
2012-10-19 8:21 ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 12/13] vfs: add debugfs support zwu.kernel
2012-10-10 16:53 ` David Sterba
2012-10-10 21:05 ` David Sterba
2012-10-15 7:55 ` Dave Chinner
2012-10-15 8:15 ` Zhi Yong Wu
2012-10-15 8:04 ` Dave Chinner
2012-10-15 8:47 ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 13/13] vfs: add documentation zwu.kernel
2012-10-15 0:35 ` Zheng Liu
2012-10-15 7:04 ` Zhi Yong Wu
2012-10-15 0:39 ` [RFC v3 00/13] vfs: hot data tracking Zheng Liu
2012-10-15 7:05 ` Zhi Yong Wu
2012-10-15 20:42 ` Dave Chinner
2012-10-17 8:57 ` Zhi Yong Wu
2012-10-18 4:29 ` Dave Chinner
2012-10-18 4:44 ` Zhi Yong Wu
2012-10-18 5:17 ` Dave Chinner
2012-10-18 5:24 ` Zhi Yong Wu
2012-10-19 8:29 ` Zhi Yong Wu [this message]
2012-10-16 0:04 ` [PATCH] xfs: add hot tracking support Dave Chinner
2012-11-07 8:38 ` Zhi Yong Wu
2012-11-08 5:13 ` Dave Chinner
2012-10-16 0:11 ` [RFC v3 00/13] vfs: hot data tracking Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAEH94LiWaaivc0-Xiq+WpS1cuHYV5jqnKMY756ZL8BW_LEWirw@mail.gmail.com \
--to=zwu.kernel@gmail.com \
--cc=cmm@us.ibm.com \
--cc=dave@jikos.cz \
--cc=david@fromorbit.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxram@linux.vnet.ibm.com \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
--cc=wuzhy@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.