From: Dave Chinner <david@fromorbit.com>
To: Zhi Yong Wu <zwu.kernel@gmail.com>
Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org,
linuxram@linux.vnet.ibm.com, viro@zeniv.linux.org.uk,
cmm@us.ibm.com, tytso@mit.edu, marco.stornelli@gmail.com,
stroetmann@ontolinux.com, diegocg@gmail.com, chris@csamuel.org,
Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Subject: Re: [RFC v2 05/10] vfs: introduce one hash table
Date: Thu, 27 Sep 2012 16:57:26 +1000 [thread overview]
Message-ID: <20120927065726.GP15236@dastard> (raw)
In-Reply-To: <CAEH94Liftb9zhTt=5TFUOay=aAQ+MA6YXsQWWt0Kn+F8U2MJNg@mail.gmail.com>
On Thu, Sep 27, 2012 at 02:23:16PM +0800, Zhi Yong Wu wrote:
> On Thu, Sep 27, 2012 at 11:43 AM, Dave Chinner <david@fromorbit.com> wrote:
> > On Sun, Sep 23, 2012 at 08:56:30PM +0800, zwu.kernel@gmail.com wrote:
> >> From: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> >>
> >> Adds a hash table structure which contains
> >> a lot of hash list and is used to efficiently
> >> look up the data temperature of a file or its
> >> ranges.
> >> In each hash list of hash table, the hash node
> >> will keep track of temperature info.
> >
> > So, let me see if I've got the relationship straight:
> >
> > - sb->s_hot_info.hot_inode_tree indexes hot_inode_items, one per inode
> >
> > - hot_inode_item contains access frequency data for that inode
> >
> > - hot_inode_item holds a heat hash node to index the access
> > frequency data for that inode
> >
> > - hot_inode_item.hot_range_tree indexes hot_range_items for that inode
> >
> > - hot_range_item contains access frequency data for that range
> >
> > - hot_range_item holds a heat hash node to index the access
> > frequency data for that range
> >
> > - sb->s_hot_info.heat_inode_hl indexes per-inode heat hash nodes
> >
> > - sb->s_hot_info.heat_range_hl indexes per-range heat hash nodes
> Correct.
> >
> > How about some ascii art? :) Just looking at the hot inode item case
> > (the range item case is the same pattern, though), we have:
> >
> >
> > heat_inode_hl hot_inode_tree
> > | |
> > | V
> > | +-------hot_inode_item-------+
> > +---+ | frequency data |
> > | V ^ V
> > | ...<--hot_inode_item-->... | ...<--hot_inode_item-->....
> > | frequency data | frequency data
> > | ^ | ^
> > | | | |
> > | | | |
> > +------>hot_hash_node-->hot_hash_node-->hot_hash_node-->....
> Great, can we put them in hot_tracking.txt in Documentation?
> >
> >
> > There's no actual data stored in the hot_hash_node, just pointer
> > back to the frequency data, a hlist_node and a pointer to the
> > hashlist head. IOWs, I agree with Ram that this does not need to
> > exist and just embedding a hlist_node inside the hot_inode_item is
> > all that is needed. i.e:
> >
> > heat_inode_hl hot_inode_tree
> > | |
> > | V
> > | +-------hot_inode_item-------+
> > | | frequency data |
> > +---+ | hlist_node |
> > | V ^ | V
> > | ...<--hot_inode_item-->... | | ...<--hot_inode_item-->....
> > | frequency data | | frequency data
> > +------>hlist_node-----------+ +------->hlist_node--->.....
> >
> > There's no need for separate allocations, initialisations, locks and
> > reference counting - all that is already in the hot_inode_item. The
> > items have the same lifecycle limitations - a hot_hash_node must be
> > torn down before the frequency data it points to is freed. Finally,
> > there's no difference in how you move it between lists.
> How will you know if one hot_inode_item should be moved between lists
> when its freq data is changed?
Record the current temperature in the frequency data, and if it
changes, change the list it is on.
> > Indeed, calling it a hash is wrong - there's not hashing at all
> > - it keeping an array of list where each entry corresponds to a
> > specific temperature. It is a *heat map*, not a hash list. i.e.
> > inode_heat_map, not heat_inode_hl. HEAT_MAP_SIZE, not HASH_SIZE.
> OK.
> >
> > As it is, there aren't any users of the heat maps that are generated
> > in this patch set - it's not even exported to userspace or to
> > debugfs, so I'm not sure how it will be used yet. How are these heat
> > maps going to be used by filesystems, Zhi?
> In hot_hash_calc_temperature(), you can see that one hot_inode or
> hot_range's freq data will be distilled into one temperature value,
> then it will be inserted to the heat map based on its temperature.
> When the file corresponding to the inode or range got hotter or cold,
> its location will be changed in the heat map based on its new
> temperature in hot_hash_update_hash_table().
Yes, but a hot_inode_item or hot_range_item can only have one
location in the heat map, right? So it doesn't need external
structure to point to the frequency data to track this....
> And the user will retrieve those freq data and temperature info via
> debugfs or ioctl interfaces.
Right - but that data is only extracted after an initial
hot_inode_tree lookup - The heat map itself is never directly used
for lookups. If it's not used for lookups based on temperature, why
is it needed?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2012-09-27 6:57 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-23 12:56 [RFC v2 00/10] vfs: hot data tracking zwu.kernel
2012-09-23 12:56 ` [RFC v2 01/10] vfs: introduce private rb structures zwu.kernel
2012-09-25 7:37 ` Dave Chinner
2012-09-25 7:57 ` Zhi Yong Wu
2012-09-25 8:00 ` Zhi Yong Wu
2012-09-25 10:20 ` Ram Pai
2012-09-26 3:20 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 02/10] vfs: add support for updating access frequency zwu.kernel
2012-09-25 9:17 ` Dave Chinner
2012-09-26 2:53 ` Zhi Yong Wu
2012-09-27 2:19 ` Dave Chinner
2012-09-27 2:30 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 03/10] vfs: add one new mount option '-o hottrack' zwu.kernel
2012-09-25 9:28 ` Dave Chinner
2012-09-26 2:56 ` Zhi Yong Wu
2012-09-27 2:20 ` Dave Chinner
2012-09-27 2:30 ` Zhi Yong Wu
2012-09-27 5:25 ` Zhi Yong Wu
2012-09-27 7:05 ` Dave Chinner
2012-09-27 7:21 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 04/10] vfs: add init and exit support zwu.kernel
2012-09-27 2:27 ` Dave Chinner
2012-09-23 12:56 ` [RFC v2 05/10] vfs: introduce one hash table zwu.kernel
2012-09-25 9:54 ` Ram Pai
2012-09-26 4:08 ` Zhi Yong Wu
2012-09-27 3:43 ` Dave Chinner
2012-09-27 6:23 ` Zhi Yong Wu
2012-09-27 6:57 ` Dave Chinner [this message]
2012-09-27 7:10 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 06/10] vfs: enable hot data tracking zwu.kernel
2012-09-27 3:54 ` Dave Chinner
2012-09-27 6:28 ` Zhi Yong Wu
2012-09-27 6:59 ` Dave Chinner
2012-09-27 7:12 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 07/10] vfs: fork one kthread to update data temperature zwu.kernel
2012-09-27 4:03 ` Dave Chinner
2012-09-27 6:54 ` Zhi Yong Wu
2012-09-27 7:01 ` Dave Chinner
2012-09-27 7:19 ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 08/10] vfs: add 3 new ioctl interfaces zwu.kernel
2012-09-23 12:56 ` [RFC v2 09/10] vfs: add debugfs support zwu.kernel
2012-09-23 12:56 ` [RFC v2 10/10] vfs: add documentation zwu.kernel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120927065726.GP15236@dastard \
--to=david@fromorbit.com \
--cc=chris@csamuel.org \
--cc=cmm@us.ibm.com \
--cc=diegocg@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxram@linux.vnet.ibm.com \
--cc=marco.stornelli@gmail.com \
--cc=stroetmann@ontolinux.com \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
--cc=wuzhy@linux.vnet.ibm.com \
--cc=zwu.kernel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).