All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: zwu.kernel@gmail.com
Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org,
	linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org,
	linuxram@linux.vnet.ibm.com, viro@zeniv.linux.org.uk,
	dave@jikos.cz, tytso@mit.edu, cmm@us.ibm.com,
	Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Subject: Re: [RFC v3 11/13] vfs: add 3 new ioctl interfaces
Date: Tue, 16 Oct 2012 14:17:10 +1100	[thread overview]
Message-ID: <20121016031710.GD2864@dastard> (raw)
In-Reply-To: <1349863655-29320-12-git-send-email-zwu.kernel@gmail.com>

On Wed, Oct 10, 2012 at 06:07:33PM +0800, zwu.kernel@gmail.com wrote:
> From: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> 
>   FS_IOC_GET_HEAT_INFO: return a struct containing the various
> metrics collected in btrfs_freq_data structs, and also return a

I think you mean hot_freq_data :P

> calculated data temperature based on those metrics. Optionally, retrieve
> the temperature from the hot data hash list instead of recalculating it.

To get the heat info for a specific file you have to know what file
you want to get that info for, right?  I can see the usefulness of
asking for the heat data on a specific file, but how do you find the
hot files in the first place? i.e. the big question the user
interface needs to answer is "what files are hot?".

Once userspace knows what the hottest files are, it can open them
and query the data via the above ioctl, but expecting userspace to
iterate millions of inodes in a filesystem to find hot files is very
inefficient.

FWIW, if you were to return file handles to the hottest files, then
the application could open and query them without even needing to
know the path name to them. This woul dbe exceedingly useful for
defragmentation programs, especially as that is the way xfs_fsr
already operates on candidate files.(*)

IOWs, sometimes the pathname is irrelevant to the operations that
applications want to perform - all they care about having an
efficient method of finding the inode they want and getting a file
descriptor that points to the file. Given the heat map info fits
right in to the sort of operations defrag and data mover tools
already do, it kind of makes sense to optimise the interface towards
those uses....

(*) i.e. finds them via bulkstat which returns handle information
along with all the other inode data, then opens the file by handle
to do the defrag work....

>   FS_IOC_GET_HEAT_OPTS: return an integer representing the current
> state of hot data tracking and migration:
> 
> 0 = do nothing
> 1 = track frequency of access
> 
>   FS_IOC_SET_HEAT_OPTS: change the state of hot data tracking and
> migration, as described above.

I can't see how this is a manageable interface. It is not
persistent, so after every filesystem mount you'd have to set the
flag on all your inodes again. Hence, for the moment, I'd suggest
that dropping per-inode tracking control until all the core issues
are sorted out....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

  parent reply	other threads:[~2012-10-16  3:17 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-10-10 10:07 [RFC v3 00/13] vfs: hot data tracking zwu.kernel
2012-10-10 10:07 ` [RFC v3 01/13] btrfs: add one new mount option '-o hot_track' zwu.kernel
     [not found]   ` <5075632c.03cc440a.1b33.7805SMTPIN_ADDED@mx.google.com>
2012-10-10 12:21     ` Zhi Yong Wu
2012-10-10 12:21       ` Zhi Yong Wu
2012-10-10 13:11       ` Lukáš Czerner
2012-10-10 13:16         ` Zhi Yong Wu
2012-10-10 16:28   ` David Sterba
2012-10-11 13:41     ` Zhi Yong Wu
2012-10-11 14:35     ` Zhi Yong Wu
2012-10-11 14:41       ` David Sterba
2012-10-11 14:46         ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 02/13] vfs: introduce private radix tree structures zwu.kernel
2012-10-10 15:34   ` David Sterba
2012-10-11 13:35     ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 03/13] vfs: Initialize and free main data structures zwu.kernel
2012-10-10 10:07 ` [RFC v3 04/13] vfs: add function for collecting raw access info zwu.kernel
2012-10-10 10:07 ` [RFC v3 05/13] vfs: add two map arrays zwu.kernel
2012-10-10 10:07 ` [RFC v3 06/13] vfs: add hooks to enable hot data tracking zwu.kernel
2012-10-10 10:07 ` [RFC v3 07/13] vfs: add function for updating map arrays zwu.kernel
2012-10-10 10:07 ` [RFC v3 08/13] vfs: add aging function for old map info zwu.kernel
2012-10-10 10:07 ` [RFC v3 09/13] vfs: add one wq to update map info periodically zwu.kernel
2012-10-16  0:27   ` Dave Chinner
2012-10-17  6:34     ` Zhi Yong Wu
2012-10-18  2:25       ` Zheng Liu
2012-10-18  2:26         ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 10/13] vfs: register one memory shrinker zwu.kernel
2012-10-10 10:07 ` [RFC v3 11/13] vfs: add 3 new ioctl interfaces zwu.kernel
2012-10-15  7:48   ` Dave Chinner
2012-10-15  7:57     ` Zhi Yong Wu
2012-10-16  3:17   ` Dave Chinner [this message]
2012-10-16  4:18     ` Zhi Yong Wu
2012-10-19  8:21     ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 12/13] vfs: add debugfs support zwu.kernel
2012-10-10 16:53   ` David Sterba
2012-10-10 21:05   ` David Sterba
2012-10-15  7:55   ` Dave Chinner
2012-10-15  8:15     ` Zhi Yong Wu
2012-10-15  8:04   ` Dave Chinner
2012-10-15  8:47     ` Zhi Yong Wu
2012-10-10 10:07 ` [RFC v3 13/13] vfs: add documentation zwu.kernel
2012-10-15  0:35   ` Zheng Liu
2012-10-15  7:04     ` Zhi Yong Wu
2012-10-15  0:39 ` [RFC v3 00/13] vfs: hot data tracking Zheng Liu
2012-10-15  7:05   ` Zhi Yong Wu
2012-10-15 20:42 ` Dave Chinner
2012-10-17  8:57   ` Zhi Yong Wu
2012-10-18  4:29     ` Dave Chinner
2012-10-18  4:44       ` Zhi Yong Wu
2012-10-18  5:17         ` Dave Chinner
2012-10-18  5:24           ` Zhi Yong Wu
2012-10-19  8:29   ` Zhi Yong Wu
2012-10-16  0:04 ` [PATCH] xfs: add hot tracking support Dave Chinner
2012-11-07  8:38   ` Zhi Yong Wu
2012-11-08  5:13     ` Dave Chinner
2012-10-16  0:11 ` [RFC v3 00/13] vfs: hot data tracking Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20121016031710.GD2864@dastard \
    --to=david@fromorbit.com \
    --cc=cmm@us.ibm.com \
    --cc=dave@jikos.cz \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxram@linux.vnet.ibm.com \
    --cc=tytso@mit.edu \
    --cc=viro@zeniv.linux.org.uk \
    --cc=wuzhy@linux.vnet.ibm.com \
    --cc=zwu.kernel@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.