From: Dave Chinner <david@fromorbit.com>
To: Josef Bacik <jbacik@fb.com>
Cc: Jan Kara <jack@suse.cz>,
linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
kernel-team@fb.com, jack@suse.com, viro@zeniv.linux.org.uk,
dchinner@redhat.com, hch@lst.de, linux-mm@kvack.org
Subject: Re: [PATCH 2/3] writeback: allow for dirty metadata accounting
Date: Tue, 13 Sep 2016 09:18:26 +1000 [thread overview]
Message-ID: <20160912231826.GK22388@dastard> (raw)
In-Reply-To: <bd00ed53-00c8-49d2-13b2-5f7dfa607185@fb.com>
On Mon, Sep 12, 2016 at 10:56:04AM -0400, Josef Bacik wrote:
> I think that looping through all the sb's in the system would be
> kinda shitty for this tho, we want the "get number of dirty pages"
> part to be relatively fast. What if I do something like the
> shrinker_control only for dirty objects. So the fs registers some
> dirty_objects_control, we call into each of those and get the counts
> from that. Does that sound less crappy? Thanks,
Hmmm - just an off-the-wall thought on this....
If you're going to do that, then why wouldn't you simply use a
"shrinker" to do the metadata writeback rather than having a hook to
count dirty objects to pass to some other writeback code that calls
a hook to write the metadata?
That way filesystems can also implement dirty accounting and
"writers" for each cache of objects they currently implement
shrinkers for. i.e. just expanding shrinkers to be able to "count
dirty objects" and "write dirty objects" so that we can tell
filesystems to write back all their different metadata caches
proportionally to the size of the page cache and it's dirty state.
The existing file data and inode writeback could then just be new
generic "superblock shrinker" operations, and the fs could have it's
own private metadata writeback similar to the private sb shrinker
callout we currently have...
And, in doing so, we might be able to completely hide memcg from the
writeback implementations similar to the way memcg is completely
hidden from the shrinker reclaim implementations...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com>
To: Josef Bacik <jbacik@fb.com>
Cc: Jan Kara <jack@suse.cz>,
linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
kernel-team@fb.com, jack@suse.com, viro@zeniv.linux.org.uk,
dchinner@redhat.com, hch@lst.de, linux-mm@kvack.org
Subject: Re: [PATCH 2/3] writeback: allow for dirty metadata accounting
Date: Tue, 13 Sep 2016 09:18:26 +1000 [thread overview]
Message-ID: <20160912231826.GK22388@dastard> (raw)
In-Reply-To: <bd00ed53-00c8-49d2-13b2-5f7dfa607185@fb.com>
On Mon, Sep 12, 2016 at 10:56:04AM -0400, Josef Bacik wrote:
> I think that looping through all the sb's in the system would be
> kinda shitty for this tho, we want the "get number of dirty pages"
> part to be relatively fast. What if I do something like the
> shrinker_control only for dirty objects. So the fs registers some
> dirty_objects_control, we call into each of those and get the counts
> from that. Does that sound less crappy? Thanks,
Hmmm - just an off-the-wall thought on this....
If you're going to do that, then why wouldn't you simply use a
"shrinker" to do the metadata writeback rather than having a hook to
count dirty objects to pass to some other writeback code that calls
a hook to write the metadata?
That way filesystems can also implement dirty accounting and
"writers" for each cache of objects they currently implement
shrinkers for. i.e. just expanding shrinkers to be able to "count
dirty objects" and "write dirty objects" so that we can tell
filesystems to write back all their different metadata caches
proportionally to the size of the page cache and it's dirty state.
The existing file data and inode writeback could then just be new
generic "superblock shrinker" operations, and the fs could have it's
own private metadata writeback similar to the private sb shrinker
callout we currently have...
And, in doing so, we might be able to completely hide memcg from the
writeback implementations similar to the way memcg is completely
hidden from the shrinker reclaim implementations...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-09-12 23:18 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-22 17:34 [PATCH 0/3][V2] Provide accounting for dirty metadata Josef Bacik
2016-08-22 17:34 ` Josef Bacik
2016-08-22 17:34 ` Josef Bacik
2016-08-22 17:35 ` [PATCH 1/3] remove mapping from balance_dirty_pages*() Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-08-22 18:29 ` kbuild test robot
2016-08-22 17:35 ` [PATCH 2/3] writeback: allow for dirty metadata accounting Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-09-09 8:17 ` Jan Kara
2016-09-09 8:17 ` Jan Kara
2016-09-12 0:46 ` Dave Chinner
2016-09-12 0:46 ` Dave Chinner
2016-09-12 7:34 ` Jan Kara
2016-09-12 7:34 ` Jan Kara
2016-09-12 14:24 ` Josef Bacik
2016-09-12 14:24 ` Josef Bacik
2016-09-12 14:24 ` Josef Bacik
2016-09-12 23:01 ` Dave Chinner
2016-09-12 23:01 ` Dave Chinner
2016-09-12 14:56 ` Josef Bacik
2016-09-12 14:56 ` Josef Bacik
2016-09-12 14:56 ` Josef Bacik
2016-09-12 23:18 ` Dave Chinner [this message]
2016-09-12 23:18 ` Dave Chinner
2016-08-22 17:35 ` [PATCH 3/3] writeback: introduce super_operations->write_metadata Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-08-22 17:35 ` Josef Bacik
2016-09-09 8:29 ` Jan Kara
2016-09-09 8:29 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160912231826.GK22388@dastard \
--to=david@fromorbit.com \
--cc=dchinner@redhat.com \
--cc=hch@lst.de \
--cc=jack@suse.com \
--cc=jack@suse.cz \
--cc=jbacik@fb.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.