From: Maxim Patlasov <mpatlasov@parallels.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: <miklos@szeredi.hu>, <riel@redhat.com>, <dev@parallels.com>,
<xemul@parallels.com>, <fuse-devel@lists.sourceforge.net>,
<bfoster@redhat.com>, <linux-kernel@vger.kernel.org>,
<jbottomley@parallels.com>, <linux-mm@kvack.org>,
<viro@zeniv.linux.org.uk>, <linux-fsdevel@vger.kernel.org>,
<fengguang.wu@intel.com>, <devel@openvz.org>, <mgorman@suse.de>
Subject: Re: [PATCH] mm: strictlimit feature -v2
Date: Wed, 3 Jul 2013 15:01:19 +0400 [thread overview]
Message-ID: <51D4047F.2010700@parallels.com> (raw)
In-Reply-To: <20130702123804.9f252487f86c12b0f4edee57@linux-foundation.org>
07/02/2013 11:38 PM, Andrew Morton пишет:
> On Tue, 02 Jul 2013 21:44:47 +0400 Maxim Patlasov <MPatlasov@parallels.com> wrote:
>
>> From: Miklos Szeredi <mszeredi@suse.cz>
>>
>> The feature prevents mistrusted filesystems to grow a large number of dirty
>> pages before throttling. For such filesystems balance_dirty_pages always
>> check bdi counters against bdi limits. I.e. even if global "nr_dirty" is under
>> "freerun", it's not allowed to skip bdi checks. The only use case for now is
>> fuse: it sets bdi max_ratio to 1% by default and system administrators are
>> supposed to expect that this limit won't be exceeded.
>>
>> The feature is on if address space is marked by AS_STRICTLIMIT flag.
>> A filesystem may set the flag when it initializes a new inode.
>>
>> Changed in v2 (thanks to Andrew Morton):
>> - added a few explanatory comments
>> - cleaned up the mess in backing_dev_info foo_stamp fields: now it's clearly
>> stated that bw_time_stamp is measured in jiffies; renamed other foo_stamp
>> fields to reflect that they are in units of number-of-pages.
>>
> Better, thanks.
>
> The writeback arithemtic makes my head spin - I'd really like Fengguang
> to go over this, please.
>
> A quick visit from the spelling police:
Great! Thank you, Andrew. I'll wait for Fengguang' feedback for a while
before respin.
>
>> ...
>>
>> @@ -41,8 +43,15 @@ typedef int (congested_fn)(void *, int);
>> enum bdi_stat_item {
>> BDI_RECLAIMABLE,
>> BDI_WRITEBACK,
>> - BDI_DIRTIED,
>> - BDI_WRITTEN,
>> +
>> + /*
>> + * The three counters below reflects number of events of specific type
>> + * happened since bdi_init(). The type is defined in comments below:
> "The three counters below reflect the number of events of specific
> types since bdi_init()"
>
>> + */
>> + BDI_DIRTIED, /* a page was dirtied */
>> + BDI_WRITTEN, /* writeout completed for a page */
>> + BDI_WRITTEN_BACK, /* a page went to writeback */
>> +
>> NR_BDI_STAT_ITEMS
>> };
>>
>>
>> ...
>>
>> @@ -680,28 +712,55 @@ static unsigned long bdi_position_ratio(struct backing_dev_info *bdi,
>> return 0;
>>
>> /*
>> - * global setpoint
>> + * The strictlimit feature is a tool preventing mistrusted filesystems
>> + * to grow a large number of dirty pages before throttling. For such
> "from growing"
>
>> + * filesystems balance_dirty_pages always checks bdi counters against
>> + * bdi limits. Even if global "nr_dirty" is under "freerun". This is
>> + * especially important for fuse who sets bdi->max_ratio to 1% by
> s/who/which/
>
>> + * default. Without strictlimit feature, fuse writeback may consume
>> + * arbitrary amount of RAM because it is accounted in
>> + * NR_WRITEBACK_TEMP which is not involved in calculating "nr_dirty".
>>
>> ...
>>
>> @@ -994,6 +1054,26 @@ static void bdi_update_dirty_ratelimit(struct backing_dev_info *bdi,
>> * keep that period small to reduce time lags).
>> */
>> step = 0;
>> +
>> + /*
>> + * For strictlimit case, balanced_dirty_ratelimit was calculated
> balance_dirty_ratelimit?
>
>> + * above based on bdi counters and limits (see bdi_position_ratio()).
>> + * Hence, to calculate "step" properly, we have to use bdi_dirty as
>> + * "dirty" and bdi_setpoint as "setpoint".
>> + *
>> + * We rampup dirty_ratelimit forcibly if bdi_dirty is low because
>> + * it's possible that bdi_thresh is close to zero due to inactivity
>> + * of backing device (see the implementation of bdi_dirty_limit()).
>> + */
>> + if (unlikely(strictlimit)) {
>> + dirty = bdi_dirty;
>> + if (bdi_dirty < 8)
>> + setpoint = bdi_dirty + 1;
>> + else
>>
>> ...
>>
>> @@ -1057,18 +1140,32 @@ void __bdi_update_bandwidth(struct backing_dev_info *bdi,
>> if (elapsed > HZ && time_before(bdi->bw_time_stamp, start_time))
>> goto snapshot;
>>
>> + /*
>> + * Skip periods when backing dev was idle due to abscence of pages
> "absence"
>
>> + * under writeback (when over_bground_thresh() returns false)
>> + */
>> + if (test_bit(BDI_idle, &bdi->state) &&
>> + bdi->writeback_nr_stamp == writeback)
>>
>> ...
>>
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-07-03 11:01 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-29 17:41 [PATCH v5 00/16] fuse: An attempt to implement a write-back cache policy Maxim Patlasov
2013-06-29 17:42 ` [PATCH 01/16] fuse: Linking file to inode helper Maxim Patlasov
2013-06-29 17:42 ` [PATCH 02/16] fuse: Getting file for writeback helper Maxim Patlasov
2013-06-29 17:42 ` [PATCH 04/16] fuse: Prepare to handle multiple pages in writeback Maxim Patlasov
2013-06-29 17:42 ` [PATCH 05/16] fuse: Connection bit for enabling writeback Maxim Patlasov
2013-06-29 17:44 ` [PATCH 06/16] fuse: Trust kernel i_size only - v4 Maxim Patlasov
2013-06-29 17:44 ` [PATCH 07/16] fuse: Trust kernel i_mtime only Maxim Patlasov
2013-07-11 16:14 ` [PATCH 07/16] fuse: Trust kernel i_mtime only -v2 Maxim Patlasov
2013-06-29 17:45 ` [PATCH 08/16] fuse: Flush files on wb close Maxim Patlasov
2013-07-11 16:18 ` [PATCH 08/16] fuse: Flush files on wb close -v2 Maxim Patlasov
[not found] ` <20130629172211.20175.70154.stgit-vWG5eQQidJHciZdyczg/7Q@public.gmane.org>
2013-06-29 17:42 ` [PATCH 03/16] fuse: Prepare to handle short reads Maxim Patlasov
2013-06-29 17:45 ` [PATCH 09/16] fuse: restructure fuse_readpage() Maxim Patlasov
2013-06-29 17:45 ` [PATCH 10/16] fuse: Implement writepages callback Maxim Patlasov
2013-07-19 16:50 ` Miklos Szeredi
2013-08-02 15:40 ` Maxim Patlasov
2013-08-06 16:25 ` Miklos Szeredi
2013-08-06 16:26 ` Eric Boxer
2013-08-09 15:02 ` Maxim Patlasov
2013-08-30 10:12 ` Miklos Szeredi
2013-08-30 14:50 ` Maxim Patlasov
2013-09-03 10:31 ` Miklos Szeredi
2013-09-03 16:02 ` Maxim Patlasov
2013-06-29 17:45 ` [PATCH 11/16] fuse: Implement write_begin/write_end callbacks Maxim Patlasov
2013-06-29 17:46 ` [PATCH 12/16] fuse: fuse_writepage_locked() should wait on writeback Maxim Patlasov
2013-06-29 17:46 ` [PATCH 13/16] fuse: fuse_flush() " Maxim Patlasov
2013-06-29 17:46 ` [PATCH 14/16] fuse: Fix O_DIRECT operations vs cached writeback misorder - v2 Maxim Patlasov
2013-06-29 17:47 ` [PATCH 15/16] fuse: Turn writeback cache on Maxim Patlasov
2013-06-29 17:48 ` [PATCH 16/16] mm: strictlimit feature Maxim Patlasov
2013-07-01 21:16 ` Andrew Morton
2013-07-02 8:33 ` Maxim Patlasov
2013-07-02 17:44 ` [PATCH] mm: strictlimit feature -v2 Maxim Patlasov
2013-07-02 19:38 ` Andrew Morton
2013-07-03 11:01 ` Maxim Patlasov [this message]
2013-07-03 23:16 ` Jan Kara
2013-07-05 13:14 ` Maxim Patlasov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51D4047F.2010700@parallels.com \
--to=mpatlasov@parallels.com \
--cc=akpm@linux-foundation.org \
--cc=bfoster@redhat.com \
--cc=dev@parallels.com \
--cc=devel@openvz.org \
--cc=fengguang.wu@intel.com \
--cc=fuse-devel@lists.sourceforge.net \
--cc=jbottomley@parallels.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=miklos@szeredi.hu \
--cc=riel@redhat.com \
--cc=viro@zeniv.linux.org.uk \
--cc=xemul@parallels.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).