All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vivek Goyal <vgoyal@redhat.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: linux-fsdevel@vger.kernel.org,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jan Kara <jack@suse.cz>, Christoph Hellwig <hch@lst.de>,
	Dave Chinner <david@fromorbit.com>,
	Greg Thelen <gthelen@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Andrea Righi <arighi@develer.com>, linux-mm <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 5/5] writeback: IO-less balance_dirty_pages()
Date: Thu, 18 Aug 2011 22:06:37 -0400	[thread overview]
Message-ID: <20110819020637.GA13597@redhat.com> (raw)
In-Reply-To: <20110816022329.190706384@intel.com>

On Tue, Aug 16, 2011 at 10:20:11AM +0800, Wu Fengguang wrote:

[..]
> +		if (dirty_exceeded && !bdi->dirty_exceeded)
>  			bdi->dirty_exceeded = 1;
>  
>  		bdi_update_bandwidth(bdi, dirty_thresh, background_thresh,
>  				     nr_dirty, bdi_thresh, bdi_dirty,
>  				     start_time);
>  
> -		/* Note: nr_reclaimable denotes nr_dirty + nr_unstable.
> -		 * Unstable writes are a feature of certain networked
> -		 * filesystems (i.e. NFS) in which data may have been
> -		 * written to the server's write cache, but has not yet
> -		 * been flushed to permanent storage.
> -		 * Only move pages to writeback if this bdi is over its
> -		 * threshold otherwise wait until the disk writes catch
> -		 * up.
> -		 */
> -		trace_balance_dirty_start(bdi);
> -		if (bdi_nr_reclaimable > task_bdi_thresh) {
> -			pages_written += writeback_inodes_wb(&bdi->wb,
> -							     write_chunk);
> -			trace_balance_dirty_written(bdi, pages_written);
> -			if (pages_written >= write_chunk)
> -				break;		/* We've done our duty */
> +		if (unlikely(!writeback_in_progress(bdi)))
> +			bdi_start_background_writeback(bdi);
> +
> +		base_rate = bdi->dirty_ratelimit;
> +		pos_ratio = bdi_position_ratio(bdi, dirty_thresh,
> +					       background_thresh, nr_dirty,
> +					       bdi_thresh, bdi_dirty);
> +		if (unlikely(pos_ratio == 0)) {
> +			pause = MAX_PAUSE;
> +			goto pause;
>  		}
> +		task_ratelimit = (u64)base_rate *
> +					pos_ratio >> RATELIMIT_CALC_SHIFT;

Hi Fenguaang,

I am little confused here. I see that you have already taken pos_ratio
into account in bdi_update_dirty_ratelimit() and wondering why to take
that into account again in balance_diry_pages().

We calculated the pos_rate and balanced_rate and adjusted the
bdi->dirty_ratelimit accordingly in bdi_update_dirty_ratelimit().

So why are we adjusting this pos_ratio() adjusted limit again with
pos_ratio(). Doesn't it become effectively following (assuming
one is decreasing the dirty rate limit).

base_rate = bdi->dirty_ratelimit
pos_rate = base_rate * pos_ratio();

			  write_bw
balance_rate = pos_rate * --------
			  dirty_bw

delta = max(pos_rate, balance_rate)
bdi->dirty_ratelimit = bdi->dirty_ratelimit - delta;

task_ratelimit = bdi->dirty_ratelimit * pos_ratio().

So we have already taken into account pos_ratio() while calculating new
bdi->dirty_ratelimit. Do we need to take that into account again.

Thanks
Vivek

WARNING: multiple messages have this Message-ID (diff)
From: Vivek Goyal <vgoyal@redhat.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: linux-fsdevel@vger.kernel.org,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jan Kara <jack@suse.cz>, Christoph Hellwig <hch@lst.de>,
	Dave Chinner <david@fromorbit.com>,
	Greg Thelen <gthelen@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Andrea Righi <arighi@develer.com>, linux-mm <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 5/5] writeback: IO-less balance_dirty_pages()
Date: Thu, 18 Aug 2011 22:06:37 -0400	[thread overview]
Message-ID: <20110819020637.GA13597@redhat.com> (raw)
In-Reply-To: <20110816022329.190706384@intel.com>

On Tue, Aug 16, 2011 at 10:20:11AM +0800, Wu Fengguang wrote:

[..]
> +		if (dirty_exceeded && !bdi->dirty_exceeded)
>  			bdi->dirty_exceeded = 1;
>  
>  		bdi_update_bandwidth(bdi, dirty_thresh, background_thresh,
>  				     nr_dirty, bdi_thresh, bdi_dirty,
>  				     start_time);
>  
> -		/* Note: nr_reclaimable denotes nr_dirty + nr_unstable.
> -		 * Unstable writes are a feature of certain networked
> -		 * filesystems (i.e. NFS) in which data may have been
> -		 * written to the server's write cache, but has not yet
> -		 * been flushed to permanent storage.
> -		 * Only move pages to writeback if this bdi is over its
> -		 * threshold otherwise wait until the disk writes catch
> -		 * up.
> -		 */
> -		trace_balance_dirty_start(bdi);
> -		if (bdi_nr_reclaimable > task_bdi_thresh) {
> -			pages_written += writeback_inodes_wb(&bdi->wb,
> -							     write_chunk);
> -			trace_balance_dirty_written(bdi, pages_written);
> -			if (pages_written >= write_chunk)
> -				break;		/* We've done our duty */
> +		if (unlikely(!writeback_in_progress(bdi)))
> +			bdi_start_background_writeback(bdi);
> +
> +		base_rate = bdi->dirty_ratelimit;
> +		pos_ratio = bdi_position_ratio(bdi, dirty_thresh,
> +					       background_thresh, nr_dirty,
> +					       bdi_thresh, bdi_dirty);
> +		if (unlikely(pos_ratio == 0)) {
> +			pause = MAX_PAUSE;
> +			goto pause;
>  		}
> +		task_ratelimit = (u64)base_rate *
> +					pos_ratio >> RATELIMIT_CALC_SHIFT;

Hi Fenguaang,

I am little confused here. I see that you have already taken pos_ratio
into account in bdi_update_dirty_ratelimit() and wondering why to take
that into account again in balance_diry_pages().

We calculated the pos_rate and balanced_rate and adjusted the
bdi->dirty_ratelimit accordingly in bdi_update_dirty_ratelimit().

So why are we adjusting this pos_ratio() adjusted limit again with
pos_ratio(). Doesn't it become effectively following (assuming
one is decreasing the dirty rate limit).

base_rate = bdi->dirty_ratelimit
pos_rate = base_rate * pos_ratio();

			  write_bw
balance_rate = pos_rate * --------
			  dirty_bw

delta = max(pos_rate, balance_rate)
bdi->dirty_ratelimit = bdi->dirty_ratelimit - delta;

task_ratelimit = bdi->dirty_ratelimit * pos_ratio().

So we have already taken into account pos_ratio() while calculating new
bdi->dirty_ratelimit. Do we need to take that into account again.

Thanks
Vivek

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-08-19  2:07 UTC|newest]

Thread overview: 82+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-08-16  2:20 [PATCH 0/5] IO-less dirty throttling v9 Wu Fengguang
2011-08-16  2:20 ` Wu Fengguang
2011-08-16  2:20 ` [PATCH 1/5] writeback: account per-bdi accumulated dirtied pages Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  2:20 ` [PATCH 2/5] writeback: dirty position control Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16 19:41   ` Jan Kara
2011-08-16 19:41     ` Jan Kara
2011-08-17 13:23     ` Wu Fengguang
2011-08-17 13:49       ` Wu Fengguang
2011-08-17 13:49         ` Wu Fengguang
2011-08-17 20:24       ` Jan Kara
2011-08-17 20:24         ` Jan Kara
2011-08-18  4:18         ` Wu Fengguang
2011-08-18  4:18           ` Wu Fengguang
2011-08-18  4:41           ` Wu Fengguang
2011-08-18  4:41             ` Wu Fengguang
2011-08-18 19:16           ` Jan Kara
2011-08-18 19:16             ` Jan Kara
2011-08-24  3:16         ` Wu Fengguang
2011-08-24  3:16           ` Wu Fengguang
2011-08-19  2:53   ` Vivek Goyal
2011-08-19  2:53     ` Vivek Goyal
2011-08-19  3:25     ` Wu Fengguang
2011-08-19  3:25       ` Wu Fengguang
2011-08-16  2:20 ` [PATCH 3/5] writeback: dirty rate control Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  2:20 ` [PATCH 4/5] writeback: per task dirty rate limit Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  7:17   ` Andrea Righi
2011-08-16  7:17     ` Andrea Righi
2011-08-16  7:22     ` Wu Fengguang
2011-08-16  7:22       ` Wu Fengguang
2011-08-16  2:20 ` [PATCH 5/5] writeback: IO-less balance_dirty_pages() Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-16  2:20   ` Wu Fengguang
2011-08-19  2:06   ` Vivek Goyal [this message]
2011-08-19  2:06     ` Vivek Goyal
2011-08-19  2:54     ` Wu Fengguang
2011-08-19  2:54       ` Wu Fengguang
2011-08-19 19:00       ` Vivek Goyal
2011-08-19 19:00         ` Vivek Goyal
2011-08-21  3:46         ` Wu Fengguang
2011-08-21  3:46           ` Wu Fengguang
2011-08-22 17:22           ` Vivek Goyal
2011-08-22 17:22             ` Vivek Goyal
2011-08-23  1:07             ` Wu Fengguang
2011-08-23  1:07               ` Wu Fengguang
2011-08-23  3:53               ` Wu Fengguang
2011-08-23  3:53                 ` Wu Fengguang
2011-08-23 13:53               ` Vivek Goyal
2011-08-23 13:53                 ` Vivek Goyal
2011-08-24  3:09                 ` Wu Fengguang
2011-08-24  3:09                   ` Wu Fengguang
  -- strict thread matches above, loose matches on Subject: below --
2011-08-06  8:44 [PATCH 0/5] IO-less dirty throttling v8 Wu Fengguang
2011-08-06  8:44 ` [PATCH 5/5] writeback: IO-less balance_dirty_pages() Wu Fengguang
2011-08-06  8:44   ` Wu Fengguang
2011-08-06  8:44   ` Wu Fengguang
2011-08-06 14:48   ` Andrea Righi
2011-08-06 14:48     ` Andrea Righi
2011-08-06 14:48     ` Andrea Righi
2011-08-07  6:44     ` Wu Fengguang
2011-08-07  6:44       ` Wu Fengguang
2011-08-07  6:44       ` Wu Fengguang
2011-08-06 16:46   ` Andrea Righi
2011-08-06 16:46     ` Andrea Righi
2011-08-07  7:18     ` Wu Fengguang
2011-08-07  9:50       ` Andrea Righi
2011-08-07  9:50         ` Andrea Righi
2011-08-09 18:15   ` Vivek Goyal
2011-08-09 18:15     ` Vivek Goyal
2011-08-09 18:41     ` Peter Zijlstra
2011-08-09 18:41       ` Peter Zijlstra
2011-08-09 18:41       ` Peter Zijlstra
2011-08-10  3:22       ` Wu Fengguang
2011-08-10  3:22         ` Wu Fengguang
2011-08-10  3:26     ` Wu Fengguang
2011-08-10  3:26       ` Wu Fengguang
2011-08-09 19:16   ` Vivek Goyal
2011-08-09 19:16     ` Vivek Goyal
2011-08-10  4:33     ` Wu Fengguang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110819020637.GA13597@redhat.com \
    --to=vgoyal@redhat.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=akpm@linux-foundation.org \
    --cc=arighi@develer.com \
    --cc=david@fromorbit.com \
    --cc=fengguang.wu@intel.com \
    --cc=gthelen@google.com \
    --cc=hch@lst.de \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=minchan.kim@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.