Re: [PATCH 3/5] writeback: dirty rate control

[Peter Zijlstra <peterz@infradead.org>]

How about something like the below, it still needs some more work, but
its more or less complete in that is now explains both controls in one
story. The actual update bit is still missing.

---

balance_dirty_pages() needs to throttle tasks dirtying pages such that
the total amount of dirty pages stays below the specified dirty limit in
order to avoid memory deadlocks. Furthermore we desire fairness in that
tasks get throttled proportionally to the amount of pages they dirty.

IOW we want to throttle tasks such that we match the dirty rate to the
writeout bandwidth, this yields a stable amount of dirty pages:

	ratelimit = writeout_bandwidth

The fairness requirements gives us:

	task_ratelimit = write_bandwidth / N

> : When started N dd, we would like to throttle each dd at
> : 
> :          balanced_rate == write_bw / N                                  (1)
> : 
> : We don't know N beforehand, but still can estimate balanced_rate
> : within 200ms.
> : 
> : Start by throttling each dd task at rate
> : 
> :         task_ratelimit = task_ratelimit_0                               (2)
> :                          (any non-zero initial value is OK)
> : 
> : After 200ms, we got
> : 
> :         dirty_rate = # of pages dirtied by all dd's / 200ms
> :         write_bw   = # of pages written to the disk / 200ms
> : 
> : For the aggressive dd dirtiers, the equality holds
> : 
> :         dirty_rate == N * task_rate
> :                    == N * task_ratelimit
> :                    == N * task_ratelimit_0                              (3)
> : Or
> :         task_ratelimit_0 = dirty_rate / N                               (4)
> :                           
> : So the balanced throttle bandwidth can be estimated by
> :                           
> :         balanced_rate = task_ratelimit_0 * (write_bw / dirty_rate)      (5)
> :                           
> : Because with (4) and (5) we can get the desired equality (1):
> :                           
> :         balanced_rate == (dirty_rate / N) * (write_bw / dirty_rate)
> :                       == write_bw / N

Then using the balance_rate we can compute task pause times like:

	task_pause = task->nr_dirtied / task_ratelimit

[ however all that still misses the primary feedback of:

   task_ratelimit_(i+1) = task_ratelimit_i * (write_bw / dirty_rate)

  there's still some confusion in the above due to task_ratelimit and
  balanced_rate.
]

However, while the above gives us means of matching the dirty rate to
the writeout bandwidth, it at best provides us with a stable dirty page
count (assuming a static system). In order to control the dirty page
count such that it is high enough to provide performance, but does not
exceed the specified limit we need another control.

> So if the dirty pages are ABOVE the setpoints, we throttle each task
> a bit more HEAVY than balanced_rate, so that the dirty pages are
> created less fast than they are cleaned, thus DROP to the setpoints
> (and the reverse). With that positional adjustment, the formula is
> transformed from
> 
>         task_ratelimit = balanced_rate
> 
> to
> 
>         task_ratelimit = balanced_rate * pos_ratio

> In terms of the negative feedback control theory, the
> bdi_position_ratio() function (control lines) can be expressed as
> 
> 1) f(setpoint) = 1.0
> 2) df/dt < 0
> 
> 3) optionally, abs(df/dt) should be large on large errors (= dirty -
>    setpoint) in order to cancel the errors fast, and be smaller when
>    dirty pages get closer to the setpoints in order to avoid overshooting.