Thanks for taking a look!

Find attached a trace file which illustrates that the device’s write
bandwidth (write_bw) decreases from the initial 100 MB/s down to,
eventually, 0 (not included in the trace). When seeing the
pathologically slow write-back performance, I observed write_bw=0!

The trace was generated with these tracepoints enabled:
echo 1 > /sys/kernel/debug/tracing/events/writeback/balance_dirty_pages/enable
echo 1 > /sys/kernel/debug/tracing/events/writeback/bdi_dirty_ratelimit/enable

I wonder why the measured write bandwidth decreases so much. Any thoughts?

On Tue, Mar 3, 2020 at 3:25 PM Tejun Heo <tj@kernel.org> wrote:
>
> On Tue, Mar 03, 2020 at 03:21:47PM +0100, Michael Stapelberg wrote:
> > Find attached trace.log (cat /sys/kernel/debug/tracing/trace) and
> > fuse-debug.log (FUSE daemon with timestamps).
> >
> > Does that tell you something, or do we need more data? (If so, how?)
>
> This is likely the culprit.
>
>  .... 1319822.406198: balance_dirty_pages: ... bdi_dirty=68 dirty_ratelimit=28 ...
>
> For whatever reason, bdp calculated that the dirty throttling
> threshold for the fuse device is 28 pages which is extremely low. Need
> to track down how that number came to be. I'm afraid from here on it'd
> mostly be reading source code and sprinkling printks around but the
> debugging really comes down to figuring out how we ended up with 68
> and 28.
>
> Thanks.
>
> --
> tejun