archive mirror
 help / color / mirror / Atom feed
From: Jan Kara <>
To: Paolo Valente <>
Cc: Jan Kara <>,
Subject: Re: False waker detection in BFQ
Date: Thu, 26 Aug 2021 19:51:03 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Thu 26-08-21 11:45:17, Paolo Valente wrote:
> > Il giorno 25 ago 2021, alle ore 18:43, Jan Kara <> ha scritto:
> > 
> > On Mon 23-08-21 18:06:18, Jan Kara wrote:
> >> On Mon 23-08-21 15:58:25, Paolo Valente wrote:
> >>>> Currently I'm running wider set of benchmarks for the patches to see
> >>>> whether I didn't regress anything else. If not, I'll post the patches to
> >>>> the list.
> >>> 
> >>> Any news?
> >> 
> >> It took a while for all those benchmarks to run. Overall results look sane,
> >> I'm just verifying by hand now whether some of the localized regressions
> >> (usually specific to a particular fs+machine config) are due to a measurement
> >> noise or real regressions...
> > 
> > OK, so after some manual analysis I've found out that dbench indeed becomes
> > more noisy with my changes for high numbers of processes. I'm leaving for
> > vacation soon so I will not be probably able to debug it before I leave but
> > let me ask you one thing: The problematic change seems to be mostly a
> > revert of 7cc4ffc55564 ("block, bfq: put reqs of waker and woken in
> > dispatch list") and I'm currently puzzled why it has such an effect. What
> > I've found out is that 7cc4ffc55564 results in IO of higher priority
> > process being injected into the time slice of lower priority process and
> > thus there's always only single busy queue (of the lower priority process)
> > and thus higher priority process queue never gets scheduled. As a result
> > higher priority IO always competes with lower priority IO and there's no
> > service differentiation (we get 50/50 split of throughput between the
> > processes despite different IO priorities).
> I need a little help here.  Since high-priority I/O is immediately
> injected, I wonder why it does not receive all the bandwidth it
> demands.  Maybe, from your analysis, you have an answer.  Perhaps it
> happens because:
> 1) high-priority I/O is FIFO-queued with lower-priority I/O in the
>    dispatch list?

Yes, this is the case.

> >  And this scenario shows that
> > always injecting IO of waker/wakee isn't desirable, especially in a way as
> > done in 7cc4ffc55564 where injected IO isn't accounted within BFQ at all
> > (which easily allows for service degradation unnoticed by BFQ).
> Not sure that accounting would help high-priority I/O in your scenario.

It did help noticeably. Because then both high and low priority bfq queues
become busy so bfq_select_queue() sees both queues and schedules higher
priority queue.

> >  That's why
> > I've basically reverted that commit on the ground that on next dispatch we
> > call bfq_select_queue() which will see waker/wakee has IO to do and can
> > decide to inject the IO anyway. We do more CPU work but the IO pattern
> > should be similar. But apparently I was wrong :)
> For the pattern to be similar, I guess that, when new high-priority
> I/O arrives, this I/O should preempt lower-priority I/O.
> Unfortunately, this is not always the case, depending on other
> parameters.  Waker/wakee I/O is guaranteed to be injected only when the
> in-service queue has no I/O.
> At any rate, probably we can solve this puzzle by just analyzing a
> trace in which you detect a loss of throughput without 7cc4ffc55564.
> The best case would be one with the minimum possible number of
> threads, to get a simpler trace.

Yeah, OK, I'll gather the trace once I return from vacation and look into
it. Thanks for help!

Jan Kara <>

      reply	other threads:[~2021-08-26 17:51 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-05 16:20 False waker detection in BFQ Jan Kara
2021-05-20 15:05 ` Paolo Valente
2021-05-21 13:10   ` Jan Kara
2021-08-13 14:01   ` Jan Kara
2021-08-23 13:58     ` Paolo Valente
2021-08-23 16:06       ` Jan Kara
2021-08-25 16:43         ` Jan Kara
2021-08-26  9:45           ` Paolo Valente
2021-08-26 17:51             ` Jan Kara [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).