Re: [PATCH RFC 09/22] block, cfq: replace CFQ with the BFQ-v0 I/O scheduler

From: Linus Walleij <linus.walleij@linaro.org>
To: Tejun Heo <tj@kernel.org>
Cc: Paolo Valente <paolo.valente@linaro.org>,
	Jens Axboe <axboe@kernel.dk>,
	Fabio Checconi <fchecconi@gmail.com>,
	Arianna Avanzini <avanzini.arianna@gmail.com>,
	linux-block@vger.kernel.org,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Ulf Hansson <ulf.hansson@linaro.org>,
	Mark Brown <broonie@kernel.org>
Subject: Re: [PATCH RFC 09/22] block, cfq: replace CFQ with the BFQ-v0 I/O scheduler
Date: Sat, 5 Mar 2016 00:29:39 +0700	[thread overview]
Message-ID: <CACRpkdYOjf3ZJsE4_EXiq0O-GoCje_FX9Q8A5JN+KcyBdtkZyw@mail.gmail.com> (raw)
In-Reply-To: <20160301184656.GI3965@htj.duckdns.org>

Hi Tejun,

I'm doing a summary of this discussion as a part of presenting
Linaro's involvement in Paolo's work. So I try to understand things.

What I'm happy to see is a convergence in the discussion on
timeslicing vs budgeting+time slicing, AFAICT there is consensus
that whatever tool does the job best should be employed, despite
some remaining disagreement on the scheduling unit (time or
bandwidth).

It also seems that BFQ delivers. It remains to be seen if it always
delivers.

The big question is this:

On Wed, Mar 2, 2016 at 1:46 AM, Tejun Heo <tj@kernel.org> wrote:

> For use cases where the differences in seekiness across workloads are
> accidental - e.g. all are trying to stream different files but some
> files are more fragmented by accident - using bandwidth as the
> resource unit would be helpful in mitigating the random gaps that the
> user shouldn't be bothered by, but that'd be focusing on a pretty
> narrow set of use cases.
>
> Workloads are varied and underlying device performs wildly differently
> depending on the specific IO pattern.

What strikes me from the overall discussion is the reference to use
cases, which ones are interesting for the I/O scheduler developers
and which are interesting for users?

I guess the current assumption is that CFQ address the needs of
the average user. The special deadline scheduler is for a special
class of users, not average. The noop scheduler is for testing.

So Paolo was asked to develop BFQ as a replacement plug-in for
CFQ on the assumption that it would better meet the needs of
the average user. He also already prepared a version that is a
selectable class like the deadline, but that was discouraged.

The notion you bring up, that BFQ may only address a narrow
set of use cases would merit it to be merged as a scheduler class
for special use cases alongside the deadline scheduler. (The only
objection I could see would be that it's a big piece of code to
maintain.)

But if it indeed meets the expectations of the "average user" the
current approach would be correct.

To me it seems it's very hard to proceed to conclude whether to
replace CFQ with BFQ or make it a special-case class without
first agreeing on what the average user use case(s) really is.
Does anyone really know? All references in this communication
seems rather handwavy.

What I know is that Paolo & his colleagues made a test suite for
BFQ that they are using, and I replicated their results with that
suite:
https://github.com/Algodev-github/S

As you can see it uses FIO to generate some (specified)
background noise and then tests some use cases such as:
- File copy
- Kernel development (git, compiles)
- Video playback
- Video streaming
- Startup of applications

Apart from the obvious that kernel devs may naively take themselves
as reference: git+compiles, that must be the most important
use case, and yes I noted my git checkouts are smackier with
BFQ but that's a subjective measure. This probably means the
average command line user issuing stuff to the left and right will
be happy with BFQ, especially since it has this boost for newly
forked processes.

There are geeksy usecases: for example video playback
on a set of random readers+writers trivially simulates someone
playing back 1080p video while downloading another one using
bittorrent in the background. That's geeky, but is it relevant for the
average user?

A NAS user for example (these boxes all run Linux more or less):
sequential reading and writing of large files is what they mostly do.
If the user downloads torrents onto the NAS, it's the former geeky
usecase.

But what is the usecase for a Google server, a NYSE transaction
system or an Oracle database? Can the people running these loads
tell us how to synthesize their typical workload using FIO and play
back something that looks like what they are doing? Is there a place
with these workload recipes that I'm obviously missing because of
being new to the block layer?

What would be helpful would be if people tested their favourite
workloads before and after this patch set and reported results.

My colleagues tried some tests on Android. It appears Android
starts up applications quicker (with cold cache) when there is
workload of dd-type sequential readers/writers in the background.
This corresponds to a smartphone user starting Candy Crush while
updating an application in the background, a usecase where Android
is known to be somewhat unresponsive. There are millions of
Android users, so this matters to them, we can add these seconds
up to years of people's lives across the userbase. (This claim may
need to be verified, but AFAICT it is genuine.)

So: can we figure out a set of FIO-based use cases that will
correspond to "average user"? That would help us determine
whether BFQ is addressing that or something fringe, so it needs
to be a special-case scheduler.

Yours,
Linus Walleij