Re: [PATCH 10/10] blkcg: implement BPF_PROG_TYPE_IO_COST

From: Alexei Starovoitov <ast@fb.com>
To: Tejun Heo <tj@kernel.org>, Quentin Monnet <quentin.monnet@netronome.com>
Cc: "axboe@kernel.dk" <axboe@kernel.dk>, Andy Newell <newella@fb.com>,
	"Chris Mason" <clm@fb.com>,
	"josef@toxicpanda.com" <josef@toxicpanda.com>,
	"Dennis Zhou" <dennisz@fb.com>,
	"lizefan@huawei.com" <lizefan@huawei.com>,
	"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	Kernel Team <Kernel-team@fb.com>,
	"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
	"ast@kernel.org" <ast@kernel.org>,
	"daniel@iogearbox.net" <daniel@iogearbox.net>,
	Martin Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>,
	"bpf@vger.kernel.org" <bpf@vger.kernel.org>
Subject: Re: [PATCH 10/10] blkcg: implement BPF_PROG_TYPE_IO_COST
Date: Fri, 14 Jun 2019 16:35:35 +0000
Message-ID: <bed0a66a-7aa6-ac36-9182-31a4937257e5@fb.com> (raw)
In-Reply-To: <20190614145239.GA538958@devbig004.ftw2.facebook.com>

On 6/14/19 7:52 AM, Tejun Heo wrote:
> Hello, Quentin.
> 
> On Fri, Jun 14, 2019 at 12:32:09PM +0100, Quentin Monnet wrote:
>> Please make sure to update the documentation and bash
>> completion when adding the new type to bpftool. You
>> probably want something like the diff below.
> 
> Thank you so much.  Will incorporate them.  Just in case, while it's
> noted in the head message, I lost the RFC marker while prepping this
> patch.  It isn't yet clear whether we'd really need custom cost
> functions and this patch is included more as a proof of concept.  

the example bpf prog looks flexible enough to allow some degree
of experiments. The question is what kind of new algorithms you envision
it will do? what other inputs it would need to make a decision?
I think it's ok to start with what it does now and extend further
when need arises.

> If
> it turns out that this is beneficial enough, the followings need to be
> answered.
> 
> * Is block ioctl the right mechanism to attach these programs?

imo ioctl is a bit weird, but since its only one program per block
device it's probably ok? Unless you see it being cgroup scoped in
the future? Then cgroup-bpf style hooks will be more suitable
and allow a chain of programs.

> * Are there more parameters that need to be exposed to the programs?
> 
> * It'd be great to have efficient access to per-blockdev and
>    per-blockdev-cgroup-pair storages available to these programs so
>    that they can keep track of history.  What'd be the best of way of
>    doing that considering the fact that these programs will be called
>    per each IO and the overhead can add up quickly?

Martin's socket local storage solved that issue for sockets.
Something very similar can work for per-blockdev-per-cgroup.