[Question] io cgroup subsystem threaded support

[Question] io cgroup subsystem threaded support

[Odin Hultgren van der Horst]

Posted to both cgroups and linux-block

Hi i read though some of the source code for cgroups, and from my understanding
the io cgroup subsystem does not support threaded cgroups. So i had some questions
regarding this.

 - Is there any future plans for supporting threaded?
 - What are the main hurdles in adding threaded support to the io cgroup subsystem?

Thanks,
Odin.

Re: [Question] io cgroup subsystem threaded support

[Tejun Heo]

On Fri, Oct 01, 2021 at 01:06:45PM +0200, Odin Hultgren van der Horst wrote:
> Posted to both cgroups and linux-block
> 
> Hi i read though some of the source code for cgroups, and from my understanding
> the io cgroup subsystem does not support threaded cgroups. So i had some questions
> regarding this.
> 
>  - Is there any future plans for supporting threaded?

Not at the moment.

>  - What are the main hurdles in adding threaded support to the io cgroup subsystem?

The biggest is the fact that page cache pagse are owned by processes not
threads. A related issue is that writeback tracks ownership per inode with a
mechanism for transferring ownership when majority writer changes. Splitting
IO control per-thread would increase friction there. So, the summary is that
the kernel doesn't track related resource consumptions at thread
granularity.

Thanks.

-- 
tejun

Re: Re: [Question] io cgroup subsystem threaded support

[Odin Hultgren van der Horst]

> The biggest is the fact that page cache pagse are owned by processes not
> threads. A related issue is that writeback tracks ownership per inode with a
> mechanism for transferring ownership when majority writer changes. Splitting
> IO control per-thread would increase friction there. So, the summary is that
> the kernel doesn't track related resource consumptions at thread
> granularity.

So mainly the writeback functionality or is it ingrained in all
functionality like latency and priority?

If not, is there a subset of io cgroup subsys that could easily support threading?

And if so, could one either split the subsystem into two subsystems a
io_threaded and io or possibly add an option to disable the functionality
that restricts it from supporting threading?

Thanks.

Re: Re: [Question] io cgroup subsystem threaded support

[Michal Koutný]

Hello.

On Thu, Oct 07, 2021 at 03:39:16PM +0200, Odin Hultgren van der Horst <odin-KgRVb285F3Y0FkRhN5fn+A@public.gmane.org> wrote:
> So mainly the writeback functionality or is it ingrained in all
> functionality like latency and priority?

What do you want to achieve actually?

> If not, is there a subset of io cgroup subsys that could easily support threading?

I'd say such a subset would be similar to what can be done with
ioprio_set(2).

Michal

Re: [Question] io cgroup subsystem threaded support

[Odin Hultgren van der Horst]

On Thu, Oct 14, 2021 at 11:29:38AM +0200, Michal Koutný wrote:
> P.S. Is the reply only to me intentional or did you forget to hit
> reply-all?
Yeah, sorry forgot to hit reply-all.
Whats to proper procedure here, re-send the original mail to retain
history in the mailing list?

Thanks,
Odin

I have gotten an answer to this from Michal Koutný

Re: [Question] io cgroup subsystem threaded support

[Odin Hultgren van der Horst]

On Thu, Oct 14, 2021 at 11:29:38AM +0200, Michal Koutný wrote:
> On Thu, Oct 14, 2021 at 10:38:30AM +0200, Odin Hultgren van der Horst <odin-KgRVb285F3Y0FkRhN5fn+A@public.gmane.org> wrote:
> > On Mon, Oct 11, 2021 at 05:34:16PM +0200, Michal Koutný wrote:
> > > What do you want to achieve actually?
> > If a application use a thread per client that is connected, and a client
> > decides to do a large read on a io device it will cause all other clients
> > to be starved of io. I want to avoid this.
> 
> IIUC, the application would have to be collaborative anyway (dividing
> threads into cgroups) and given you're concered about reads,
> set_ioprio(2) might be enough for this case.
> 
> Also, if you've seen starvation (likely depends on IO scheduler), you
> may raise this with linux-block ML.
> 
> HTH,
> Michal

I also want to be able to limit amount of io for a given thread, just
like you could to with io.max. So a given client cant just use all my io
even though there is no competing thread.

Thanks,
Odin

Re: [Question] io cgroup subsystem threaded support

[Tejun Heo]

Hello,

On Fri, Oct 15, 2021 at 09:07:45AM +0200, Odin Hultgren van der Horst wrote:
> > IIUC, the application would have to be collaborative anyway (dividing
> > threads into cgroups) and given you're concered about reads,
> > set_ioprio(2) might be enough for this case.

One problem there is that set_ioprio(2) only works w/ bfq.

> > Also, if you've seen starvation (likely depends on IO scheduler), you
> > may raise this with linux-block ML.
> 
> I also want to be able to limit amount of io for a given thread, just
> like you could to with io.max. So a given client cant just use all my io
> even though there is no competing thread.

This isn't what cgroup is too useful at right now. CPU control is an
exception partly due to historical reasons but it's silly to enable support
for per-thread IO control when a chunk as big as buffered writes is not only
missing but doesn't even have a plausible future path for support.

Another rather fundamental problem is that cgroup interface ends up
entangling application behavior with system admin operations - e.g. whether
IO control is available for a given application is determined by not just
the kernel configuration but how the system's cgroup hierarchy is configured
at the moment and an application's configurations can be wiped out by system
configuration changing.

Being able to extend resource control to per-thread level can definitely be
useful but we'd need a lot more work in providing a usable programmable
interface which can make cgroup system configuration transparent to the
applications so that applications can basically do something like
set_ioprio(2) and have resource control working as best as possible on top
of the system configuration at the moment.

Thanks.

-- 
tejun