Re: Sum of weights idea for CFS PI

From: Joel Fernandes <joel@joelfernandes.org>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Qais Yousef <qais.yousef@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	juri.lelli@redhat.com, vincent.guittot@linaro.org,
	Youssef Esmat <youssefesmat@google.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	bristot@redhat.com, clark.williams@gmail.com,
	"Paul E. McKenney" <paulmck@kernel.org>
Subject: Re: Sum of weights idea for CFS PI
Date: Tue, 4 Oct 2022 10:43:33 -0400	[thread overview]
Message-ID: <f453e273-26b6-77c3-5511-6dc4fd5054db@joelfernandes.org> (raw)
In-Reply-To: <YzweGw5l3HxQVcGN@linutronix.de>

On 10/4/2022 7:50 AM, Sebastian Andrzej Siewior wrote:
> On 2022-09-30 13:34:49 [-0400], Joel Fernandes wrote:
>> In this case, there is no lock involved yet you have a dependency. But I don't
>> mean to sound depressing, and just because there are cases like this does not
>> mean we should not solve the lock-based ones. When I looked at Android, I saw
>> that it uses futex directly from Android Runtime code instead of using pthread.
>> So perhaps this can be trivially converted to FUTEX_LOCK_PI and then what we do
>> in the kernel will JustWork(Tm) ?
> 
> The easy part is just to replace the lock/unlock functions with
> FUTEX_LOCK_PI/UNLOCK_PI syscalls. The slightly advanced part is where
> you use an atomic operation to replace 0 with threads's ID in the lock
> path to avoid going into the kernel for locking if the lock is not
> contended. If it is, then you need to use the syscall.
> 
> …
>>> Proxy execution seems to be the nice solution to all of these problems, but
>>> it's a long way away. I'm interested to learn how this inheritance will be
>>> implemented. And whether there are any userspace conversion issues. i.e: do
>>> we need to convert all locks to rt-mutex locks?
>>
>> I am not an expert on FUTEX_LOCK_PI and this could be a good time for tglx to
>> weigh in, but I think converting all userspace locks to use FUTEX_LOCK_PI sounds
>> reasonable to me.
> 
> Based on my understanding with proxy-execution, all in-kernel locks
> should be covered.
> Priority inheritance (PI) works only with FUTEX_LOCK_PI for userpace and
> rtmutex for the in-kernel locks. Regular FUTEX_LOCK does only wait/wake
> in userspace so there is no way for the kernel to "help". Ah and for PI
> to work you need priorities that you can inherit. With two threads
> running as SCHED_OTHER there will be just "normal" sleep+wake in the
> kernel. If the blocking thread is SCHED_FIFO then it will inherit its
> priority to the lock owner.

Hi Sebastian, I agree with your thoughts on this. Yes proxy execution idea
should cover this. Basically, any primitive that allows userspace to let the
kernel know is a dependency can use this AFAICS, FUTEX_LOCK_PI being a prime
example. Perhaps Android's binder being another where A sends a message to C and
blocks till C responds. Meanwhile medium prio B blocks C.

thanks,

- Joel