Re: [PATCH] BPF: Disable on PREEMPT_RT

[Thomas Gleixner <tglx@linutronix.de>]

On Thu, 17 Oct 2019, Alexei Starovoitov wrote:
> On Thu, Oct 17, 2019 at 2:54 PM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > I'm all ears for an alternative solution. Here are the pain points:
> 
> Let's talk about them one by one.
> 
> >   #1) BPF disables preemption unconditionally with no way to do a proper RT
> >       substitution like most other infrastructure in the kernel provides
> >       via spinlocks or other locking primitives.
> 
> Kernel has a ton of code that disables preemption.
> Why BPF is somehow special?
> Are you saying RT kernel doesn't disable preemption at all?
> I'm complete noob in RT.

The basic principle of RT is to break up the arbitrary long
preemption/interrupt disabled sections of the mainline kernel.

Most preempt/interrupt disabled sections are implicit by taking locks
(spinlock, rwlock). Just a few are explicit by issuing
preempt/local_irq_disable()

RT substitutes spinlock/rwlock with RT aware counterparts which

 - Do not disable preemption/interrupts

 - Prevent migration to keep the implicit migrate disable semantics
   of preempt disable

 - Convert the underlying lock primitive to a priority inheritance aware
   mechanism, aka. rtmutex.

In order to make the above work, RT forces interrupt and soft interrupt
processing into thread context except for interrupts which are explicitely
marked as interrupt safe (IRQF_NOTHREAD).

As a consequence most of the kernel code becomes fully preemptible. Of
course there are still code parts which require that preemption/interrupts
are hard disabled. That's pretty much initial low level entry code, hard
interrupt handling code (which just wakes up the threads), context switch
code and some other rather low level functions (vmenter/exit ....).

That also requires that we have still locks which disable
preemption/interrupts. That's why we have raw_spinlock and
spinlock. spinlock is substituted with a RT primitive while raw_spinlock
behaves like the traditional spinlock on a non RT kernel (disables
preemption/interrupts).

But that also means any code which explcitely disables preemption or
interrupts without taking a spin/rw lock can trigger the following issues:

  - Calling into code which requires to be preemtible/sleepable on RT
    results in a might sleep splat.

  - Has in RT terms potentially unbound or undesired runtime length without
    any chance for the scheduler to control it.

Aside of that RT has a more strict view vs. lock ownership because almost
all lock primitives except real counting semaphores are substituted by
priority inheritance aware counterparts. PI aware locks have not only the
requirement that they can only be taken in preemptible context (see above),
they also have a strict locker == unlocker requirement for obvious reasons.
up_read_non_owner() can't obviously fulfil that requirement.

I surely answered more than your initial question and probably not enough,
so feel free to ask for clarification.

Thanks for caring!

       Thomas