bpf_override_return out of order execution?

bpf_override_return out of order execution?

[Giulia]

Hi all,

I'm experimenting with the bpf_override_return() helper for the
copyout function (using kernel 5.4) to the whitelist. (
https://elixir.bootlin.com/linux/v5.4/source/lib/iov_iter.c#L138 )
My goal is to avoid the buffer copy from kernel to user that happens
in copyout, so I'm calling  bpf_override_return with return value 0 in
a kprobe.

It works most of the times, but when I test the function with
relatively many iterations of a read from file I find that sometimes
the copyout is actually executed with the buffer being copied.

Below is an execution output with sample parameters and with the kinds
of numbers I usually find

The numbers match with debug printks in the copyout function that I
find in dmesg, so I'm quite positive that the function actually gets
called.

The counter in the bpf kprobe arrives to 10000 executions which is
what I am expecting, so the only explanation I have for now is that
the kprobe execution is reordered and executed while the copyout is
already triggered, and the instruction pointer does not get
effectively diverted on time in the bpf_override_return. Could this be
the case? Is there any potential security implication also for cases
outside of mine?

Any insight will be highly appreciated! Thank you for your time,
Giulia

---------------------------------------------------------
read size: 4096
iters: 10000

success: 9725
fail: 275

kprobe executed 10000 times
---------------------------------------------------------

Re: bpf_override_return out of order execution?

[Alexei Starovoitov]

On Wed, Apr 29, 2020 at 7:50 AM Giulia <giulia.frascaria@gmail.com> wrote:
>
> Hi all,
>
> I'm experimenting with the bpf_override_return() helper for the
> copyout function (using kernel 5.4) to the whitelist. (
> https://elixir.bootlin.com/linux/v5.4/source/lib/iov_iter.c#L138 )
> My goal is to avoid the buffer copy from kernel to user that happens
> in copyout, so I'm calling  bpf_override_return with return value 0 in
> a kprobe.
>
> It works most of the times, but when I test the function with
> relatively many iterations of a read from file I find that sometimes
> the copyout is actually executed with the buffer being copied.
>
> Below is an execution output with sample parameters and with the kinds
> of numbers I usually find
>
> The numbers match with debug printks in the copyout function that I
> find in dmesg, so I'm quite positive that the function actually gets
> called.
>
> The counter in the bpf kprobe arrives to 10000 executions which is
> what I am expecting, so the only explanation I have for now is that
> the kprobe execution is reordered and executed while the copyout is
> already triggered, and the instruction pointer does not get
> effectively diverted on time in the bpf_override_return. Could this be
> the case? Is there any potential security implication also for cases
> outside of mine?

kprobe+bpf won't get reordered but there are few limitations in kprobes.
First is kprobe maxactive.
"The maximum number of instances of the probed function that
 * can be active concurrently"
And another is per-cpu bpf_prog_active counter that
allows only one bpf prog attached to kprobe execute on a cpu.

Do you have more than one kprobe ?
In such cases other active kprobe+bpf may suppress the one
attached to copyout.

If you can upgrade to the latest kernel bpf_modify_return program
type is faster and doesn't have these limitations.
See example in selftests/bpf/progs/modify_return.c