SOCKET_FILTER regression - eBPF can't subtract when attached from unprivileged user

From: Marek Majkowski <marek@cloudflare.com>
To: ast@kernel.org, Daniel Borkmann <daniel@iogearbox.net>,
	netdev@vger.kernel.org
Subject: SOCKET_FILTER regression - eBPF can't subtract when attached from unprivileged user
Date: Fri, 1 Mar 2019 02:06:42 +0100	[thread overview]
Message-ID: <CAJPywTJqP34cK20iLM5YmUMz9KXQOdu1-+BZrGMAGgLuBWz7fg@mail.gmail.com> (raw)

Howdy,

After some dramatic debugging, I think I managed to isolate a problem
that looks like a funny eBPF runtime regression. It seems to be
introduced somewhere after 4.14.

The eBPF in question is running on network sockets with
SO_ATTACH_BPF. The BPF_PROG_TYPE_SOCKET_FILTER code:

    uint64_t a = bpf_ktime_get_ns();
    uint64_t b = bpf_ktime_get_ns();
    uint64_t delta = b - a;
    if ((int64_t)delta > 0) {

Depending on a context, the "delta" variable is set to
a wrong value. The compiled bytecode seems fine:

Disassembly of section socket1:
bpf_prog1:
; {
       0:  85 00 00 00 05 00 00 00   call 5
; uint64_t a = bpf_ktime_get_ns();
       1:  bf 07 00 00 00 00 00 00   r7 = r0
; uint64_t b = bpf_ktime_get_ns();
       2:  85 00 00 00 05 00 00 00   call 5
       3:  bf 06 00 00 00 00 00 00   r6 = r0
; uint64_t delta = b - a;
       4:  bf 68 00 00 00 00 00 00   r8 = r6
       5:  1f 78 00 00 00 00 00 00   r8 -= r7
; if ((int64_t)delta > 0) {
       6:  b7 01 00 00 01 00 00 00   r1 = 1
       7:  6d 81 0a 00 00 00 00 00   if r1 s> r8 goto +10 XXX

The code runs fine from root. From unprivileged user though, the
value of "delta" is a wrapped negative. Both "a" and "b" are fine in
both root and non-root cases. Technically bpf_ktime_get_ns() can
go backwards, but this isn't the case here. It does seem like the
problem is with the behaiviour of ... the subtraction operation when
running from SO_ATTACH_BPF executed by non-root?

Code:
https://gist.github.com/majek/d0bb75a8c62cc35bec2b342054084aab

git clone https://gist.github.com/majek/d0bb75a8c62cc35bec2b342054084aab
cd d0bb75a8c62cc35bec2b342054084aab
make

Then:
$ sudo ./ebpf-bug
0 ->                    0 0x0000000000000000
1 ->       12585651690481 0x00000b72534c5bf1
2 ->       12585651690697 0x00000b72534c5cc9
3 ->                  216 0x00000000000000d8

$ ./ebpf-bug
0 ->                    1 0x0000000000000001
1 ->       12581437028489 0x00000b715815b889
2 ->       12581437028689 0x00000b715815b951
3 -> 18446731492272523127 0xfffff48ea7ea4777

"1" shows "a"
"2" shows "b"
"3" shows "detla"

As you see "delta" is off the scale for unprivileged user run. I
don't see any reason why root vs non-root should make any difference
for this code.

For completeness, this was tested with jit disabled:

$ cat /proc/version
Linux version 5.0.0-rc6+ (marek@mrbreeze) (gcc version 7.3.0)

$ sudo sysctl -a|grep -i jit
net.core.bpf_jit_enable = 0
net.core.bpf_jit_harden = 0

The same test on 4.14 seems fine:

$ cat /proc/version
Linux version 4.14.83-cloudflare-2018.11.4  (gcc version 8.2.0 (GCC))

$ sudo ./ebpf-bug
0 ->                    0 0x0000000000000000
1 ->     7435203111991321 0x001a6a472052b819
2 ->     7435203111991429 0x001a6a472052b885
3 ->                  108 0x000000000000006c

$ ./ebpf-bug
0 ->                    0 0x0000000000000000
1 ->     7435205114618775 0x001a6a4797b06397
2 ->     7435205114618883 0x001a6a4797b06403
3 ->                  108 0x000000000000006c

Cheers,
   Marek