[PATCH bpf-next] [tools/bpf] workaround a verifier failure for test_progs

[PATCH bpf-next] [tools/bpf] workaround a verifier failure for test_progs

[Yonghong Song]

With latest llvm compiler, running test_progs will have
the following verifier failure for test_sysctl_loop1.o:
  libbpf: load bpf program failed: Permission denied
  libbpf: -- BEGIN DUMP LOG ---
  libbpf:
  invalid indirect read from stack var_off (0x0; 0xff)+196 size 7
  ...
  libbpf: -- END LOG --
  libbpf: failed to load program 'cgroup/sysctl'
  libbpf: failed to load object 'test_sysctl_loop1.o'

The related bytecodes look below:
  0000000000000308 LBB0_8:
      97:       r4 = r10
      98:       r4 += -288
      99:       r4 += r7
     100:       w8 &= 255
     101:       r1 = r10
     102:       r1 += -488
     103:       r1 += r8
     104:       r2 = 7
     105:       r3 = 0
     106:       call 106
     107:       w1 = w0
     108:       w1 += -1
     109:       if w1 > 6 goto -24 <LBB0_5>
     110:       w0 += w8
     111:       r7 += 8
     112:       w8 = w0
     113:       if r7 != 224 goto -17 <LBB0_8>
and source code:
     for (i = 0; i < ARRAY_SIZE(tcp_mem); ++i) {
             ret = bpf_strtoul(value + off, MAX_ULONG_STR_LEN, 0,
                               tcp_mem + i);
             if (ret <= 0 || ret > MAX_ULONG_STR_LEN)
                     return 0;
             off += ret & MAX_ULONG_STR_LEN;
     }
Current verifier is not able to conclude register w8 at
insn 110 has a range of 1 to 7, which caused later
verifier complaint.

Let us workaound this issue until we found a compiler and/or
verifier solution. The workaround in this patch is
to make variable 'ret' volatile, which will force a reload
and then '&' operation to ensure better value range.
With this patch, I got:
  #3/17 test_sysctl_loop1.o:OK

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 tools/testing/selftests/bpf/progs/test_sysctl_loop1.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c b/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c
index 608a06871572..d22e438198cf 100644
--- a/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c
+++ b/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c
@@ -44,7 +44,10 @@ int sysctl_tcp_mem(struct bpf_sysctl *ctx)
 	unsigned long tcp_mem[TCP_MEM_LOOPS] = {};
 	char value[MAX_VALUE_STR_LEN];
 	unsigned char i, off = 0;
-	int ret;
+	/* a workaround to prevent compiler from generating
+	 * codes verifier cannot handle yet.
+	 */
+	volatile int ret;
 
 	if (ctx->write)
 		return 0;
-- 
2.17.1

Re: [PATCH bpf-next] [tools/bpf] workaround a verifier failure for test_progs

[Song Liu]

> On Nov 6, 2019, at 10:21 PM, Yonghong Song <yhs@fb.com> wrote:
> 
> With latest llvm compiler, running test_progs will have
> the following verifier failure for test_sysctl_loop1.o:
>  libbpf: load bpf program failed: Permission denied
>  libbpf: -- BEGIN DUMP LOG ---
>  libbpf:
>  invalid indirect read from stack var_off (0x0; 0xff)+196 size 7
>  ...
>  libbpf: -- END LOG --
>  libbpf: failed to load program 'cgroup/sysctl'
>  libbpf: failed to load object 'test_sysctl_loop1.o'
> 
> The related bytecodes look below:
>  0000000000000308 LBB0_8:
>      97:       r4 = r10
>      98:       r4 += -288
>      99:       r4 += r7
>     100:       w8 &= 255
>     101:       r1 = r10
>     102:       r1 += -488
>     103:       r1 += r8
>     104:       r2 = 7
>     105:       r3 = 0
>     106:       call 106
>     107:       w1 = w0
>     108:       w1 += -1
>     109:       if w1 > 6 goto -24 <LBB0_5>
>     110:       w0 += w8
>     111:       r7 += 8
>     112:       w8 = w0
>     113:       if r7 != 224 goto -17 <LBB0_8>
> and source code:
>     for (i = 0; i < ARRAY_SIZE(tcp_mem); ++i) {
>             ret = bpf_strtoul(value + off, MAX_ULONG_STR_LEN, 0,
>                               tcp_mem + i);
>             if (ret <= 0 || ret > MAX_ULONG_STR_LEN)
>                     return 0;
>             off += ret & MAX_ULONG_STR_LEN;
>     }
> Current verifier is not able to conclude register w8 at
> insn 110 has a range of 1 to 7, which caused later
> verifier complaint.
> 
> Let us workaound this issue until we found a compiler and/or
> verifier solution. The workaround in this patch is
> to make variable 'ret' volatile, which will force a reload
> and then '&' operation to ensure better value range.
> With this patch, I got:
>  #3/17 test_sysctl_loop1.o:OK
> 
> Signed-off-by: Yonghong Song <yhs@fb.com>

Could you please add the byte code after the patch to the commit
log? 

Thanks,
Song