* bpf selftest failed in 5.4.210 kernel
@ 2022-08-17 18:22 RAJESH DASARI
2022-08-18 5:24 ` Greg KH
0 siblings, 1 reply; 14+ messages in thread
From: RAJESH DASARI @ 2022-08-17 18:22 UTC (permalink / raw)
To: stable; +Cc: linux-kernel, jean-philippe, df
[-- Attachment #1: Type: text/plain, Size: 754 bytes --]
Hi ,
We are running bpf selftests on 5.4.210 kernel version and we see that
test case 11 of test_align failed. Please find the below error.
selftests: bpf: test_align
Test 11: pointer variable subtraction ... Failed to find match 16:
R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
0xfffffffc)
# func#0 @0
# 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
# 0: (61) r2 = *(u32 *)(r1 +76)
# 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
# 1: (61) r3 = *(u32 *)(r1 +80)
For complete errors please see the attached file. The same test case
execution was successful in the 5.4.209 version , could you please let
me know any known issue with the recent changes in 5.4.210 and how to
fix these errors.
Thanks,
Rajesh Dasari.
[-- Attachment #2: test_align_bpf_error --]
[-- Type: application/octet-stream, Size: 7435 bytes --]
selftests: bpf: test_align
Test 11: pointer variable subtraction ... Failed to find match 16: R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0xfffffffc)
# func#0 @0
# 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
# 0: (61) r2 = *(u32 *)(r1 +76)
# 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
# 1: (61) r3 = *(u32 *)(r1 +80)
# 2: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 2: (bf) r0 = r2
# 3: R0_w=pkt(id=0,off=0,r=0,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 3: (07) r0 += 8
# 4: R0_w=pkt(id=0,off=8,r=0,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 4: (3d) if r3 >= r0 goto pc+1
# R0_w=pkt(id=0,off=8,r=0,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 5: R0_w=pkt(id=0,off=8,r=0,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 5: (95) exit
# 6: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
# 6: (71) r6 = *(u8 *)(r2 +0)
# 7: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R6_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 7: (bf) r7 = r6
# 8: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R6_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R7_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 8: (57) r6 &= 15
# 9: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R6_w=inv(id=0,umax_value=15,var_off=(0x0; 0xf)) R7_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 9: (67) r6 <<= 2
# 10: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R6_w=inv(id=0,umax_value=60,var_off=(0x0; 0x3c)) R7_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 10: (07) r6 += 14
11: (bf) r5 = r2
# 12: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R5_w=pkt(id=0,off=0,r=8,imm=0) R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 12: (1f) r5 -= r6
# last_idx 12 first_idx 0
# regs=40 stack=0 before 11: (bf) r5 = r2
# regs=40 stack=0 before 10: (07) r6 += 14
# regs=40 stack=0 before 9: (67) r6 <<= 2
# regs=40 stack=0 before 8: (57) r6 &= 15
# regs=40 stack=0 before 7: (bf) r7 = r6
# regs=40 stack=0 before 6: (71) r6 = *(u8 *)(r2 +0)
# 13: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82; 0x7c)) R6_w=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0
# 13: (67) r7 <<= 2
# 14: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82; 0x7c)) R6_w=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=inv(id=0,umax_value=1020,var_off=(0x0; 0x3fc)) R10=fp0
# 14: (07) r7 += 76
# 15: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82; 0x7c)) R6_w=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 15: (0f) r5 += r7
# last_idx 15 first_idx 0
# regs=80 stack=0 before 14: (07) r7 += 76
# regs=80 stack=0 before 13: (67) r7 <<= 2
# regs=80 stack=0 before 12: (1f) r5 -= r6
# regs=80 stack=0 before 11: (bf) r5 = r2
# regs=80 stack=0 before 10: (07) r6 += 14
# regs=80 stack=0 before 9: (67) r6 <<= 2
# regs=80 stack=0 before 8: (57) r6 &= 15
# regs=80 stack=0 before 7: (bf) r7 = r6
# regs=40 stack=0 before 6: (71) r6 = *(u8 *)(r2 +0)
6: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6_w=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 16: (bf) r4 = r5
# 17: R0_w=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=8,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R4_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6_w=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7_w=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 17: (07) r4 += 4
# 18: R0=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=8,imm=0) R3=pkt_end(id=0,off=0,imm=0) R4=pkt(id=2,off=4,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 18: (3d) if r3 >= r4 goto pc+1
# R0=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=8,imm=0) R3=pkt_end(id=0,off=0,imm=0) R4=pkt(id=2,off=4,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 19: R0=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=8,imm=0) R3=pkt_end(id=0,off=0,imm=0) R4=pkt(id=2,off=4,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 19: (95) exit
# 20: R0=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=8,imm=0) R3=pkt_end(id=0,off=0,imm=0) R4=pkt(id=2,off=4,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6=invP(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c)) R7=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 20: (61) r6 = *(u32 *)(r5 +0)
# 21: R0=pkt(id=0,off=8,r=8,imm=0) R1=ctx(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=8,imm=0) R3=pkt_end(id=0,off=0,imm=0) R4=pkt(id=2,off=4,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc)) R6_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=invP(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc)) R10=fp0
# 21: (95) exit
# processed 22 insns (limit 1000000) max_states_per_insn 0 total_states 1 peak_states 1 mark_read 1
# FAIL
# Results: 11 pass 1 fail
not ok 7 selftests: bpf: test_align # exit=1
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-17 18:22 bpf selftest failed in 5.4.210 kernel RAJESH DASARI
@ 2022-08-18 5:24 ` Greg KH
2022-08-18 7:55 ` Jean-Philippe Brucker
2022-08-18 18:33 ` RAJESH DASARI
0 siblings, 2 replies; 14+ messages in thread
From: Greg KH @ 2022-08-18 5:24 UTC (permalink / raw)
To: RAJESH DASARI; +Cc: stable, linux-kernel, jean-philippe, df
On Wed, Aug 17, 2022 at 09:22:00PM +0300, RAJESH DASARI wrote:
> Hi ,
>
> We are running bpf selftests on 5.4.210 kernel version and we see that
> test case 11 of test_align failed. Please find the below error.
>
> selftests: bpf: test_align
> Test 11: pointer variable subtraction ... Failed to find match 16:
> R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> 0xfffffffc)
> # func#0 @0
> # 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
> # 0: (61) r2 = *(u32 *)(r1 +76)
> # 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
> # 1: (61) r3 = *(u32 *)(r1 +80)
>
> For complete errors please see the attached file. The same test case
> execution was successful in the 5.4.209 version , could you please let
> me know any known issue with the recent changes in 5.4.210 and how to
> fix these errors.
Can you use 'git bisect' to find the offending commit?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-18 5:24 ` Greg KH
@ 2022-08-18 7:55 ` Jean-Philippe Brucker
2022-08-19 10:57 ` Greg KH
2022-08-18 18:33 ` RAJESH DASARI
1 sibling, 1 reply; 14+ messages in thread
From: Jean-Philippe Brucker @ 2022-08-18 7:55 UTC (permalink / raw)
To: Greg KH; +Cc: RAJESH DASARI, stable, linux-kernel, df
On Thu, Aug 18, 2022 at 07:24:03AM +0200, Greg KH wrote:
> On Wed, Aug 17, 2022 at 09:22:00PM +0300, RAJESH DASARI wrote:
> > Hi ,
> >
> > We are running bpf selftests on 5.4.210 kernel version and we see that
> > test case 11 of test_align failed. Please find the below error.
> >
> > selftests: bpf: test_align
> > Test 11: pointer variable subtraction ... Failed to find match 16:
> > R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> > 0xfffffffc)
> > # func#0 @0
> > # 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
> > # 0: (61) r2 = *(u32 *)(r1 +76)
> > # 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
> > # 1: (61) r3 = *(u32 *)(r1 +80)
> >
> > For complete errors please see the attached file. The same test case
> > execution was successful in the 5.4.209 version , could you please let
> > me know any known issue with the recent changes in 5.4.210 and how to
> > fix these errors.
>
> Can you use 'git bisect' to find the offending commit?
It probably is 6098562ed9df ("selftests/bpf: Fix "dubious pointer
arithmetic" test")
https://lore.kernel.org/all/20220803145005.2385039-6-ovidiu.panait@windriver.com/
Could you try reverting that?
The patch didn't have a Fixes: tags, because the bugfix it refers to was
merged at the same time. That bugfix is upstream commit b02709587ea3
("bpf: Fix propagation of 32-bit signed bounds from 64-bit bounds.")
Since b02709587ea3 was only backported down to 5.10, this fix shouldn't be
in 5.4. Sorry for not catching this earlier.
Thanks,
Jean
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-18 5:24 ` Greg KH
2022-08-18 7:55 ` Jean-Philippe Brucker
@ 2022-08-18 18:33 ` RAJESH DASARI
1 sibling, 0 replies; 14+ messages in thread
From: RAJESH DASARI @ 2022-08-18 18:33 UTC (permalink / raw)
To: Greg KH; +Cc: stable, linux-kernel, jean-philippe, df
I tried with git bisect between v5.4.210(bad) and v5.4.209(good).
Everytime I did bisect I compiled the kernel , booted my instance with
the new kernel and ran the selftests after trying out for 3 times ,
git bisect pointed to the below commit as a first bad commit.
git bisect bad
9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
commit 9d6f67365d9cdb389fbdac2bb5b00e59e345930e
Author: John Fastabend <john.fastabend@gmail.com>
Date: Wed Aug 3 17:50:03 2022 +0300
bpf: Test_verifier, #70 error message updates for 32-bit right shift
Thanks,
Rajesh Dasari.
On Thu, Aug 18, 2022 at 8:24 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Wed, Aug 17, 2022 at 09:22:00PM +0300, RAJESH DASARI wrote:
> > Hi ,
> >
> > We are running bpf selftests on 5.4.210 kernel version and we see that
> > test case 11 of test_align failed. Please find the below error.
> >
> > selftests: bpf: test_align
> > Test 11: pointer variable subtraction ... Failed to find match 16:
> > R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> > 0xfffffffc)
> > # func#0 @0
> > # 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
> > # 0: (61) r2 = *(u32 *)(r1 +76)
> > # 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
> > # 1: (61) r3 = *(u32 *)(r1 +80)
> >
> > For complete errors please see the attached file. The same test case
> > execution was successful in the 5.4.209 version , could you please let
> > me know any known issue with the recent changes in 5.4.210 and how to
> > fix these errors.
>
> Can you use 'git bisect' to find the offending commit?
>
> thanks,
>
> greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-18 7:55 ` Jean-Philippe Brucker
@ 2022-08-19 10:57 ` Greg KH
2022-08-19 16:20 ` RAJESH DASARI
0 siblings, 1 reply; 14+ messages in thread
From: Greg KH @ 2022-08-19 10:57 UTC (permalink / raw)
To: Jean-Philippe Brucker; +Cc: RAJESH DASARI, stable, linux-kernel, df
On Thu, Aug 18, 2022 at 08:55:16AM +0100, Jean-Philippe Brucker wrote:
> On Thu, Aug 18, 2022 at 07:24:03AM +0200, Greg KH wrote:
> > On Wed, Aug 17, 2022 at 09:22:00PM +0300, RAJESH DASARI wrote:
> > > Hi ,
> > >
> > > We are running bpf selftests on 5.4.210 kernel version and we see that
> > > test case 11 of test_align failed. Please find the below error.
> > >
> > > selftests: bpf: test_align
> > > Test 11: pointer variable subtraction ... Failed to find match 16:
> > > R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> > > 0xfffffffc)
> > > # func#0 @0
> > > # 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
> > > # 0: (61) r2 = *(u32 *)(r1 +76)
> > > # 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
> > > # 1: (61) r3 = *(u32 *)(r1 +80)
> > >
> > > For complete errors please see the attached file. The same test case
> > > execution was successful in the 5.4.209 version , could you please let
> > > me know any known issue with the recent changes in 5.4.210 and how to
> > > fix these errors.
> >
> > Can you use 'git bisect' to find the offending commit?
>
> It probably is 6098562ed9df ("selftests/bpf: Fix "dubious pointer
> arithmetic" test")
> https://lore.kernel.org/all/20220803145005.2385039-6-ovidiu.panait@windriver.com/
> Could you try reverting that?
>
> The patch didn't have a Fixes: tags, because the bugfix it refers to was
> merged at the same time. That bugfix is upstream commit b02709587ea3
> ("bpf: Fix propagation of 32-bit signed bounds from 64-bit bounds.")
>
> Since b02709587ea3 was only backported down to 5.10, this fix shouldn't be
> in 5.4. Sorry for not catching this earlier.
Can you send in a revert for this?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-19 10:57 ` Greg KH
@ 2022-08-19 16:20 ` RAJESH DASARI
2022-08-20 7:00 ` Greg KH
0 siblings, 1 reply; 14+ messages in thread
From: RAJESH DASARI @ 2022-08-19 16:20 UTC (permalink / raw)
To: Greg KH; +Cc: Jean-Philippe Brucker, stable, linux-kernel, df
[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]
Hi ,
I did some more tests , Please find the observation below.
step 1: On v5.4.210 kernel , I reverted only commit bpf: Verifer,
adjust_scalar_min_max_vals to always call update_reg_bounds()
7c1134c7da997523e2834dd516e2ddc51920699a , compiled the kernel and
booted the system with the new kernel.
step 2: On system with newly compiled kernel , I clone the v54.4.210
source code and reverted commit selftests/bpf: Fix test_align
verifier log patterns and selftests/bpf: Fix "dubious pointer
arithmetic" test , then ran the selftests, test_align test cases
execution was successful.
step 3: If i revert only selftests/bpf: Fix "dubious pointer
arithmetic" test , test cases are still failing.
Please find the attached PDF for the other scenarios which I have executed.
Thanks,
Rajesh Dasari.
On Fri, Aug 19, 2022 at 1:57 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Thu, Aug 18, 2022 at 08:55:16AM +0100, Jean-Philippe Brucker wrote:
> > On Thu, Aug 18, 2022 at 07:24:03AM +0200, Greg KH wrote:
> > > On Wed, Aug 17, 2022 at 09:22:00PM +0300, RAJESH DASARI wrote:
> > > > Hi ,
> > > >
> > > > We are running bpf selftests on 5.4.210 kernel version and we see that
> > > > test case 11 of test_align failed. Please find the below error.
> > > >
> > > > selftests: bpf: test_align
> > > > Test 11: pointer variable subtraction ... Failed to find match 16:
> > > > R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> > > > 0xfffffffc)
> > > > # func#0 @0
> > > > # 0: R1=ctx(id=0,off=0,imm=0) R10=fp0
> > > > # 0: (61) r2 = *(u32 *)(r1 +76)
> > > > # 1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
> > > > # 1: (61) r3 = *(u32 *)(r1 +80)
> > > >
> > > > For complete errors please see the attached file. The same test case
> > > > execution was successful in the 5.4.209 version , could you please let
> > > > me know any known issue with the recent changes in 5.4.210 and how to
> > > > fix these errors.
> > >
> > > Can you use 'git bisect' to find the offending commit?
> >
> > It probably is 6098562ed9df ("selftests/bpf: Fix "dubious pointer
> > arithmetic" test")
> > https://lore.kernel.org/all/20220803145005.2385039-6-ovidiu.panait@windriver.com/
> > Could you try reverting that?
> >
> > The patch didn't have a Fixes: tags, because the bugfix it refers to was
> > merged at the same time. That bugfix is upstream commit b02709587ea3
> > ("bpf: Fix propagation of 32-bit signed bounds from 64-bit bounds.")
> >
> > Since b02709587ea3 was only backported down to 5.10, this fix shouldn't be
> > in 5.4. Sorry for not catching this earlier.
>
> Can you send in a revert for this?
>
> thanks,
>
> greg k-h
[-- Attachment #2: selftest-on-5.4.210.pdf --]
[-- Type: application/pdf, Size: 48577 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-19 16:20 ` RAJESH DASARI
@ 2022-08-20 7:00 ` Greg KH
2022-08-22 19:23 ` RAJESH DASARI
0 siblings, 1 reply; 14+ messages in thread
From: Greg KH @ 2022-08-20 7:00 UTC (permalink / raw)
To: RAJESH DASARI; +Cc: Jean-Philippe Brucker, stable, linux-kernel, df
On Fri, Aug 19, 2022 at 07:20:11PM +0300, RAJESH DASARI wrote:
> Hi ,
>
> I did some more tests , Please find the observation below.
>
> step 1: On v5.4.210 kernel , I reverted only commit bpf: Verifer,
> adjust_scalar_min_max_vals to always call update_reg_bounds()
> 7c1134c7da997523e2834dd516e2ddc51920699a , compiled the kernel and
> booted the system with the new kernel.
> step 2: On system with newly compiled kernel , I clone the v54.4.210
> source code and reverted commit selftests/bpf: Fix test_align
> verifier log patterns and selftests/bpf: Fix "dubious pointer
> arithmetic" test , then ran the selftests, test_align test cases
> execution was successful.
> step 3: If i revert only selftests/bpf: Fix "dubious pointer
> arithmetic" test , test cases are still failing.
>
>
>
> Please find the attached PDF for the other scenarios which I have executed.
For obvious reasons, we can't read random .pdf files sent to us. Please
put it all in text.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-20 7:00 ` Greg KH
@ 2022-08-22 19:23 ` RAJESH DASARI
2022-08-23 7:04 ` Greg KH
0 siblings, 1 reply; 14+ messages in thread
From: RAJESH DASARI @ 2022-08-22 19:23 UTC (permalink / raw)
To: Greg KH; +Cc: Jean-Philippe Brucker, stable, linux-kernel, df
Hi,
Please find the test scenarios which I have tried.
Test 1:
Running system Kernel version (tag/commit) : v5.4.210
Kernel source code checkout : v5.4.210
test_align test case execution status : Failure
Test 2:
Running system Kernel version (tag/commit) : v5.4.210
Kernel source code checkout : v5.4.209
test_align test case execution status : Failure
Test 3:
Running system Kernel version (tag/commit) : v5.4.209
Kernel source code checkout : v5.4.209
test_align test case execution status : Success
Test 4:
Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
avoid spamming the console with old error logs ( Kernel compiled at
this commit and system is booted with this change)
Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
test_align verifier log patterns and selftests/bpf: Fix "dubious
pointer arithmetic" test. If I revert only the Fix "dubious pointer
arithmetic" test, the testcase still fails.
test_align test case execution status : Success
Test 5:
Running system Kernel version (tag/commit) : v5.4.210 but reverted
commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
update_reg_bounds() )
Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
test_align verifier log patterns and selftests/bpf: Fix "dubious
pointer arithmetic" test.
test_align test case execution status : Success
Test 6 :
Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
error message updates for 32-bit right shift( Kernel compiled at this
commit and system is booted with this change)
Kernel source code checkout : v5.4.209 or v5.4.210
test_align test case execution status : Failure
Thanks,
Rajesh Dasari.
On Sat, Aug 20, 2022 at 10:00 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Fri, Aug 19, 2022 at 07:20:11PM +0300, RAJESH DASARI wrote:
> > Hi ,
> >
> > I did some more tests , Please find the observation below.
> >
> > step 1: On v5.4.210 kernel , I reverted only commit bpf: Verifer,
> > adjust_scalar_min_max_vals to always call update_reg_bounds()
> > 7c1134c7da997523e2834dd516e2ddc51920699a , compiled the kernel and
> > booted the system with the new kernel.
> > step 2: On system with newly compiled kernel , I clone the v54.4.210
> > source code and reverted commit selftests/bpf: Fix test_align
> > verifier log patterns and selftests/bpf: Fix "dubious pointer
> > arithmetic" test , then ran the selftests, test_align test cases
> > execution was successful.
> > step 3: If i revert only selftests/bpf: Fix "dubious pointer
> > arithmetic" test , test cases are still failing.
> >
> >
> >
> > Please find the attached PDF for the other scenarios which I have executed.
>
> For obvious reasons, we can't read random .pdf files sent to us. Please
> put it all in text.
>
> thanks,
>
> greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-22 19:23 ` RAJESH DASARI
@ 2022-08-23 7:04 ` Greg KH
2022-08-23 7:31 ` RAJESH DASARI
0 siblings, 1 reply; 14+ messages in thread
From: Greg KH @ 2022-08-23 7:04 UTC (permalink / raw)
To: RAJESH DASARI; +Cc: Jean-Philippe Brucker, stable, linux-kernel, df
On Mon, Aug 22, 2022 at 10:23:02PM +0300, RAJESH DASARI wrote:
> Hi,
>
> Please find the test scenarios which I have tried.
>
> Test 1:
>
> Running system Kernel version (tag/commit) : v5.4.210
> Kernel source code checkout : v5.4.210
> test_align test case execution status : Failure
>
> Test 2:
>
> Running system Kernel version (tag/commit) : v5.4.210
> Kernel source code checkout : v5.4.209
> test_align test case execution status : Failure
>
> Test 3:
>
> Running system Kernel version (tag/commit) : v5.4.209
> Kernel source code checkout : v5.4.209
> test_align test case execution status : Success
>
> Test 4:
>
> Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
> avoid spamming the console with old error logs ( Kernel compiled at
> this commit and system is booted with this change)
> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> test_align verifier log patterns and selftests/bpf: Fix "dubious
> pointer arithmetic" test. If I revert only the Fix "dubious pointer
> arithmetic" test, the testcase still fails.
> test_align test case execution status : Success
>
> Test 5:
>
> Running system Kernel version (tag/commit) : v5.4.210 but reverted
> commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
> update_reg_bounds() )
> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> test_align verifier log patterns and selftests/bpf: Fix "dubious
> pointer arithmetic" test.
> test_align test case execution status : Success
>
> Test 6 :
>
> Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
> error message updates for 32-bit right shift( Kernel compiled at this
> commit and system is booted with this change)
> Kernel source code checkout : v5.4.209 or v5.4.210
> test_align test case execution status : Failure
I'm sorry, but I don't know what to do with this report at all.
Is there some failure somewhere? If you use 'git bisect' do you find
the offending commit?
confused,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-23 7:04 ` Greg KH
@ 2022-08-23 7:31 ` RAJESH DASARI
2022-08-23 18:34 ` Jean-Philippe Brucker
0 siblings, 1 reply; 14+ messages in thread
From: RAJESH DASARI @ 2022-08-23 7:31 UTC (permalink / raw)
To: Greg KH; +Cc: Jean-Philippe Brucker, stable, linux-kernel, df
Sorry for the confusion, results are indeed confusing to me .
If I try with git bisect I get
git bisect bad
9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
If I try to test myself with multiple test scenarios(I have mentioned
in the previous mails) for the bad commits , I see that bad commits
are
bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()
selftests/bpf: Fix test_align verifier log patterns
selftests/bpf: Fix "dubious pointer arithmetic" test
Thanks,
Rajesh Dasari.
On Tue, Aug 23, 2022 at 10:04 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, Aug 22, 2022 at 10:23:02PM +0300, RAJESH DASARI wrote:
> > Hi,
> >
> > Please find the test scenarios which I have tried.
> >
> > Test 1:
> >
> > Running system Kernel version (tag/commit) : v5.4.210
> > Kernel source code checkout : v5.4.210
> > test_align test case execution status : Failure
> >
> > Test 2:
> >
> > Running system Kernel version (tag/commit) : v5.4.210
> > Kernel source code checkout : v5.4.209
> > test_align test case execution status : Failure
> >
> > Test 3:
> >
> > Running system Kernel version (tag/commit) : v5.4.209
> > Kernel source code checkout : v5.4.209
> > test_align test case execution status : Success
> >
> > Test 4:
> >
> > Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
> > avoid spamming the console with old error logs ( Kernel compiled at
> > this commit and system is booted with this change)
> > Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> > test_align verifier log patterns and selftests/bpf: Fix "dubious
> > pointer arithmetic" test. If I revert only the Fix "dubious pointer
> > arithmetic" test, the testcase still fails.
> > test_align test case execution status : Success
> >
> > Test 5:
> >
> > Running system Kernel version (tag/commit) : v5.4.210 but reverted
> > commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
> > update_reg_bounds() )
> > Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> > test_align verifier log patterns and selftests/bpf: Fix "dubious
> > pointer arithmetic" test.
> > test_align test case execution status : Success
> >
> > Test 6 :
> >
> > Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
> > error message updates for 32-bit right shift( Kernel compiled at this
> > commit and system is booted with this change)
> > Kernel source code checkout : v5.4.209 or v5.4.210
> > test_align test case execution status : Failure
>
> I'm sorry, but I don't know what to do with this report at all.
>
> Is there some failure somewhere? If you use 'git bisect' do you find
> the offending commit?
>
> confused,
>
> greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-23 7:31 ` RAJESH DASARI
@ 2022-08-23 18:34 ` Jean-Philippe Brucker
2022-08-23 20:03 ` Ovidiu Panait
2022-08-23 20:25 ` Ovidiu Panait
0 siblings, 2 replies; 14+ messages in thread
From: Jean-Philippe Brucker @ 2022-08-23 18:34 UTC (permalink / raw)
To: RAJESH DASARI
Cc: Greg KH, stable, linux-kernel, ovidiu.panait, alexei.starovoitov,
john.fastabend
On Tue, Aug 23, 2022 at 10:31:40AM +0300, RAJESH DASARI wrote:
> Sorry for the confusion, results are indeed confusing to me .
> If I try with git bisect I get
>
> git bisect bad
> 9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
For me bisecting points to:
(A) 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()")
This changes the BPF verifier output and (as expected) breaks the
test_align selftest. That's why in the same series [1] another patch fixed
test_align. In v5.4.y, that patch is:
(B) 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
Unfortunately commit (B) addresses multiple verifier changes, not solely
(A). My guess is those changes were in series [1] and haven't been
backported to v5.4. So multiple solutions:
* Partially revert (B), only keeping the changes needed by (A)
* Revert (A) and (B)
* Add the missing commits that (B) also addresses
I don't know which, I suppose it depends on the intent behind backporting
(A). Ovidiu?
In any case 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic"
test") can be reverted, I can send that once we figure out the rest.
Thanks,
Jean
[1] https://lore.kernel.org/bpf/158507130343.15666.8018068546764556975.stgit@john-Precision-5820-Tower/
>
> If I try to test myself with multiple test scenarios(I have mentioned
> in the previous mails) for the bad commits , I see that bad commits
> are
> bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()
> selftests/bpf: Fix test_align verifier log patterns
> selftests/bpf: Fix "dubious pointer arithmetic" test
>
> Thanks,
> Rajesh Dasari.
>
> On Tue, Aug 23, 2022 at 10:04 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Mon, Aug 22, 2022 at 10:23:02PM +0300, RAJESH DASARI wrote:
> > > Hi,
> > >
> > > Please find the test scenarios which I have tried.
> > >
> > > Test 1:
> > >
> > > Running system Kernel version (tag/commit) : v5.4.210
> > > Kernel source code checkout : v5.4.210
> > > test_align test case execution status : Failure
> > >
> > > Test 2:
> > >
> > > Running system Kernel version (tag/commit) : v5.4.210
> > > Kernel source code checkout : v5.4.209
> > > test_align test case execution status : Failure
> > >
> > > Test 3:
> > >
> > > Running system Kernel version (tag/commit) : v5.4.209
> > > Kernel source code checkout : v5.4.209
> > > test_align test case execution status : Success
> > >
> > > Test 4:
> > >
> > > Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
> > > avoid spamming the console with old error logs ( Kernel compiled at
> > > this commit and system is booted with this change)
> > > Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> > > test_align verifier log patterns and selftests/bpf: Fix "dubious
> > > pointer arithmetic" test. If I revert only the Fix "dubious pointer
> > > arithmetic" test, the testcase still fails.
> > > test_align test case execution status : Success
> > >
> > > Test 5:
> > >
> > > Running system Kernel version (tag/commit) : v5.4.210 but reverted
> > > commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
> > > update_reg_bounds() )
> > > Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
> > > test_align verifier log patterns and selftests/bpf: Fix "dubious
> > > pointer arithmetic" test.
> > > test_align test case execution status : Success
> > >
> > > Test 6 :
> > >
> > > Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
> > > error message updates for 32-bit right shift( Kernel compiled at this
> > > commit and system is booted with this change)
> > > Kernel source code checkout : v5.4.209 or v5.4.210
> > > test_align test case execution status : Failure
> >
> > I'm sorry, but I don't know what to do with this report at all.
> >
> > Is there some failure somewhere? If you use 'git bisect' do you find
> > the offending commit?
> >
> > confused,
> >
> > greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-23 18:34 ` Jean-Philippe Brucker
@ 2022-08-23 20:03 ` Ovidiu Panait
2022-08-23 20:25 ` Ovidiu Panait
1 sibling, 0 replies; 14+ messages in thread
From: Ovidiu Panait @ 2022-08-23 20:03 UTC (permalink / raw)
To: Jean-Philippe Brucker, RAJESH DASARI
Cc: Greg KH, stable, linux-kernel, alexei.starovoitov, john.fastabend
Hi Jean-Philippe,
On 8/23/22 21:34, Jean-Philippe Brucker wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
>
> On Tue, Aug 23, 2022 at 10:31:40AM +0300, RAJESH DASARI wrote:
>> Sorry for the confusion, results are indeed confusing to me .
>> If I try with git bisect I get
>>
>> git bisect bad
>> 9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
> For me bisecting points to:
>
> (A) 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()")
>
> This changes the BPF verifier output and (as expected) breaks the
> test_align selftest. That's why in the same series [1] another patch fixed
> test_align. In v5.4.y, that patch is:
>
> (B) 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
>
> Unfortunately commit (B) addresses multiple verifier changes, not solely
> (A). My guess is those changes were in series [1] and haven't been
> backported to v5.4. So multiple solutions:
>
> * Partially revert (B), only keeping the changes needed by (A)
> * Revert (A) and (B)
> * Add the missing commits that (B) also addresses
>
> I don't know which, I suppose it depends on the intent behind backporting
> (A). Ovidiu?
The intent behind backporting 7c1134c7da99 ("bpf: Verifer,
adjust_scalar_min_max_vals to always call update_reg_bounds()") was to
fix CVE-2021-4159.
If we revert test 11 changes brought in by 6a9b3f0f3bad ("selftests/bpf:
Fix test_align verifier log patterns") backport, all test_align
testcases pass on my side:
diff --git a/tools/testing/selftests/bpf/test_align.c
b/tools/testing/selftests/bpf/test_align.c
index c9c9bdce9d6d..4726e3eca9b2 100644
--- a/tools/testing/selftests/bpf/test_align.c
+++ b/tools/testing/selftests/bpf/test_align.c
@@ -580,18 +580,18 @@ static struct bpf_align_test tests[] = {
/* Adding 14 makes R6 be (4n+2) */
{11,
"R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c))"},
/* Subtracting from packet pointer overflows
ubounds */
- {13,
"R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
0x7c)"},
+ {13,
"R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
0x7c))"},
/* New unknown value in R7 is (4n), >= 76 */
{15,
"R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc))"},
/* Adding it to packet pointer gives nice
bounds again */
- {16,
"R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
0xfffffffc)"},
+ {16,
"R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
0x7fc))"},
/* At the time the word size load is performed
from R5,
* its total fixed offset is NET_IP_ALIGN +
reg->off (0)
* which is 2. Then the variable offset is
(4n+2), so
* the total offset is 4-byte aligned and meets the
* load's requirements.
*/
- {20,
"R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2;
0xfffffffc)"},
+ {20,
"R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc))"},
},
},
};
root@intel-x86-64:~/bpf# ./test_align
Test 0: mov ... PASS
Test 1: shift ... PASS
Test 2: addsub ... PASS
Test 3: mul ... PASS
Test 4: unknown shift ... PASS
Test 5: unknown mul ... PASS
Test 6: packet const offset ... PASS
Test 7: packet variable offset ... PASS
Test 8: packet variable offset 2 ... PASS
Test 9: dubious pointer arithmetic ... PASS
Test 10: variable subtraction ... PASS
Test 11: pointer variable subtraction ... PASS
Results: 12 pass 0 fail
> In any case 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic"
> test") can be reverted, I can send that once we figure out the rest.
In my testing, with [1] and [2] applied, but without [3], the following
test_align selftest would still fail:
Test 9: dubious pointer arithmetic ... Failed to find match 9:
R5=inv(id=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2;
0x7fffffff7ffffffc)
[1] 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always
call update_reg_bounds()")
[2] 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
[3] 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic" test")
> Thanks,
> Jean
>
> [1] https://lore.kernel.org/bpf/158507130343.15666.8018068546764556975.stgit@john-Precision-5820-Tower/
>
>> If I try to test myself with multiple test scenarios(I have mentioned
>> in the previous mails) for the bad commits , I see that bad commits
>> are
>> bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()
>> selftests/bpf: Fix test_align verifier log patterns
>> selftests/bpf: Fix "dubious pointer arithmetic" test
>>
>> Thanks,
>> Rajesh Dasari.
>>
>> On Tue, Aug 23, 2022 at 10:04 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>>> On Mon, Aug 22, 2022 at 10:23:02PM +0300, RAJESH DASARI wrote:
>>>> Hi,
>>>>
>>>> Please find the test scenarios which I have tried.
>>>>
>>>> Test 1:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210
>>>> Kernel source code checkout : v5.4.210
>>>> test_align test case execution status : Failure
>>>>
>>>> Test 2:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210
>>>> Kernel source code checkout : v5.4.209
>>>> test_align test case execution status : Failure
>>>>
>>>> Test 3:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.209
>>>> Kernel source code checkout : v5.4.209
>>>> test_align test case execution status : Success
>>>>
>>>> Test 4:
>>>>
>>>> Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
>>>> avoid spamming the console with old error logs ( Kernel compiled at
>>>> this commit and system is booted with this change)
>>>> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
>>>> test_align verifier log patterns and selftests/bpf: Fix "dubious
>>>> pointer arithmetic" test. If I revert only the Fix "dubious pointer
>>>> arithmetic" test, the testcase still fails.
>>>> test_align test case execution status : Success
>>>>
>>>> Test 5:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210 but reverted
>>>> commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
>>>> update_reg_bounds() )
>>>> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
>>>> test_align verifier log patterns and selftests/bpf: Fix "dubious
>>>> pointer arithmetic" test.
>>>> test_align test case execution status : Success
>>>>
>>>> Test 6 :
>>>>
>>>> Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
>>>> error message updates for 32-bit right shift( Kernel compiled at this
>>>> commit and system is booted with this change)
>>>> Kernel source code checkout : v5.4.209 or v5.4.210
>>>> test_align test case execution status : Failure
>>> I'm sorry, but I don't know what to do with this report at all.
>>>
>>> Is there some failure somewhere? If you use 'git bisect' do you find
>>> the offending commit?
>>>
>>> confused,
>>>
>>> greg k-h
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-23 18:34 ` Jean-Philippe Brucker
2022-08-23 20:03 ` Ovidiu Panait
@ 2022-08-23 20:25 ` Ovidiu Panait
2022-08-24 13:55 ` Jean-Philippe Brucker
1 sibling, 1 reply; 14+ messages in thread
From: Ovidiu Panait @ 2022-08-23 20:25 UTC (permalink / raw)
To: Jean-Philippe Brucker, RAJESH DASARI
Cc: Greg KH, stable, linux-kernel, alexei.starovoitov, john.fastabend
Hi Jean-Philippe,
On 8/23/22 21:34, Jean-Philippe Brucker wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
>
> On Tue, Aug 23, 2022 at 10:31:40AM +0300, RAJESH DASARI wrote:
>> Sorry for the confusion, results are indeed confusing to me .
>> If I try with git bisect I get
>>
>> git bisect bad
>> 9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
> For me bisecting points to:
>
> (A) 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()")
>
> This changes the BPF verifier output and (as expected) breaks the
> test_align selftest. That's why in the same series [1] another patch fixed
> test_align. In v5.4.y, that patch is:
>
> (B) 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
>
> Unfortunately commit (B) addresses multiple verifier changes, not solely
> (A). My guess is those changes were in series [1] and haven't been
> backported to v5.4. So multiple solutions:
>
> * Partially revert (B), only keeping the changes needed by (A)
> * Revert (A) and (B)
> * Add the missing commits that (B) also addresses
>
> I don't know which, I suppose it depends on the intent behind backporting
> (A). Ovidiu?
The intent behind backporting 7c1134c7da99 ("bpf: Verifer,
adjust_scalar_min_max_vals to always call update_reg_bounds()") was to
fix CVE-2021-4159.
If we revert test 11 changes brought in by 6a9b3f0f3bad ("selftests/bpf:
Fix test_align verifier log patterns") backport, all test_align
testcases pass on my side:
diff --git a/tools/testing/selftests/bpf/test_align.c
b/tools/testing/selftests/bpf/test_align.c
index c9c9bdce9d6d..4726e3eca9b2 100644
--- a/tools/testing/selftests/bpf/test_align.c
+++ b/tools/testing/selftests/bpf/test_align.c
@@ -580,18 +580,18 @@ static struct bpf_align_test tests[] = {
/* Adding 14 makes R6 be (4n+2) */
{11,
"R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c))"},
/* Subtracting from packet pointer overflows
ubounds */
- {13,
"R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
0x7c)"},
+ {13,
"R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
0x7c))"},
/* New unknown value in R7 is (4n), >= 76 */
{15,
"R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc))"},
/* Adding it to packet pointer gives nice
bounds again */
- {16,
"R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
0xfffffffc)"},
+ {16,
"R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
0x7fc))"},
/* At the time the word size load is performed
from R5,
* its total fixed offset is NET_IP_ALIGN +
reg->off (0)
* which is 2. Then the variable offset is
(4n+2), so
* the total offset is 4-byte aligned and meets
the
* load's requirements.
*/
- {20,
"R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2;
0xfffffffc)"},
+ {20,
"R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2;
0x7fc))"},
},
},
};
root@intel-x86-64:~/bpf# ./test_align
Test 0: mov ... PASS
Test 1: shift ... PASS
Test 2: addsub ... PASS
Test 3: mul ... PASS
Test 4: unknown shift ... PASS
Test 5: unknown mul ... PASS
Test 6: packet const offset ... PASS
Test 7: packet variable offset ... PASS
Test 8: packet variable offset 2 ... PASS
Test 9: dubious pointer arithmetic ... PASS
Test 10: variable subtraction ... PASS
Test 11: pointer variable subtraction ... PASS
Results: 12 pass 0 fail
> In any case 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic"
> test") can be reverted, I can send that once we figure out the rest.
In my testing, with [1] and [2] applied, but without [3], the following
test_align selftest would still fail:
Test 9: dubious pointer arithmetic ... Failed to find match 9:
R5=inv(id=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2;
0x7fffffff7ffffffc)
[1] 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always
call update_reg_bounds()")
[2] 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
[3] 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic" test")
Ovidiu
> Thanks,
> Jean
>
> [1] https://lore.kernel.org/bpf/158507130343.15666.8018068546764556975.stgit@john-Precision-5820-Tower/
>
>> If I try to test myself with multiple test scenarios(I have mentioned
>> in the previous mails) for the bad commits , I see that bad commits
>> are
>> bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()
>> selftests/bpf: Fix test_align verifier log patterns
>> selftests/bpf: Fix "dubious pointer arithmetic" test
>>
>> Thanks,
>> Rajesh Dasari.
>>
>> On Tue, Aug 23, 2022 at 10:04 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>>> On Mon, Aug 22, 2022 at 10:23:02PM +0300, RAJESH DASARI wrote:
>>>> Hi,
>>>>
>>>> Please find the test scenarios which I have tried.
>>>>
>>>> Test 1:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210
>>>> Kernel source code checkout : v5.4.210
>>>> test_align test case execution status : Failure
>>>>
>>>> Test 2:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210
>>>> Kernel source code checkout : v5.4.209
>>>> test_align test case execution status : Failure
>>>>
>>>> Test 3:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.209
>>>> Kernel source code checkout : v5.4.209
>>>> test_align test case execution status : Success
>>>>
>>>> Test 4:
>>>>
>>>> Running system Kernel version (tag/commit) : ACPI: APEI: Better fix to
>>>> avoid spamming the console with old error logs ( Kernel compiled at
>>>> this commit and system is booted with this change)
>>>> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
>>>> test_align verifier log patterns and selftests/bpf: Fix "dubious
>>>> pointer arithmetic" test. If I revert only the Fix "dubious pointer
>>>> arithmetic" test, the testcase still fails.
>>>> test_align test case execution status : Success
>>>>
>>>> Test 5:
>>>>
>>>> Running system Kernel version (tag/commit) : v5.4.210 but reverted
>>>> commit (bpf: Verifer, adjust_scalar_min_max_vals to always call
>>>> update_reg_bounds() )
>>>> Kernel source code checkout : v5.4.210 but reverted selftests/bpf: Fix
>>>> test_align verifier log patterns and selftests/bpf: Fix "dubious
>>>> pointer arithmetic" test.
>>>> test_align test case execution status : Success
>>>>
>>>> Test 6 :
>>>>
>>>> Running system Kernel version (tag/commit) : bpf: Test_verifier, #70
>>>> error message updates for 32-bit right shift( Kernel compiled at this
>>>> commit and system is booted with this change)
>>>> Kernel source code checkout : v5.4.209 or v5.4.210
>>>> test_align test case execution status : Failure
>>> I'm sorry, but I don't know what to do with this report at all.
>>>
>>> Is there some failure somewhere? If you use 'git bisect' do you find
>>> the offending commit?
>>>
>>> confused,
>>>
>>> greg k-h
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: bpf selftest failed in 5.4.210 kernel
2022-08-23 20:25 ` Ovidiu Panait
@ 2022-08-24 13:55 ` Jean-Philippe Brucker
0 siblings, 0 replies; 14+ messages in thread
From: Jean-Philippe Brucker @ 2022-08-24 13:55 UTC (permalink / raw)
To: Ovidiu Panait
Cc: RAJESH DASARI, Greg KH, stable, linux-kernel, alexei.starovoitov,
john.fastabend
On Tue, Aug 23, 2022 at 11:25:15PM +0300, Ovidiu Panait wrote:
> Hi Jean-Philippe,
>
> On 8/23/22 21:34, Jean-Philippe Brucker wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> >
> > On Tue, Aug 23, 2022 at 10:31:40AM +0300, RAJESH DASARI wrote:
> > > Sorry for the confusion, results are indeed confusing to me .
> > > If I try with git bisect I get
> > >
> > > git bisect bad
> > > 9d6f67365d9cdb389fbdac2bb5b00e59e345930e is the first bad commit
> > For me bisecting points to:
> >
> > (A) 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always call update_reg_bounds()")
> >
> > This changes the BPF verifier output and (as expected) breaks the
> > test_align selftest. That's why in the same series [1] another patch fixed
> > test_align. In v5.4.y, that patch is:
> >
> > (B) 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
> >
> > Unfortunately commit (B) addresses multiple verifier changes, not solely
> > (A). My guess is those changes were in series [1] and haven't been
> > backported to v5.4. So multiple solutions:
> >
> > * Partially revert (B), only keeping the changes needed by (A)
> > * Revert (A) and (B)
> > * Add the missing commits that (B) also addresses
> >
> > I don't know which, I suppose it depends on the intent behind backporting
> > (A). Ovidiu?
> The intent behind backporting 7c1134c7da99 ("bpf: Verifer,
> adjust_scalar_min_max_vals to always call update_reg_bounds()") was to fix
> CVE-2021-4159.
>
> If we revert test 11 changes brought in by 6a9b3f0f3bad ("selftests/bpf: Fix
> test_align verifier log patterns") backport, all test_align testcases pass
> on my side:
>
> diff --git a/tools/testing/selftests/bpf/test_align.c
> b/tools/testing/selftests/bpf/test_align.c
> index c9c9bdce9d6d..4726e3eca9b2 100644
> --- a/tools/testing/selftests/bpf/test_align.c
> +++ b/tools/testing/selftests/bpf/test_align.c
> @@ -580,18 +580,18 @@ static struct bpf_align_test tests[] = {
> /* Adding 14 makes R6 be (4n+2) */
> {11,
> "R6_w=inv(id=0,umin_value=14,umax_value=74,var_off=(0x2; 0x7c))"},
> /* Subtracting from packet pointer overflows ubounds
> */
> - {13, "R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
> 0x7c)"},
> + {13, "R5_w=pkt(id=1,off=0,r=8,umin_value=18446744073709551542,umax_value=18446744073709551602,var_off=(0xffffffffffffff82;
> 0x7c))"},
> /* New unknown value in R7 is (4n), >= 76 */
> {15,
> "R7_w=inv(id=0,umin_value=76,umax_value=1096,var_off=(0x0; 0x7fc))"},
> /* Adding it to packet pointer gives nice bounds
> again */
> - {16,
> "R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> 0xfffffffc)"},
> + {16,
> "R5_w=pkt(id=2,off=0,r=0,umin_value=2,umax_value=1082,var_off=(0x2;
> 0x7fc))"},
> /* At the time the word size load is performed from
> R5,
> * its total fixed offset is NET_IP_ALIGN + reg->off
> (0)
> * which is 2. Then the variable offset is (4n+2),
> so
> * the total offset is 4-byte aligned and meets the
> * load's requirements.
> */
> - {20,
> "R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2;
> 0xfffffffc)"},
> + {20,
> "R5=pkt(id=2,off=0,r=4,umin_value=2,umax_value=1082,var_off=(0x2; 0x7fc))"},
> },
> },
> };
>
> root@intel-x86-64:~/bpf# ./test_align
> Test 0: mov ... PASS
> Test 1: shift ... PASS
> Test 2: addsub ... PASS
> Test 3: mul ... PASS
> Test 4: unknown shift ... PASS
> Test 5: unknown mul ... PASS
> Test 6: packet const offset ... PASS
> Test 7: packet variable offset ... PASS
> Test 8: packet variable offset 2 ... PASS
> Test 9: dubious pointer arithmetic ... PASS
> Test 10: variable subtraction ... PASS
> Test 11: pointer variable subtraction ... PASS
> Results: 12 pass 0 fail
> > In any case 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic"
> > test") can be reverted, I can send that once we figure out the rest.
>
> In my testing, with [1] and [2] applied, but without [3], the following
> test_align selftest would still fail:
>
> Test 9: dubious pointer arithmetic ... Failed to find match 9:
> R5=inv(id=0,umin_value=2,umax_value=9223372034707292158,var_off=(0x2;
> 0x7fffffff7ffffffc)
Right thanks for the details, so I think the cleanest is to revert [3] and
partially revert [2], tests 11 and part of 9. I'll send that out
Thanks,
Jean
>
>
> [1] 7c1134c7da99 ("bpf: Verifer, adjust_scalar_min_max_vals to always call
> update_reg_bounds()")
> [2] 6a9b3f0f3bad ("selftests/bpf: Fix test_align verifier log patterns")
> [3] 6098562ed9df ("selftests/bpf: Fix "dubious pointer arithmetic" test")
>
> Ovidiu
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2022-08-24 13:57 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-17 18:22 bpf selftest failed in 5.4.210 kernel RAJESH DASARI
2022-08-18 5:24 ` Greg KH
2022-08-18 7:55 ` Jean-Philippe Brucker
2022-08-19 10:57 ` Greg KH
2022-08-19 16:20 ` RAJESH DASARI
2022-08-20 7:00 ` Greg KH
2022-08-22 19:23 ` RAJESH DASARI
2022-08-23 7:04 ` Greg KH
2022-08-23 7:31 ` RAJESH DASARI
2022-08-23 18:34 ` Jean-Philippe Brucker
2022-08-23 20:03 ` Ovidiu Panait
2022-08-23 20:25 ` Ovidiu Panait
2022-08-24 13:55 ` Jean-Philippe Brucker
2022-08-18 18:33 ` RAJESH DASARI
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.