* [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
@ 2023-08-29 9:39 syzbot
2023-08-29 18:29 ` Yonghong Song
0 siblings, 1 reply; 6+ messages in thread
From: syzbot @ 2023-08-29 9:39 UTC (permalink / raw)
To: andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa, kpsingh,
linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs
Hello,
syzbot found the following issue on:
HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000
kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25
dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com
==================================================================
BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update
write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
bpf_long_memcpy include/linux/bpf.h:428 [inline]
bpf_obj_memcpy include/linux/bpf.h:441 [inline]
copy_map_value_long include/linux/bpf.h:464 [inline]
bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
__sys_bpf+0x28a/0x780
__do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
__x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
bpf_long_memcpy include/linux/bpf.h:428 [inline]
bpf_obj_memcpy include/linux/bpf.h:441 [inline]
copy_map_value_long include/linux/bpf.h:464 [inline]
bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
__sys_bpf+0x28a/0x780
__do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
__x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
value changed: 0x0000000000000000 -> 0xfffffff000002788
Reported by Kernel Concurrency Sanitizer on:
CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
==================================================================
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
2023-08-29 9:39 [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) syzbot
@ 2023-08-29 18:29 ` Yonghong Song
2023-08-29 18:53 ` Marco Elver
0 siblings, 1 reply; 6+ messages in thread
From: Yonghong Song @ 2023-08-29 18:29 UTC (permalink / raw)
To: syzbot, andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa,
kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs,
yhs
On 8/29/23 5:39 AM, syzbot wrote:
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k..
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000
> kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25
> dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com
>
> ==================================================================
> BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update
>
> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> bpf_long_memcpy include/linux/bpf.h:428 [inline]
> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> copy_map_value_long include/linux/bpf.h:464 [inline]
> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> __sys_bpf+0x28a/0x780
> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
> bpf_long_memcpy include/linux/bpf.h:428 [inline]
> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> copy_map_value_long include/linux/bpf.h:464 [inline]
> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> __sys_bpf+0x28a/0x780
> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
> value changed: 0x0000000000000000 -> 0xfffffff000002788
>
> Reported by Kernel Concurrency Sanitizer on:
> CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
> ==================================================================
This case is with two tasks doing bpf_map batch update together for the
same map and key.
> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
So concurrency is introduced by user applications.
In my opinion, this probably not an issue from kernel perspective.
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the bug is already fixed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want to overwrite bug's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the bug is a duplicate of another bug, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
2023-08-29 18:29 ` Yonghong Song
@ 2023-08-29 18:53 ` Marco Elver
2023-08-29 20:04 ` Daniel Borkmann
0 siblings, 1 reply; 6+ messages in thread
From: Marco Elver @ 2023-08-29 18:53 UTC (permalink / raw)
To: yonghong.song
Cc: syzbot, andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa,
kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs,
yhs, Paul E. McKenney
On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote:
>
>
>
> On 8/29/23 5:39 AM, syzbot wrote:
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k..
> > git tree: upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25
> > dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f
> > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> >
> > Unfortunately, I don't have any reproducer for this issue yet.
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com
> >
> > ==================================================================
> > BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update
> >
> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> > bpf_long_memcpy include/linux/bpf.h:428 [inline]
> > bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> > copy_map_value_long include/linux/bpf.h:464 [inline]
> > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> > __sys_bpf+0x28a/0x780
> > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> > do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> > entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >
> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
> > bpf_long_memcpy include/linux/bpf.h:428 [inline]
> > bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> > copy_map_value_long include/linux/bpf.h:464 [inline]
> > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> > __sys_bpf+0x28a/0x780
> > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> > do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> > entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >
> > value changed: 0x0000000000000000 -> 0xfffffff000002788
> >
> > Reported by Kernel Concurrency Sanitizer on:
> > CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
> > ==================================================================
>
> This case is with two tasks doing bpf_map batch update together for the
> same map and key.
> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
>
> So concurrency is introduced by user applications.
> In my opinion, this probably not an issue from kernel perspective.
Perhaps not, but I recall there being a discussion about making KCSAN
aware of memory accesses done by BPF programs (memcpy being a tiny
subset of those). Not sure if the above data race qualifies as
something we might want to still detect, i.e. a kernel dev testing
their kernel might be interested in such a report.
Regardless, in this case we should teach syzkaller to ignore KCSAN
data races that originate from bpf user operations whatever the
origin.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
2023-08-29 18:53 ` Marco Elver
@ 2023-08-29 20:04 ` Daniel Borkmann
2023-08-29 20:36 ` Marco Elver
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Borkmann @ 2023-08-29 20:04 UTC (permalink / raw)
To: Marco Elver, yonghong.song
Cc: syzbot, andrii, ast, bpf, haoluo, john.fastabend, jolsa, kpsingh,
linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs,
Paul E. McKenney
On 8/29/23 8:53 PM, Marco Elver wrote:
> On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote:
>> On 8/29/23 5:39 AM, syzbot wrote:
>>> Hello,
>>>
>>> syzbot found the following issue on:
>>>
>>> HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k..
>>> git tree: upstream
>>> console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000
>>> kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f
>>> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
>>>
>>> Unfortunately, I don't have any reproducer for this issue yet.
>>>
>>> Downloadable assets:
>>> disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz
>>> vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz
>>> kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz
>>>
>>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
>>> Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com
>>>
>>> ==================================================================
>>> BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update
>>>
>>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
>>> bpf_long_memcpy include/linux/bpf.h:428 [inline]
>>> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
>>> copy_map_value_long include/linux/bpf.h:464 [inline]
>>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
>>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
>>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
>>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
>>> __sys_bpf+0x28a/0x780
>>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
>>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
>>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
>>> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
>>> entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>
>>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
>>> bpf_long_memcpy include/linux/bpf.h:428 [inline]
>>> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
>>> copy_map_value_long include/linux/bpf.h:464 [inline]
>>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
>>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
>>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
>>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
>>> __sys_bpf+0x28a/0x780
>>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
>>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
>>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
>>> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
>>> entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>
>>> value changed: 0x0000000000000000 -> 0xfffffff000002788
>>>
>>> Reported by Kernel Concurrency Sanitizer on:
>>> CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0
>>> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
>>> ==================================================================
>>
>> This case is with two tasks doing bpf_map batch update together for the
>> same map and key.
>> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
>> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
>>
>> So concurrency is introduced by user applications.
>> In my opinion, this probably not an issue from kernel perspective.
>
> Perhaps not, but I recall there being a discussion about making KCSAN
> aware of memory accesses done by BPF programs (memcpy being a tiny
> subset of those). Not sure if the above data race qualifies as
> something we might want to still detect, i.e. a kernel dev testing
> their kernel might be interested in such a report.
>
> Regardless, in this case we should teach syzkaller to ignore KCSAN
> data races that originate from bpf user operations whatever the
> origin.
I presume KCSAN could be silenced here via READ_ONCE/WRITE_ONCE conversion?
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index f58895830ada..32c4a37045f2 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -424,8 +424,11 @@ static inline void bpf_long_memcpy(void *dst, const void *src, u32 size)
long *ldst = dst;
size /= sizeof(long);
- while (size--)
- *ldst++ = *lsrc++;
+ while (size--) {
+ WRITE_ONCE(*ldst, READ_ONCE(*lsrc));
+ ldst++;
+ lsrc++;
+ }
}
/* copy everything but bpf_spin_lock, bpf_timer, and kptrs. There could be one of each. */
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
2023-08-29 20:04 ` Daniel Borkmann
@ 2023-08-29 20:36 ` Marco Elver
2023-08-29 20:54 ` Daniel Borkmann
0 siblings, 1 reply; 6+ messages in thread
From: Marco Elver @ 2023-08-29 20:36 UTC (permalink / raw)
To: Daniel Borkmann
Cc: yonghong.song, syzbot, andrii, ast, bpf, haoluo, john.fastabend,
jolsa, kpsingh, linux-kernel, martin.lau, sdf, song,
syzkaller-bugs, yhs, Paul E. McKenney
On Tue, 29 Aug 2023 at 22:04, Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 8/29/23 8:53 PM, Marco Elver wrote:
> > On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote:
> >> On 8/29/23 5:39 AM, syzbot wrote:
> >>> Hello,
> >>>
> >>> syzbot found the following issue on:
> >>>
> >>> HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k..
> >>> git tree: upstream
> >>> console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000
> >>> kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25
> >>> dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f
> >>> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> >>>
> >>> Unfortunately, I don't have any reproducer for this issue yet.
> >>>
> >>> Downloadable assets:
> >>> disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz
> >>> vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz
> >>> kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz
> >>>
> >>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> >>> Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com
> >>>
> >>> ==================================================================
> >>> BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update
> >>>
> >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> >>> bpf_long_memcpy include/linux/bpf.h:428 [inline]
> >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> >>> copy_map_value_long include/linux/bpf.h:464 [inline]
> >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> >>> __sys_bpf+0x28a/0x780
> >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >>>
> >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
> >>> bpf_long_memcpy include/linux/bpf.h:428 [inline]
> >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline]
> >>> copy_map_value_long include/linux/bpf.h:464 [inline]
> >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380
> >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175
> >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749
> >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648
> >>> __sys_bpf+0x28a/0x780
> >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline]
> >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline]
> >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239
> >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
> >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >>>
> >>> value changed: 0x0000000000000000 -> 0xfffffff000002788
> >>>
> >>> Reported by Kernel Concurrency Sanitizer on:
> >>> CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0
> >>> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
> >>> ==================================================================
> >>
> >> This case is with two tasks doing bpf_map batch update together for the
> >> same map and key.
> >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1:
> >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0:
> >>
> >> So concurrency is introduced by user applications.
> >> In my opinion, this probably not an issue from kernel perspective.
> >
> > Perhaps not, but I recall there being a discussion about making KCSAN
> > aware of memory accesses done by BPF programs (memcpy being a tiny
> > subset of those). Not sure if the above data race qualifies as
> > something we might want to still detect, i.e. a kernel dev testing
> > their kernel might be interested in such a report.
> >
> > Regardless, in this case we should teach syzkaller to ignore KCSAN
> > data races that originate from bpf user operations whatever the
> > origin.
>
> I presume KCSAN could be silenced here via READ_ONCE/WRITE_ONCE conversion?
>
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index f58895830ada..32c4a37045f2 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -424,8 +424,11 @@ static inline void bpf_long_memcpy(void *dst, const void *src, u32 size)
> long *ldst = dst;
>
> size /= sizeof(long);
> - while (size--)
> - *ldst++ = *lsrc++;
> + while (size--) {
> + WRITE_ONCE(*ldst, READ_ONCE(*lsrc));
> + ldst++;
> + lsrc++;
> + }
In this case, maybe data_race(*ldst++ = *lsrc++) would be more
appropriate and efficient. Unlikely that READ_ONCE()/WRITE_ONCE() here
helps make this any safer, i.e. the memcpy is still not atomic and if
it's a bug on the user's side, it'll corrupt data either way.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2)
2023-08-29 20:36 ` Marco Elver
@ 2023-08-29 20:54 ` Daniel Borkmann
0 siblings, 0 replies; 6+ messages in thread
From: Daniel Borkmann @ 2023-08-29 20:54 UTC (permalink / raw)
To: Marco Elver
Cc: yonghong.song, syzbot, andrii, ast, bpf, haoluo, john.fastabend,
jolsa, kpsingh, linux-kernel, martin.lau, sdf, song,
syzkaller-bugs, yhs, Paul E. McKenney
On 8/29/23 10:36 PM, Marco Elver wrote:
[...]
> In this case, maybe data_race(*ldst++ = *lsrc++) would be more
> appropriate and efficient. Unlikely that READ_ONCE()/WRITE_ONCE() here
> helps make this any safer, i.e. the memcpy is still not atomic and if
> it's a bug on the user's side, it'll corrupt data either way.
I wasn't aware of data_race(), thanks for the suggestion! Just flushed out
a small patch.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-08-29 20:55 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-29 9:39 [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) syzbot
2023-08-29 18:29 ` Yonghong Song
2023-08-29 18:53 ` Marco Elver
2023-08-29 20:04 ` Daniel Borkmann
2023-08-29 20:36 ` Marco Elver
2023-08-29 20:54 ` Daniel Borkmann
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.