* kernel: signal: NULL ptr deref when killing process
@ 2014-08-20 11:18 Sasha Levin
2014-08-20 14:12 ` Oleg Nesterov
0 siblings, 1 reply; 10+ messages in thread
From: Sasha Levin @ 2014-08-20 11:18 UTC (permalink / raw)
To: Andrew Morton, Oleg Nesterov, richard; +Cc: Dave Jones, LKML
Hi all,
While fuzzing with trinity inside a KVM tools guest running the latest -next
kernel, I've stumbled on the following spew:
[ 512.602559] BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
[ 512.602566] IP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
[ 512.602576] PGD 4c10d0067 PUD 4c1738067 PMD 0
[ 512.602584] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 512.602612] Dumping ftrace buffer:
[ 512.602697] (ftrace buffer empty)
[ 512.602704] Modules linked in:
[ 512.602708] CPU: 18 PID: 8516 Comm: trinity-watchdo Tainted: G B 3.16.0-next-20140815-sasha-00034-g615561b #1071
[ 512.602711] task: ffff8804c1e88000 ti: ffff8804c1290000 task.ti: ffff8804c1290000
[ 512.602718] RIP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
[ 512.602720] RSP: 0018:ffff8804c1293e00 EFLAGS: 00010246
[ 512.602722] RAX: 0000000000000000 RBX: ffff8804c1293ec0 RCX: 0000000000000000
[ 512.602723] RDX: ffff880546803000 RSI: ffff8804c1293ec0 RDI: 0000000000000000
[ 512.602726] RBP: ffff8804c1293e28 R08: 0000000000000000 R09: 0000000000000000
[ 512.602728] R10: 0000000000000000 R11: 0000000000000246 R12: ffff880546803000
[ 512.602729] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8804c3fe4200
[ 512.602732] FS: 00007fb2ed5e8700(0000) GS:ffff88071c400000(0000) knlGS:0000000000000000
[ 512.602735] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 512.602737] CR2: 000000000000001c CR3: 00000004c16ff000 CR4: 00000000000006a0
[ 512.602750] DR0: 00000000006f0000 DR1: 0000000000000000 DR2: 0000000000000000
[ 512.602753] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
[ 512.602754] Stack:
[ 512.602761] 00000000000026ce 0000000000000000 ffff8804c1293ec0 ffff880546803000
[ 512.602769] 0000000000000002 ffff8804c1293e60 ffffffff8c17ccb5 ffffffff8c17cc55
[ 512.602777] 00000000000026ce ffff880565319180 0000000000000000 ffff8804c1293ec0
[ 512.602779] Call Trace:
[ 512.602786] group_send_sig_info (kernel/signal.c:1300)
[ 512.602791] ? group_send_sig_info (kernel/signal.c:1296)
[ 512.602795] kill_pid_info (kernel/signal.c:1338)
[ 512.602800] ? kill_pid_info (kernel/signal.c:1330)
[ 512.602805] SYSC_kill (kernel/signal.c:1423 kernel/signal.c:2896)
[ 512.602809] ? SYSC_kill (include/linux/rcupdate.h:814 kernel/signal.c:1422 kernel/signal.c:2896)
[ 512.602814] ? _raw_spin_unlock (./arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 512.602820] ? trace_hardirqs_on (kernel/locking/lockdep.c:2609)
[ 512.602824] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 512.602828] SyS_kill (kernel/signal.c:2886)
[ 512.602833] tracesys (arch/x86/kernel/entry_64.S:541)
[ 512.602921] Code: 06 00 4d 8b b4 24 38 0a 00 00 65 48 8b 04 25 c0 da 00 00 4c 8b b8 40 0a 00 00 e8 ab e3 06 00 85 c0 0f 85 8b 00 00 00 41 8b 47 24 <41> 8b 56 1c 39 d0 74 60 41 8b 4e 14 39 c8 74 58 41 8b 47 14 39
All code
========
0: 06 (bad)
1: 00 4d 8b add %cl,-0x75(%rbp)
4: b4 24 mov $0x24,%ah
6: 38 0a cmp %cl,(%rdx)
8: 00 00 add %al,(%rax)
a: 65 48 8b 04 25 c0 da mov %gs:0xdac0,%rax
11: 00 00
13: 4c 8b b8 40 0a 00 00 mov 0xa40(%rax),%r15
1a: e8 ab e3 06 00 callq 0x6e3ca
1f: 85 c0 test %eax,%eax
21: 0f 85 8b 00 00 00 jne 0xb2
27: 41 8b 47 24 mov 0x24(%r15),%eax
2b:* 41 8b 56 1c mov 0x1c(%r14),%edx <-- trapping instruction
2f: 39 d0 cmp %edx,%eax
31: 74 60 je 0x93
33: 41 8b 4e 14 mov 0x14(%r14),%ecx
37: 39 c8 cmp %ecx,%eax
39: 74 58 je 0x93
3b: 41 8b 47 14 mov 0x14(%r15),%eax
3f: 39 00 cmp %eax,(%rax)
Code starting with the faulting instruction
===========================================
0: 41 8b 56 1c mov 0x1c(%r14),%edx
4: 39 d0 cmp %edx,%eax
6: 74 60 je 0x68
8: 41 8b 4e 14 mov 0x14(%r14),%ecx
c: 39 c8 cmp %ecx,%eax
e: 74 58 je 0x68
10: 41 8b 47 14 mov 0x14(%r15),%eax
14: 39 00 cmp %eax,(%rax)
[ 512.602927] RIP check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
[ 512.602929] RSP <ffff8804c1293e00>
[ 512.602931] CR2: 000000000000001c
Thanks,
Sasha
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-20 11:18 kernel: signal: NULL ptr deref when killing process Sasha Levin
@ 2014-08-20 14:12 ` Oleg Nesterov
2014-08-20 15:06 ` Oleg Nesterov
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-20 14:12 UTC (permalink / raw)
To: Sasha Levin, David Howells; +Cc: Andrew Morton, richard, Dave Jones, LKML
On 08/20, Sasha Levin wrote:
>
> Hi all,
>
> While fuzzing with trinity inside a KVM tools guest running the latest -next
> kernel, I've stumbled on the following spew:
Thanks...
looks like, kill_ok_by_cred()->__task_cred(t) returns NULL at first glance.
perhaps you can show the result of "make kernel/signal.s" to be sure? Or at
least the full "objdump -d kernel/signal.o".
This looks strange. ->real_cred can be NULL only after __put_task_struct(),
so this task_struct is already freed or soon-to-be-freed. But everything looks
correct in this path, this task was found by pid_task() under rcu_read_lock().
> [ 512.602559] BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
> [ 512.602566] IP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> [ 512.602576] PGD 4c10d0067 PUD 4c1738067 PMD 0
> [ 512.602584] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> [ 512.602612] Dumping ftrace buffer:
> [ 512.602697] (ftrace buffer empty)
> [ 512.602704] Modules linked in:
> [ 512.602708] CPU: 18 PID: 8516 Comm: trinity-watchdo Tainted: G B 3.16.0-next-20140815-sasha-00034-g615561b #1071
> [ 512.602711] task: ffff8804c1e88000 ti: ffff8804c1290000 task.ti: ffff8804c1290000
> [ 512.602718] RIP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> [ 512.602720] RSP: 0018:ffff8804c1293e00 EFLAGS: 00010246
> [ 512.602722] RAX: 0000000000000000 RBX: ffff8804c1293ec0 RCX: 0000000000000000
> [ 512.602723] RDX: ffff880546803000 RSI: ffff8804c1293ec0 RDI: 0000000000000000
> [ 512.602726] RBP: ffff8804c1293e28 R08: 0000000000000000 R09: 0000000000000000
> [ 512.602728] R10: 0000000000000000 R11: 0000000000000246 R12: ffff880546803000
> [ 512.602729] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8804c3fe4200
> [ 512.602732] FS: 00007fb2ed5e8700(0000) GS:ffff88071c400000(0000) knlGS:0000000000000000
> [ 512.602735] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 512.602737] CR2: 000000000000001c CR3: 00000004c16ff000 CR4: 00000000000006a0
> [ 512.602750] DR0: 00000000006f0000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 512.602753] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
> [ 512.602754] Stack:
> [ 512.602761] 00000000000026ce 0000000000000000 ffff8804c1293ec0 ffff880546803000
> [ 512.602769] 0000000000000002 ffff8804c1293e60 ffffffff8c17ccb5 ffffffff8c17cc55
> [ 512.602777] 00000000000026ce ffff880565319180 0000000000000000 ffff8804c1293ec0
> [ 512.602779] Call Trace:
> [ 512.602786] group_send_sig_info (kernel/signal.c:1300)
> [ 512.602791] ? group_send_sig_info (kernel/signal.c:1296)
> [ 512.602795] kill_pid_info (kernel/signal.c:1338)
> [ 512.602800] ? kill_pid_info (kernel/signal.c:1330)
> [ 512.602805] SYSC_kill (kernel/signal.c:1423 kernel/signal.c:2896)
> [ 512.602809] ? SYSC_kill (include/linux/rcupdate.h:814 kernel/signal.c:1422 kernel/signal.c:2896)
> [ 512.602814] ? _raw_spin_unlock (./arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
> [ 512.602820] ? trace_hardirqs_on (kernel/locking/lockdep.c:2609)
> [ 512.602824] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
> [ 512.602828] SyS_kill (kernel/signal.c:2886)
> [ 512.602833] tracesys (arch/x86/kernel/entry_64.S:541)
> [ 512.602921] Code: 06 00 4d 8b b4 24 38 0a 00 00 65 48 8b 04 25 c0 da 00 00 4c 8b b8 40 0a 00 00 e8 ab e3 06 00 85 c0 0f 85 8b 00 00 00 41 8b 47 24 <41> 8b 56 1c 39 d0 74 60 41 8b 4e 14 39 c8 74 58 41 8b 47 14 39
> All code
> ========
> 0: 06 (bad)
> 1: 00 4d 8b add %cl,-0x75(%rbp)
> 4: b4 24 mov $0x24,%ah
> 6: 38 0a cmp %cl,(%rdx)
> 8: 00 00 add %al,(%rax)
> a: 65 48 8b 04 25 c0 da mov %gs:0xdac0,%rax
> 11: 00 00
> 13: 4c 8b b8 40 0a 00 00 mov 0xa40(%rax),%r15
> 1a: e8 ab e3 06 00 callq 0x6e3ca
> 1f: 85 c0 test %eax,%eax
> 21: 0f 85 8b 00 00 00 jne 0xb2
> 27: 41 8b 47 24 mov 0x24(%r15),%eax
> 2b:* 41 8b 56 1c mov 0x1c(%r14),%edx <-- trapping instruction
> 2f: 39 d0 cmp %edx,%eax
> 31: 74 60 je 0x93
> 33: 41 8b 4e 14 mov 0x14(%r14),%ecx
> 37: 39 c8 cmp %ecx,%eax
> 39: 74 58 je 0x93
> 3b: 41 8b 47 14 mov 0x14(%r15),%eax
> 3f: 39 00 cmp %eax,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 41 8b 56 1c mov 0x1c(%r14),%edx
> 4: 39 d0 cmp %edx,%eax
> 6: 74 60 je 0x68
> 8: 41 8b 4e 14 mov 0x14(%r14),%ecx
> c: 39 c8 cmp %ecx,%eax
> e: 74 58 je 0x68
> 10: 41 8b 47 14 mov 0x14(%r15),%eax
> 14: 39 00 cmp %eax,(%rax)
> [ 512.602927] RIP check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> [ 512.602929] RSP <ffff8804c1293e00>
> [ 512.602931] CR2: 000000000000001c
>
>
> Thanks,
> Sasha
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-20 14:12 ` Oleg Nesterov
@ 2014-08-20 15:06 ` Oleg Nesterov
2014-08-20 15:30 ` Oleg Nesterov
2014-08-21 15:20 ` Sasha Levin
2014-08-21 16:22 ` David Howells
2 siblings, 1 reply; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-20 15:06 UTC (permalink / raw)
To: Sasha Levin, David Howells; +Cc: Andrew Morton, richard, Dave Jones, LKML
On 08/20, Oleg Nesterov wrote:
>
> On 08/20, Sasha Levin wrote:
> >
> > Hi all,
> >
> > While fuzzing with trinity inside a KVM tools guest running the latest -next
> > kernel, I've stumbled on the following spew:
>
> Thanks...
>
> looks like, kill_ok_by_cred()->__task_cred(t) returns NULL at first glance.
> perhaps you can show the result of "make kernel/signal.s" to be sure? Or at
> least the full "objdump -d kernel/signal.o".
>
> This looks strange. ->real_cred can be NULL only after __put_task_struct(),
> so this task_struct is already freed or soon-to-be-freed. But everything looks
> correct in this path, this task was found by pid_task() under rcu_read_lock().
Hmm. I'll recheck and report tomorrow, but it seems that I found a bug:
bad_fork_free_pid()->free_pid() looks wrong. This _can_ explain this trace
afaics, but only in unlikely case...
David, any other reason why ->real_cred can be NULL ? (assuming that I interpret
this asm correctly).
> > [ 512.602559] BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
> > [ 512.602566] IP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> > [ 512.602576] PGD 4c10d0067 PUD 4c1738067 PMD 0
> > [ 512.602584] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> > [ 512.602612] Dumping ftrace buffer:
> > [ 512.602697] (ftrace buffer empty)
> > [ 512.602704] Modules linked in:
> > [ 512.602708] CPU: 18 PID: 8516 Comm: trinity-watchdo Tainted: G B 3.16.0-next-20140815-sasha-00034-g615561b #1071
> > [ 512.602711] task: ffff8804c1e88000 ti: ffff8804c1290000 task.ti: ffff8804c1290000
> > [ 512.602718] RIP: check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> > [ 512.602720] RSP: 0018:ffff8804c1293e00 EFLAGS: 00010246
> > [ 512.602722] RAX: 0000000000000000 RBX: ffff8804c1293ec0 RCX: 0000000000000000
> > [ 512.602723] RDX: ffff880546803000 RSI: ffff8804c1293ec0 RDI: 0000000000000000
> > [ 512.602726] RBP: ffff8804c1293e28 R08: 0000000000000000 R09: 0000000000000000
> > [ 512.602728] R10: 0000000000000000 R11: 0000000000000246 R12: ffff880546803000
> > [ 512.602729] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8804c3fe4200
> > [ 512.602732] FS: 00007fb2ed5e8700(0000) GS:ffff88071c400000(0000) knlGS:0000000000000000
> > [ 512.602735] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [ 512.602737] CR2: 000000000000001c CR3: 00000004c16ff000 CR4: 00000000000006a0
> > [ 512.602750] DR0: 00000000006f0000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 512.602753] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
> > [ 512.602754] Stack:
> > [ 512.602761] 00000000000026ce 0000000000000000 ffff8804c1293ec0 ffff880546803000
> > [ 512.602769] 0000000000000002 ffff8804c1293e60 ffffffff8c17ccb5 ffffffff8c17cc55
> > [ 512.602777] 00000000000026ce ffff880565319180 0000000000000000 ffff8804c1293ec0
> > [ 512.602779] Call Trace:
> > [ 512.602786] group_send_sig_info (kernel/signal.c:1300)
> > [ 512.602791] ? group_send_sig_info (kernel/signal.c:1296)
> > [ 512.602795] kill_pid_info (kernel/signal.c:1338)
> > [ 512.602800] ? kill_pid_info (kernel/signal.c:1330)
> > [ 512.602805] SYSC_kill (kernel/signal.c:1423 kernel/signal.c:2896)
> > [ 512.602809] ? SYSC_kill (include/linux/rcupdate.h:814 kernel/signal.c:1422 kernel/signal.c:2896)
> > [ 512.602814] ? _raw_spin_unlock (./arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
> > [ 512.602820] ? trace_hardirqs_on (kernel/locking/lockdep.c:2609)
> > [ 512.602824] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
> > [ 512.602828] SyS_kill (kernel/signal.c:2886)
> > [ 512.602833] tracesys (arch/x86/kernel/entry_64.S:541)
> > [ 512.602921] Code: 06 00 4d 8b b4 24 38 0a 00 00 65 48 8b 04 25 c0 da 00 00 4c 8b b8 40 0a 00 00 e8 ab e3 06 00 85 c0 0f 85 8b 00 00 00 41 8b 47 24 <41> 8b 56 1c 39 d0 74 60 41 8b 4e 14 39 c8 74 58 41 8b 47 14 39
> > All code
> > ========
> > 0: 06 (bad)
> > 1: 00 4d 8b add %cl,-0x75(%rbp)
> > 4: b4 24 mov $0x24,%ah
> > 6: 38 0a cmp %cl,(%rdx)
> > 8: 00 00 add %al,(%rax)
> > a: 65 48 8b 04 25 c0 da mov %gs:0xdac0,%rax
> > 11: 00 00
> > 13: 4c 8b b8 40 0a 00 00 mov 0xa40(%rax),%r15
> > 1a: e8 ab e3 06 00 callq 0x6e3ca
> > 1f: 85 c0 test %eax,%eax
> > 21: 0f 85 8b 00 00 00 jne 0xb2
> > 27: 41 8b 47 24 mov 0x24(%r15),%eax
> > 2b:* 41 8b 56 1c mov 0x1c(%r14),%edx <-- trapping instruction
> > 2f: 39 d0 cmp %edx,%eax
> > 31: 74 60 je 0x93
> > 33: 41 8b 4e 14 mov 0x14(%r14),%ecx
> > 37: 39 c8 cmp %ecx,%eax
> > 39: 74 58 je 0x93
> > 3b: 41 8b 47 14 mov 0x14(%r15),%eax
> > 3f: 39 00 cmp %eax,(%rax)
> >
> > Code starting with the faulting instruction
> > ===========================================
> > 0: 41 8b 56 1c mov 0x1c(%r14),%edx
> > 4: 39 d0 cmp %edx,%eax
> > 6: 74 60 je 0x68
> > 8: 41 8b 4e 14 mov 0x14(%r14),%ecx
> > c: 39 c8 cmp %ecx,%eax
> > e: 74 58 je 0x68
> > 10: 41 8b 47 14 mov 0x14(%r15),%eax
> > 14: 39 00 cmp %eax,(%rax)
> > [ 512.602927] RIP check_kill_permission (kernel/signal.c:746 kernel/signal.c:781)
> > [ 512.602929] RSP <ffff8804c1293e00>
> > [ 512.602931] CR2: 000000000000001c
> >
> >
> > Thanks,
> > Sasha
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-20 15:06 ` Oleg Nesterov
@ 2014-08-20 15:30 ` Oleg Nesterov
0 siblings, 0 replies; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-20 15:30 UTC (permalink / raw)
To: Sasha Levin, David Howells; +Cc: Andrew Morton, richard, Dave Jones, LKML
On 08/20, Oleg Nesterov wrote:
>
> Hmm. I'll recheck and report tomorrow, but it seems that I found a bug:
> bad_fork_free_pid()->free_pid() looks wrong.
No, sorry for noise, I was wrong.
> David, any other reason why ->real_cred can be NULL ? (assuming that I interpret
> this asm correctly).
Oleg.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-20 14:12 ` Oleg Nesterov
2014-08-20 15:06 ` Oleg Nesterov
@ 2014-08-21 15:20 ` Sasha Levin
2014-08-21 17:11 ` Oleg Nesterov
2014-08-21 16:22 ` David Howells
2 siblings, 1 reply; 10+ messages in thread
From: Sasha Levin @ 2014-08-21 15:20 UTC (permalink / raw)
To: Oleg Nesterov, David Howells; +Cc: Andrew Morton, richard, Dave Jones, LKML
[-- Attachment #1: Type: text/plain, Size: 475 bytes --]
On 08/20/2014 10:12 AM, Oleg Nesterov wrote:
> On 08/20, Sasha Levin wrote:
>> >
>> > Hi all,
>> >
>> > While fuzzing with trinity inside a KVM tools guest running the latest -next
>> > kernel, I've stumbled on the following spew:
> Thanks...
>
> looks like, kill_ok_by_cred()->__task_cred(t) returns NULL at first glance.
> perhaps you can show the result of "make kernel/signal.s" to be sure? Or at
> least the full "objdump -d kernel/signal.o".
Attached.
Thanks,
Sasha
[-- Attachment #2: signal.s.xz --]
[-- Type: application/x-xz, Size: 168828 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-20 14:12 ` Oleg Nesterov
2014-08-20 15:06 ` Oleg Nesterov
2014-08-21 15:20 ` Sasha Levin
@ 2014-08-21 16:22 ` David Howells
2014-08-21 17:17 ` Oleg Nesterov
2 siblings, 1 reply; 10+ messages in thread
From: David Howells @ 2014-08-21 16:22 UTC (permalink / raw)
To: Oleg Nesterov
Cc: dhowells, Sasha Levin, Andrew Morton, richard, Dave Jones, LKML
Oleg Nesterov <oleg@redhat.com> wrote:
> David, any other reason why ->real_cred can be NULL ? (assuming that I
> interpret this asm correctly).
It should only be possible to see ->real_cred as being NULL after exit_creds()
has been called from __put_task_struct() for a task that has finished
construction.
It shouldn't be possible to introduce a NULL pointer through commit_creds() or
override_creds() since both of those should crash immediately if given one,
but it's possible revert_creds() could be so used.
Is there a race between kill() and exit() brought on by the kill path only
using the RCU read lock? This doesn't prevent ->real_cred from being
modified, but it looks like this should, in combination with
delayed_put_task_struct(), prevent it from being cleared.
David
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-21 15:20 ` Sasha Levin
@ 2014-08-21 17:11 ` Oleg Nesterov
0 siblings, 0 replies; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-21 17:11 UTC (permalink / raw)
To: Sasha Levin; +Cc: David Howells, Andrew Morton, richard, Dave Jones, LKML
On 08/21, Sasha Levin wrote:
>
> On 08/20/2014 10:12 AM, Oleg Nesterov wrote:
> > On 08/20, Sasha Levin wrote:
> >> >
> >> > Hi all,
> >> >
> >> > While fuzzing with trinity inside a KVM tools guest running the latest -next
> >> > kernel, I've stumbled on the following spew:
> > Thanks...
> >
> > looks like, kill_ok_by_cred()->__task_cred(t) returns NULL at first glance.
> > perhaps you can show the result of "make kernel/signal.s" to be sure? Or at
> > least the full "objdump -d kernel/signal.o".
>
> Attached.
Thanks.
Yes, t->real_cred == NULL (r14).
Interestingly, t->signal is NULL too (rcx). And ->signal must be never NULL.
So it looks like this task_struct was reallocated/reused. Or corrupted.
t == 0xffff880546803000 (r12), this doesn't look wrong.
Oleg.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-21 16:22 ` David Howells
@ 2014-08-21 17:17 ` Oleg Nesterov
2014-08-27 3:38 ` Sasha Levin
0 siblings, 1 reply; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-21 17:17 UTC (permalink / raw)
To: David Howells; +Cc: Sasha Levin, Andrew Morton, richard, Dave Jones, LKML
On 08/21, David Howells wrote:
>
> Oleg Nesterov <oleg@redhat.com> wrote:
>
> > David, any other reason why ->real_cred can be NULL ? (assuming that I
> > interpret this asm correctly).
>
> It should only be possible to see ->real_cred as being NULL after exit_creds()
> has been called from __put_task_struct() for a task that has finished
> construction.
Yes, thanks, this was my understanding...
> Is there a race between kill() and exit() brought on by the kill path only
> using the RCU read lock? This doesn't prevent ->real_cred from being
> modified, but it looks like this should, in combination with
> delayed_put_task_struct(), prevent it from being cleared.
Yes, rcu should protect us from both delayed_put_pid() and delayed_put_task().
Everything looks correct... And there are a lot of other similar users of
find_vpid/find_task_by_vpid/pid_task/etc under rcu, I can't recall any bug
in this area.
I am puzzled. Note also that ->signal == NULL. Will try to think more,
but so far I have no any idea.
Oleg.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-21 17:17 ` Oleg Nesterov
@ 2014-08-27 3:38 ` Sasha Levin
2014-08-27 14:16 ` Oleg Nesterov
0 siblings, 1 reply; 10+ messages in thread
From: Sasha Levin @ 2014-08-27 3:38 UTC (permalink / raw)
To: Oleg Nesterov, David Howells; +Cc: Andrew Morton, richard, Dave Jones, LKML
On 08/21/2014 01:17 PM, Oleg Nesterov wrote:
>> Is there a race between kill() and exit() brought on by the kill path only
>> > using the RCU read lock? This doesn't prevent ->real_cred from being
>> > modified, but it looks like this should, in combination with
>> > delayed_put_task_struct(), prevent it from being cleared.
> Yes, rcu should protect us from both delayed_put_pid() and delayed_put_task().
> Everything looks correct... And there are a lot of other similar users of
> find_vpid/find_task_by_vpid/pid_task/etc under rcu, I can't recall any bug
> in this area.
>
> I am puzzled. Note also that ->signal == NULL. Will try to think more,
> but so far I have no any idea.
I've hit something similar earlier today, and it might be related:
[ 973.452840] BUG: unable to handle kernel NULL pointer dereference at 00000000000002b0
[ 973.455347] IP: flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
[ 973.457526] PGD 4dfdc7067 PUD 5f77d9067 PMD 0
[ 973.459216] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 973.460086] Dumping ftrace buffer:
[ 973.460086] (ftrace buffer empty)
[ 973.460086] Modules linked in:
[ 973.460086] CPU: 4 PID: 13145 Comm: trinity-c767 Not tainted 3.17.0-rc2-next-20140826-sasha-00031-gc48c9ac-dirty #1079
[ 973.460086] task: ffff880604800000 ti: ffff880586648000 task.ti: ffff880586648000
[ 973.460086] RIP: flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
[ 973.460086] RSP: 0018:ffff88058664bec8 EFLAGS: 00010046
[ 973.460086] RAX: 0000000000000000 RBX: fffffffffffff730 RCX: 0000000000010000
[ 973.460086] RDX: 0000000000000000 RSI: 00000000000002a0 RDI: ffff88058664bed8
[ 973.460086] RBP: ffff88058664bf10 R08: 0000000000000001 R09: 0000000000000001
[ 973.460086] R10: 000000000002d201 R11: 0000000000000254 R12: 0000000000000000
[ 973.460086] R13: ffff88058664bf40 R14: ffff880604800000 R15: 0000000000000010
[ 973.460086] FS: 00007fe3a3045700(0000) GS:ffff880277c00000(0000) knlGS:0000000000000000
[ 973.460086] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 973.460086] CR2: 00000000000002b0 CR3: 00000004e23d5000 CR4: 00000000000006a0
[ 973.460086] Stack:
[ 973.460086] ffffffffac183690 01017fffb3247180 0000000000010000 0000000000000000
[ 973.460086] 00007fffb3247180 00007fffb3247220 0000000000000011 0000000000000000
[ 973.460086] 0000000000000000 ffff88058664bf78 ffffffffac183ef5 0000000000000000
[ 973.460086] Call Trace:
[ 973.460086] ? do_sigaction (kernel/signal.c:3124 (discriminator 17))
[ 973.460086] SyS_rt_sigaction (kernel/signal.c:3360 kernel/signal.c:3341)
[ 973.460086] tracesys (arch/x86/kernel/entry_64.S:542)
[ 973.460086] Code: b7 49 09 d5 4d 89 6e 10 48 83 c4 08 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 0f 31 c0 <48> 8b 56 10 48 85 ca 74 7b 55 48 f7 d1 48 89 e5 41 56 48 21 ca
All code
========
0: b7 49 mov $0x49,%bh
2: 09 d5 or %edx,%ebp
4: 4d 89 6e 10 mov %r13,0x10(%r14)
8: 48 83 c4 08 add $0x8,%rsp
c: 5b pop %rbx
d: 41 5c pop %r12
f: 41 5d pop %r13
11: 41 5e pop %r14
13: 41 5f pop %r15
15: 5d pop %rbp
16: c3 retq
17: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
1e: 00 00 00
21: 66 66 66 66 90 data32 data32 data32 xchg %ax,%ax
26: 48 8b 0f mov (%rdi),%rcx
29: 31 c0 xor %eax,%eax
2b:* 48 8b 56 10 mov 0x10(%rsi),%rdx <-- trapping instruction
2f: 48 85 ca test %rcx,%rdx
32: 74 7b je 0xaf
34: 55 push %rbp
35: 48 f7 d1 not %rcx
38: 48 89 e5 mov %rsp,%rbp
3b: 41 56 push %r14
3d: 48 21 ca and %rcx,%rdx
...
Code starting with the faulting instruction
===========================================
0: 48 8b 56 10 mov 0x10(%rsi),%rdx
4: 48 85 ca test %rcx,%rdx
7: 74 7b je 0x84
9: 55 push %rbp
a: 48 f7 d1 not %rcx
d: 48 89 e5 mov %rsp,%rbp
10: 41 56 push %r14
12: 48 21 ca and %rcx,%rdx
...
[ 973.460086] RIP flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
[ 973.460086] RSP <ffff88058664bec8>
[ 973.460086] CR2: 00000000000002b0
Thanks,
Sasha
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: kernel: signal: NULL ptr deref when killing process
2014-08-27 3:38 ` Sasha Levin
@ 2014-08-27 14:16 ` Oleg Nesterov
0 siblings, 0 replies; 10+ messages in thread
From: Oleg Nesterov @ 2014-08-27 14:16 UTC (permalink / raw)
To: Sasha Levin; +Cc: David Howells, Andrew Morton, richard, Dave Jones, LKML
On 08/26, Sasha Levin wrote:
>
> On 08/21/2014 01:17 PM, Oleg Nesterov wrote:
> >> Is there a race between kill() and exit() brought on by the kill path only
> >> > using the RCU read lock? This doesn't prevent ->real_cred from being
> >> > modified, but it looks like this should, in combination with
> >> > delayed_put_task_struct(), prevent it from being cleared.
> > Yes, rcu should protect us from both delayed_put_pid() and delayed_put_task().
> > Everything looks correct... And there are a lot of other similar users of
> > find_vpid/find_task_by_vpid/pid_task/etc under rcu, I can't recall any bug
> > in this area.
> >
> > I am puzzled. Note also that ->signal == NULL. Will try to think more,
> > but so far I have no any idea.
>
> I've hit something similar earlier today, and it might be related:
Thanks.
rsi == ¤t->signal->shared_pending == 0x2a0, so current->signal == 544.
Given that this task is current, we can rule out RCU/find_pind/etc problems,
this task_struct and its ->signal must be stable.
Looks like, something was broken recently. I do not remember the bug reports
like this. Say, an unbalanced put_task_struct()... Hmm.
> [ 973.452840] BUG: unable to handle kernel NULL pointer dereference at 00000000000002b0
> [ 973.455347] IP: flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
> [ 973.457526] PGD 4dfdc7067 PUD 5f77d9067 PMD 0
> [ 973.459216] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> [ 973.460086] Dumping ftrace buffer:
> [ 973.460086] (ftrace buffer empty)
> [ 973.460086] Modules linked in:
> [ 973.460086] CPU: 4 PID: 13145 Comm: trinity-c767 Not tainted 3.17.0-rc2-next-20140826-sasha-00031-gc48c9ac-dirty #1079
> [ 973.460086] task: ffff880604800000 ti: ffff880586648000 task.ti: ffff880586648000
> [ 973.460086] RIP: flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
> [ 973.460086] RSP: 0018:ffff88058664bec8 EFLAGS: 00010046
> [ 973.460086] RAX: 0000000000000000 RBX: fffffffffffff730 RCX: 0000000000010000
> [ 973.460086] RDX: 0000000000000000 RSI: 00000000000002a0 RDI: ffff88058664bed8
> [ 973.460086] RBP: ffff88058664bf10 R08: 0000000000000001 R09: 0000000000000001
> [ 973.460086] R10: 000000000002d201 R11: 0000000000000254 R12: 0000000000000000
> [ 973.460086] R13: ffff88058664bf40 R14: ffff880604800000 R15: 0000000000000010
> [ 973.460086] FS: 00007fe3a3045700(0000) GS:ffff880277c00000(0000) knlGS:0000000000000000
> [ 973.460086] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 973.460086] CR2: 00000000000002b0 CR3: 00000004e23d5000 CR4: 00000000000006a0
> [ 973.460086] Stack:
> [ 973.460086] ffffffffac183690 01017fffb3247180 0000000000010000 0000000000000000
> [ 973.460086] 00007fffb3247180 00007fffb3247220 0000000000000011 0000000000000000
> [ 973.460086] 0000000000000000 ffff88058664bf78 ffffffffac183ef5 0000000000000000
> [ 973.460086] Call Trace:
> [ 973.460086] ? do_sigaction (kernel/signal.c:3124 (discriminator 17))
> [ 973.460086] SyS_rt_sigaction (kernel/signal.c:3360 kernel/signal.c:3341)
> [ 973.460086] tracesys (arch/x86/kernel/entry_64.S:542)
> [ 973.460086] Code: b7 49 09 d5 4d 89 6e 10 48 83 c4 08 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 0f 31 c0 <48> 8b 56 10 48 85 ca 74 7b 55 48 f7 d1 48 89 e5 41 56 48 21 ca
> All code
> ========
> 0: b7 49 mov $0x49,%bh
> 2: 09 d5 or %edx,%ebp
> 4: 4d 89 6e 10 mov %r13,0x10(%r14)
> 8: 48 83 c4 08 add $0x8,%rsp
> c: 5b pop %rbx
> d: 41 5c pop %r12
> f: 41 5d pop %r13
> 11: 41 5e pop %r14
> 13: 41 5f pop %r15
> 15: 5d pop %rbp
> 16: c3 retq
> 17: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
> 1e: 00 00 00
> 21: 66 66 66 66 90 data32 data32 data32 xchg %ax,%ax
> 26: 48 8b 0f mov (%rdi),%rcx
> 29: 31 c0 xor %eax,%eax
> 2b:* 48 8b 56 10 mov 0x10(%rsi),%rdx <-- trapping instruction
> 2f: 48 85 ca test %rcx,%rdx
> 32: 74 7b je 0xaf
> 34: 55 push %rbp
> 35: 48 f7 d1 not %rcx
> 38: 48 89 e5 mov %rsp,%rbp
> 3b: 41 56 push %r14
> 3d: 48 21 ca and %rcx,%rdx
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 48 8b 56 10 mov 0x10(%rsi),%rdx
> 4: 48 85 ca test %rcx,%rdx
> 7: 74 7b je 0x84
> 9: 55 push %rbp
> a: 48 f7 d1 not %rcx
> d: 48 89 e5 mov %rsp,%rbp
> 10: 41 56 push %r14
> 12: 48 21 ca and %rcx,%rdx
> ...
> [ 973.460086] RIP flush_sigqueue_mask (include/linux/signal.h:118 kernel/signal.c:715)
> [ 973.460086] RSP <ffff88058664bec8>
> [ 973.460086] CR2: 00000000000002b0
>
>
> Thanks,
> Sasha
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-08-27 14:18 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-20 11:18 kernel: signal: NULL ptr deref when killing process Sasha Levin
2014-08-20 14:12 ` Oleg Nesterov
2014-08-20 15:06 ` Oleg Nesterov
2014-08-20 15:30 ` Oleg Nesterov
2014-08-21 15:20 ` Sasha Levin
2014-08-21 17:11 ` Oleg Nesterov
2014-08-21 16:22 ` David Howells
2014-08-21 17:17 ` Oleg Nesterov
2014-08-27 3:38 ` Sasha Levin
2014-08-27 14:16 ` Oleg Nesterov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.