possible use after free in 4.9.228-rt147

possible use after free in 4.9.228-rt147

[Buehler, Matthias]

Hello,

I found these bugs in 4.9.228-rt147: 

[  665.048394] BUG: unable to handle kernel NULL pointer dereference at 0000000000000001
[  665.048404] IP: [<ffffffff81197daf>] kmem_cache_alloc+0x6f/0x1d0
[  665.048406] PGD 0 
[  665.048406] 
[  665.048409] Oops: 0000 [#1] PREEMPT SMP
[  665.048435] Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_nat nfsd 
iptable_nat nf_nat_ipv4 nf_nat xt_hl nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter 
ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables rtc_pcf8563 
coretemp rtc_cmos lm75 max6650 i2c_i801 i2c_smbus uio_cif uvcvideo videobuf2_vmalloc 
videobuf2_memops videobuf2_v4l2 videobuf2_core videodev fuse ipv6 autofs4 whiteheat 
ti_usb_3410_5052 spcp8x5 safe_serial pl2303 oti6858 option usb_wwan mos7840 mos7720 
mct_u232 keyspan_pda keyspan ezusb iuu_phoenix io_ti io_edgeport ftdi_sio f81232 
empeg digi_acceleport cypress_m8 ch341 belkin_sa ark3116 aircable radeon ttm i915
[  665.048439] CPU: 0 PID: 4794 Comm: nmbd Not tainted 4.9.228-rt147cpx+ #172
[  665.048440] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop 
Reference Platform, BIOS 6.00 07/02/2015
[  665.048441] task: ffff88003a1063c0 task.stack: ffffc900038a8000
[  665.048445] RIP: 0010:[<ffffffff81197daf>]  [<ffffffff81197daf>] kmem_cache_alloc+0x6f/0x1d0
[  665.048447] RSP: 0018:ffffc900038abb40  EFLAGS: 00010202
[  665.048448] RAX: 0000000000000000 RBX: ffff880037810e00 RCX: 000000000026ebb0
[  665.048449] RDX: 000000000026eba8 RSI: 000000000026eba8 RDI: ffff880000090700
[  665.048450] RBP: ffffc900038abb70 R08: 0000000000041fe0 R09: ffffffff816a21f2
[  665.048450] R10: ffffc900038abd18 R11: 0000000000000000 R12: 0000000002080020
[  665.048451] R13: 0000000000000001 R14: ffff880000090700 R15: ffff880000090700
[  665.048453] FS:  00007fd37700f700(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  665.048500] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  665.048514] CR2: 0000000000000001 CR3: 000000003dae6000 CR4: 00000000003606f0
[  665.048550] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  665.048551] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  665.048552] Stack:
[  665.048555]  ffffc900038abb68 ffff880037810e00 0000000002080020 ffffffff81d11240
[  665.048557]  ffff880039cea300 0000000000004b00 ffffc900038abb90 ffffffff816a21f2
[  665.048559]  ffff880037810e00 ffff880037810f00 ffffc900038abc10 ffffffff817031c5
[  665.048560] Call Trace:
[  665.048571]  [<ffffffff816a21f2>] skb_clone+0x52/0xa0
[  665.048578]  [<ffffffff817031c5>] ip_mc_output+0x155/0x2a0
[  665.048581]  [<ffffffff81702a3c>] ? __ip_local_out+0xdc/0x140
[  665.048583]  [<ffffffff81700a10>] ? ip_forward_options+0x1b0/0x1b0
[  665.048586]  [<ffffffff81702ad5>] ip_local_out+0x35/0x40
[  665.048589]  [<ffffffff81703d39>] ip_send_skb+0x19/0x40
[  665.048592]  [<ffffffff8172bb54>] udp_send_skb+0x114/0x2b0
[  665.048594]  [<ffffffff8172de17>] udp_sendmsg+0x4f7/0x9b0
[  665.048597]  [<ffffffff81701a20>] ? ip_reply_glue_bits+0x50/0x50
[  665.048601]  [<ffffffff8173883f>] inet_sendmsg+0x7f/0xb0
[  665.048604]  [<ffffffff81699288>] sock_sendmsg+0x38/0x50
[  665.048606]  [<ffffffff8169a793>] SyS_sendto+0xf3/0x140
[  665.048612]  [<ffffffff810d3449>] ? ktime_get_ts64+0x49/0xf0
[  665.048615]  [<ffffffff81002bab>] do_syscall_64+0x5b/0xd0
[  665.048621]  [<ffffffff817aec7e>] entry_SYSCALL_64_after_swapgs+0x58/0xc6
[  665.048645] Code: 48 8b 51 08 48 89 c8 65 48 03 05 b5 53 e7 7e 48 8b 70 08 48 
39 f2 75 e7 4c 8b 28 4d 85 ed 74 6d 49 63 46 20 4d 8b 06 48 8d 4a 08 <49> 8b 5c 05 
00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 bd 49 
[  665.048647] RIP  [<ffffffff81197daf>] kmem_cache_alloc+0x6f/0x1d0
[  665.048648]  RSP <ffffc900038abb40>
[  665.048649] CR2: 0000000000000001
[  665.243152] ---[ end trace 0000000000000002 ]---

and 

[  576.690934] general protection fault: 0000 [#1] PREEMPT SMP
[  576.690960] Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_nat nfsd 
iptable_nat nf_nat_ipv4 nf_nat xt_hl nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter 
ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables rtc_pcf8563 
coretemp rtc_cmos lm75 max6650 i2c_i801 i2c_smbus uio_cif uvcvideo videobuf2_vmalloc 
videobuf2_memops videobuf2_v4l2 videobuf2_core videodev fuse ipv6 autofs4 whiteheat 
ti_usb_3410_5052 spcp8x5 safe_serial pl2303 oti6858 option usb_wwan mos7840 mos7720 
mct_u232 keyspan_pda keyspan ezusb iuu_phoenix io_ti io_edgeport ftdi_sio f81232 
empeg digi_acceleport cypress_m8 ch341 belkin_sa ark3116 aircable radeon ttm i915
[  576.690963] CPU: 0 PID: 11963 Comm: kworker/0:0 Not tainted 4.9.228-rt147cpx+ 
#170
[  576.690965] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop 
Reference Platform, BIOS 6.00 07/02/2015
[  576.690975] Workqueue: events key_garbage_collector
[  576.690977] task: ffff8800397663c0 task.stack: ffffc90004000000
[  576.690985] RIP: 0010:[<ffffffff8139ccfa>]  [<ffffffff8139ccfa>] rb_next+0x1a/0x50
[  576.690986] RSP: 0018:ffffc90004003dd8  EFLAGS: 00010286
[  576.690987] RAX: ff880039484308ff RBX: 7fffffffffffffff RCX: 0000000000000000
[  576.690988] RDX: ff88003781050914 RSI: ffff8800397663c0 RDI: ffff880037810f08
[  576.690989] RBP: ffffc90004003dd8 R08: ffff88003ca61100 R09: 0000000000000000
[  576.690990] R10: ffff8800370ee080 R11: 0000000000000000 R12: ffff8800397663c0
[  576.690991] R13: ffff8800397663c0 R14: 000000005efc6657 R15: ffff880037810f08
[  576.690993] FS:  0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  576.691036] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  576.691050] CR2: 00007f2113b2c000 CR3: 000000003a5bd000 CR4: 00000000003606f0
[  576.691088] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  576.691089] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  576.691090] Stack:
[  576.691093]  ffffc90004003e28 ffffffff8133bef3 ffff880039766970 ffff8800370e0000
[  576.691095]  ffffc90004003e40 ffff88003d5236c0 ffff88003fc3e340 0000000000000000
[  576.691097]  ffff88003fc42b00 ffffffff81cc3e60 ffffc90004003e68 ffffffff8107b035
[  576.691098] Call Trace:
[  576.691104]  [<ffffffff8133bef3>] key_garbage_collector+0xa3/0x3d0
[  576.691112]  [<ffffffff8107b035>] process_one_work+0x145/0x450
[  576.691114]  [<ffffffff8107b3a9>] worker_thread+0x69/0x4f0
[  576.691117]  [<ffffffff8107b340>] ? process_one_work+0x450/0x450
[  576.691120]  [<ffffffff810806ac>] kthread+0xec/0x110
[  576.691122]  [<ffffffff810805c0>] ? kthread_park+0x60/0x60
[  576.691128]  [<ffffffff817aed8e>] ret_from_fork+0x3e/0x50
[  576.691151] Code: 52 08 48 85 d2 75 eb 5d c3 31 c0 5d c3 0f 1f 40 00 55 48 8b 
17 48 89 e5 48 39 d7 74 35 48 8b 47 08 48 85 c0 75 05 eb 1a 48 89 d0 <48> 8b 50 10 
48 85 d2 75 f4 5d c3 48 3b 79 08 75 11 48 8b 11 48 
[  576.691154] RIP  [<ffffffff8139ccfa>] rb_next+0x1a/0x50
[  576.691155]  RSP <ffffc90004003dd8>
[  576.832119] ---[ end trace 0000000000000002 ]---

Digging into this showed that this happens when sending rt signals to rt processes.
When sending rt signals to a rt process and at the same time connecting multiple
times via ssh triggered the bug within minutes.

When searching for a reason I stumbled over a this commit by Linus:

4306259ff6b8b682322d9aeb0c12b27c61c4a548
signal: avoid double atomic counter increments for user accounting
[ Upstream commit fda31c50292a5062332fa0343c084bd9f46604d9 ]

These changes were not back ported to the following rt patch by Thomas which 
resulted in a possible use after free:

bbe92cd3d383dafe071b0f1c45b3603b8666fb88
signals: Allow rt tasks to cache one sigqueue struct
To avoid allocation allow rt tasks to cache one sigqueue struct in
task struct.

So with the following changes I could not reproduce the bugs:

diff --git a/kernel/signal.c b/kernel/signal.c
index aca4b4cd11a2..1f335772615c 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -456,8 +456,8 @@ static void sigqueue_free_current(struct sigqueue *q)
 
        up = q->user;
        if (rt_prio(current->normal_prio) && !put_task_cache(current, q)) {
-               atomic_dec(&up->sigpending);
-               free_uid(up);
+               if (atomic_dec_and_test(&q->user->sigpending))
+                 free_uid(up);
        } else
                  __sigqueue_free(q);
 }

When answering to this mail could you please cc me for I am not subscribed to 
linux-rt-users list.

Thanks and kind regards,
Matthias