All of lore.kernel.org
 help / color / mirror / Atom feed
* 4.14.29 - tcp_push() - null skb's cb dereference
@ 2018-03-28 10:51 Krzysztof Blaszkowski
  2018-03-28 13:38 ` Eric Dumazet
  0 siblings, 1 reply; 5+ messages in thread
From: Krzysztof Blaszkowski @ 2018-03-28 10:51 UTC (permalink / raw)
  To: netdev

Hi,

I noticed a kernel bug report like below:

[95576.826393] BUG: unable to handle kernel NULL pointer dereference at
0000000000000038
[95576.834296] IP: tcp_push+0x3d/0x110
[95576.837829] PGD 2c8474067 P4D 2c8474067 PUD 1119cf067 PMD 0 
[95576.843536] Oops: 0002 [#1] SMP NOPTI
[95576.847247] CPU: 3 PID: 1682 Comm: nginx Not tainted 4.14.29 #1
[95576.854421] Hardware name: PC-FACTORY empty/Tyan Transport GT24-
B3992-E, BIOS 'V1.06.B10 ' 06/23/2009
[95576.863678] task: ffff9e4f150b2580 task.stack: ffffb21401a50000
[95576.869641] RIP: 0010:tcp_push+0x3d/0x110
[95576.873692] RSP: 0018:ffffb21401a53be0 EFLAGS: 00010246
[95576.878959] RAX: 0000000000000000 RBX: 0000000000009310 RCX:
0000000000000000
[95576.886133] RDX: 0000000000000001 RSI: 0000000000000040 RDI:
ffff9e4e79871f00
[95576.893306] RBP: ffffb21401a53c78 R08: 00000000000065d0 R09:
ffff9e4e79872048
[95576.900479] R10: 00000000000005a8 R11: 0000000000000000 R12:
00000000ffffffe0
[95576.907652] R13: ffffb21401a53cf8 R14: ffff9e4e79871f00 R15:
ffff9e4f68727100
[95576.914824] FS:  00007f683b4e1700(0000) GS:ffff9e50732c0000(0000)
knlGS:0000000000000000
[95576.922953] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[95576.928737] CR2: 0000000000000038 CR3: 000000020aa0e000 CR4:
00000000000006e0
[95576.935908] Call Trace:
[95576.938403]  ? tcp_sendmsg_locked+0x60a/0xd90
[95576.942872]  tcp_sendmsg+0x27/0x40
[95576.946386]  inet_sendmsg+0x2c/0xa0
[95576.949987]  sock_sendmsg+0x33/0x40
[95576.953587]  sock_write_iter+0x76/0xd0
[95576.957448]  do_iter_readv_writev+0x108/0x160
[95576.961917]  do_iter_write+0x82/0x190
[95576.965690]  vfs_writev+0xbb/0x120
[95576.969203]  ? ep_poll+0x240/0x3f0
[95576.972714]  do_writev+0x4d/0xd0
[95576.976052]  ? do_writev+0x4d/0xd0
[95576.979564]  SyS_writev+0xb/0x10
[95576.982905]  do_syscall_64+0x5a/0x110
[95576.986678]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[95576.991838] RIP: 0033:0x7f6839e56170
[95576.995523] RSP: 002b:00007ffe34c9cf48 EFLAGS: 00000246 ORIG_RAX:
0000000000000014
[95577.003265] RAX: ffffffffffffffda RBX: 0000000004b622c0 RCX:
00007f6839e56170
[95577.010507] RDX: 0000000000000017 RSI: 00007ffe34c9cfd0 RDI:
0000000000000012
[95577.017749] RBP: 00007ffe34c9cfb0 R08: 00000000004b1c2f R09:
00007ffe34c9d130
[95577.024989] R10: 000000000001d6c2 R11: 0000000000000246 R12:
0000000002f3eec0
[95577.032228] R13: 000000007fffefff R14: 0000000000000000 R15:
0000000002f3eec0
[95577.039469] Code: 01 00 00 4c 8d 8f 48 01 00 00 41 89 d2 41 89 f3 89
ca b9 00 00 00 00 49 39 c1 48 0f 44 c1 41 81 e3 00 80 00 00 0f 85 ac 00
00 00 <80> 48 38 08 8b 8f 64 06 00 00 89 8f 6c 06 00 00 83 e6 01 74 0c 
[95577.058617] RIP: tcp_push+0x3d/0x110 RSP: ffffb21401a53be0
[95577.064208] CR2: 0000000000000038
[95577.068056] ---[ end trace 19bfaf872fd3ef10 ]--



further report analysis ended up in net/ipv4/tcp.c : 


(gdb) list *(tcp_push+0x3d)
0xfd is in tcp_push (/data/work/linux-4.14.29/net/ipv4/tcp.c:630).
625	}
626	EXPORT_SYMBOL(tcp_ioctl);
627	
628	static inline void tcp_mark_push(struct tcp_sock *tp, struct
sk_buff *skb)
629	{
630		TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_PSH;
631		tp->pushed_seq = tp->write_seq;
632	}


and tcp_mark_push() is inlined in tcp_push() indeed.

Looking forward to seeing rationale for tcp_write_queue_tail(sk)
returning null skb - assuming the above is correct.

Thanks

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: 4.14.29 - tcp_push() - null skb's cb dereference
  2018-03-28 10:51 4.14.29 - tcp_push() - null skb's cb dereference Krzysztof Blaszkowski
@ 2018-03-28 13:38 ` Eric Dumazet
  2018-03-28 13:43   ` Krzysztof Blaszkowski
  2018-03-28 14:38   ` David Miller
  0 siblings, 2 replies; 5+ messages in thread
From: Eric Dumazet @ 2018-03-28 13:38 UTC (permalink / raw)
  To: Krzysztof Blaszkowski, netdev; +Cc: Greg Kroah-Hartman, David S. Miller



On 03/28/2018 03:51 AM, Krzysztof Blaszkowski wrote:
> Hi,
> 
> I noticed a kernel bug report like below:
> 
> [95576.826393] BUG: unable to handle kernel NULL pointer dereference at
> 0000000000000038
> [95576.834296] IP: tcp_push+0x3d/0x110
> [95576.837829] PGD 2c8474067 P4D 2c8474067 PUD 1119cf067 PMD 0 
> [95576.843536] Oops: 0002 [#1] SMP NOPTI
> [95576.847247] CPU: 3 PID: 1682 Comm: nginx Not tainted 4.14.29 #1
> [95576.854421] Hardware name: PC-FACTORY empty/Tyan Transport GT24-
> B3992-E, BIOS 'V1.06.B10 ' 06/23/2009
> [95576.863678] task: ffff9e4f150b2580 task.stack: ffffb21401a50000
> [95576.869641] RIP: 0010:tcp_push+0x3d/0x110
> [95576.873692] RSP: 0018:ffffb21401a53be0 EFLAGS: 00010246
> [95576.878959] RAX: 0000000000000000 RBX: 0000000000009310 RCX:
> 0000000000000000
> [95576.886133] RDX: 0000000000000001 RSI: 0000000000000040 RDI:
> ffff9e4e79871f00
> [95576.893306] RBP: ffffb21401a53c78 R08: 00000000000065d0 R09:
> ffff9e4e79872048
> [95576.900479] R10: 00000000000005a8 R11: 0000000000000000 R12:
> 00000000ffffffe0
> [95576.907652] R13: ffffb21401a53cf8 R14: ffff9e4e79871f00 R15:
> ffff9e4f68727100
> [95576.914824] FS:  00007f683b4e1700(0000) GS:ffff9e50732c0000(0000)
> knlGS:0000000000000000
> [95576.922953] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [95576.928737] CR2: 0000000000000038 CR3: 000000020aa0e000 CR4:
> 00000000000006e0
> [95576.935908] Call Trace:
> [95576.938403]  ? tcp_sendmsg_locked+0x60a/0xd90
> [95576.942872]  tcp_sendmsg+0x27/0x40
> [95576.946386]  inet_sendmsg+0x2c/0xa0
> [95576.949987]  sock_sendmsg+0x33/0x40
> [95576.953587]  sock_write_iter+0x76/0xd0
> [95576.957448]  do_iter_readv_writev+0x108/0x160
> [95576.961917]  do_iter_write+0x82/0x190
> [95576.965690]  vfs_writev+0xbb/0x120
> [95576.969203]  ? ep_poll+0x240/0x3f0
> [95576.972714]  do_writev+0x4d/0xd0
> [95576.976052]  ? do_writev+0x4d/0xd0
> [95576.979564]  SyS_writev+0xb/0x10
> [95576.982905]  do_syscall_64+0x5a/0x110
> [95576.986678]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> [95576.991838] RIP: 0033:0x7f6839e56170
> [95576.995523] RSP: 002b:00007ffe34c9cf48 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000014
> [95577.003265] RAX: ffffffffffffffda RBX: 0000000004b622c0 RCX:
> 00007f6839e56170
> [95577.010507] RDX: 0000000000000017 RSI: 00007ffe34c9cfd0 RDI:
> 0000000000000012
> [95577.017749] RBP: 00007ffe34c9cfb0 R08: 00000000004b1c2f R09:
> 00007ffe34c9d130
> [95577.024989] R10: 000000000001d6c2 R11: 0000000000000246 R12:
> 0000000002f3eec0
> [95577.032228] R13: 000000007fffefff R14: 0000000000000000 R15:
> 0000000002f3eec0
> [95577.039469] Code: 01 00 00 4c 8d 8f 48 01 00 00 41 89 d2 41 89 f3 89
> ca b9 00 00 00 00 49 39 c1 48 0f 44 c1 41 81 e3 00 80 00 00 0f 85 ac 00
> 00 00 <80> 48 38 08 8b 8f 64 06 00 00 89 8f 6c 06 00 00 83 e6 01 74 0c 
> [95577.058617] RIP: tcp_push+0x3d/0x110 RSP: ffffb21401a53be0
> [95577.064208] CR2: 0000000000000038
> [95577.068056] ---[ end trace 19bfaf872fd3ef10 ]--
> 
> 
> 
> further report analysis ended up in net/ipv4/tcp.c : 
> 
> 
> (gdb) list *(tcp_push+0x3d)
> 0xfd is in tcp_push (/data/work/linux-4.14.29/net/ipv4/tcp.c:630).
> 625	}
> 626	EXPORT_SYMBOL(tcp_ioctl);
> 627	
> 628	static inline void tcp_mark_push(struct tcp_sock *tp, struct
> sk_buff *skb)
> 629	{
> 630		TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_PSH;
> 631		tp->pushed_seq = tp->write_seq;
> 632	}
> 
> 
> and tcp_mark_push() is inlined in tcp_push() indeed.
> 
> Looking forward to seeing rationale for tcp_write_queue_tail(sk)
> returning null skb - assuming the above is correct.
> 
> Thanks

This has been reported 5 or 6 times already :/

For some reason the fix is missing from 4.14 stable tree.

https://patchwork.ozlabs.org/patch/886324/

Note this patch was needed even before a27fid7a8ed38 (tcp: purge write queue upon RST)
since tcp disconnect would have caused the same issue.



Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: 4.14.29 - tcp_push() - null skb's cb dereference
  2018-03-28 13:38 ` Eric Dumazet
@ 2018-03-28 13:43   ` Krzysztof Blaszkowski
  2018-03-28 14:38   ` David Miller
  1 sibling, 0 replies; 5+ messages in thread
From: Krzysztof Blaszkowski @ 2018-03-28 13:43 UTC (permalink / raw)
  To: Eric Dumazet, netdev; +Cc: Greg Kroah-Hartman, David S. Miller

On Wed, 2018-03-28 at 06:38 -0700, Eric Dumazet wrote:
> 
> On 03/28/2018 03:51 AM, Krzysztof Blaszkowski wrote:
> > 
> > Hi,
> > 
> > I noticed a kernel bug report like below:
> > 
> > [95576.826393] BUG: unable to handle kernel NULL pointer
> > dereference at
> > 0000000000000038
> > [95576.834296] IP: tcp_push+0x3d/0x110
> > [95576.837829] PGD 2c8474067 P4D 2c8474067 PUD 1119cf067 PMD 0 
> > [95576.843536] Oops: 0002 [#1] SMP NOPTI
> > [95576.847247] CPU: 3 PID: 1682 Comm: nginx Not tainted 4.14.29 #1
> > [95576.854421] Hardware name: PC-FACTORY empty/Tyan Transport GT24-
> > B3992-E, BIOS 'V1.06.B10 ' 06/23/2009
> > [95576.863678] task: ffff9e4f150b2580 task.stack: ffffb21401a50000
> > [95576.869641] RIP: 0010:tcp_push+0x3d/0x110
> > [95576.873692] RSP: 0018:ffffb21401a53be0 EFLAGS: 00010246
> > [95576.878959] RAX: 0000000000000000 RBX: 0000000000009310 RCX:
> > 0000000000000000
> > [95576.886133] RDX: 0000000000000001 RSI: 0000000000000040 RDI:
> > ffff9e4e79871f00
> > [95576.893306] RBP: ffffb21401a53c78 R08: 00000000000065d0 R09:
> > ffff9e4e79872048
> > [95576.900479] R10: 00000000000005a8 R11: 0000000000000000 R12:
> > 00000000ffffffe0
> > [95576.907652] R13: ffffb21401a53cf8 R14: ffff9e4e79871f00 R15:
> > ffff9e4f68727100
> > [95576.914824] FS:  00007f683b4e1700(0000)
> > GS:ffff9e50732c0000(0000)
> > knlGS:0000000000000000
> > [95576.922953] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [95576.928737] CR2: 0000000000000038 CR3: 000000020aa0e000 CR4:
> > 00000000000006e0
> > [95576.935908] Call Trace:
> > [95576.938403]  ? tcp_sendmsg_locked+0x60a/0xd90
> > [95576.942872]  tcp_sendmsg+0x27/0x40
> > [95576.946386]  inet_sendmsg+0x2c/0xa0
> > [95576.949987]  sock_sendmsg+0x33/0x40
> > [95576.953587]  sock_write_iter+0x76/0xd0
> > [95576.957448]  do_iter_readv_writev+0x108/0x160
> > [95576.961917]  do_iter_write+0x82/0x190
> > [95576.965690]  vfs_writev+0xbb/0x120
> > [95576.969203]  ? ep_poll+0x240/0x3f0
> > [95576.972714]  do_writev+0x4d/0xd0
> > [95576.976052]  ? do_writev+0x4d/0xd0
> > [95576.979564]  SyS_writev+0xb/0x10
> > [95576.982905]  do_syscall_64+0x5a/0x110
> > [95576.986678]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> > [95576.991838] RIP: 0033:0x7f6839e56170
> > [95576.995523] RSP: 002b:00007ffe34c9cf48 EFLAGS: 00000246
> > ORIG_RAX:
> > 0000000000000014
> > [95577.003265] RAX: ffffffffffffffda RBX: 0000000004b622c0 RCX:
> > 00007f6839e56170
> > [95577.010507] RDX: 0000000000000017 RSI: 00007ffe34c9cfd0 RDI:
> > 0000000000000012
> > [95577.017749] RBP: 00007ffe34c9cfb0 R08: 00000000004b1c2f R09:
> > 00007ffe34c9d130
> > [95577.024989] R10: 000000000001d6c2 R11: 0000000000000246 R12:
> > 0000000002f3eec0
> > [95577.032228] R13: 000000007fffefff R14: 0000000000000000 R15:
> > 0000000002f3eec0
> > [95577.039469] Code: 01 00 00 4c 8d 8f 48 01 00 00 41 89 d2 41 89
> > f3 89
> > ca b9 00 00 00 00 49 39 c1 48 0f 44 c1 41 81 e3 00 80 00 00 0f 85
> > ac 00
> > 00 00 <80> 48 38 08 8b 8f 64 06 00 00 89 8f 6c 06 00 00 83 e6 01 74
> > 0c 
> > [95577.058617] RIP: tcp_push+0x3d/0x110 RSP: ffffb21401a53be0
> > [95577.064208] CR2: 0000000000000038
> > [95577.068056] ---[ end trace 19bfaf872fd3ef10 ]--
> > 
> > 
> > 
> > further report analysis ended up in net/ipv4/tcp.c : 
> > 
> > 
> > (gdb) list *(tcp_push+0x3d)
> > 0xfd is in tcp_push (/data/work/linux-4.14.29/net/ipv4/tcp.c:630).
> > 625	}
> > 626	EXPORT_SYMBOL(tcp_ioctl);
> > 627	
> > 628	static inline void tcp_mark_push(struct tcp_sock *tp,
> > struct
> > sk_buff *skb)
> > 629	{
> > 630		TCP_SKB_CB(skb)->tcp_flags |= TCPHDR_PSH;
> > 631		tp->pushed_seq = tp->write_seq;
> > 632	}
> > 
> > 
> > and tcp_mark_push() is inlined in tcp_push() indeed.
> > 
> > Looking forward to seeing rationale for tcp_write_queue_tail(sk)
> > returning null skb - assuming the above is correct.
> > 
> > Thanks
> 
> This has been reported 5 or 6 times already :/
> 
> For some reason the fix is missing from 4.14 stable tree.
> 
> https://patchwork.ozlabs.org/patch/886324/
> 
> Note this patch was needed even before a27fid7a8ed38 (tcp: purge
> write queue upon RST)
> since tcp disconnect would have caused the same issue.
> 
> 
> 
> Thanks.

I see. Many thanks.


> 
> 

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: 4.14.29 - tcp_push() - null skb's cb dereference
  2018-03-28 13:38 ` Eric Dumazet
  2018-03-28 13:43   ` Krzysztof Blaszkowski
@ 2018-03-28 14:38   ` David Miller
  2018-03-28 16:33     ` Eric Dumazet
  1 sibling, 1 reply; 5+ messages in thread
From: David Miller @ 2018-03-28 14:38 UTC (permalink / raw)
  To: eric.dumazet; +Cc: krzysztof.blaszkowski, netdev, gregkh

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Wed, 28 Mar 2018 06:38:21 -0700

> https://patchwork.ozlabs.org/patch/886324/

I have this in my current -stable submission set, and I'm working
actively on this right now.

Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: 4.14.29 - tcp_push() - null skb's cb dereference
  2018-03-28 14:38   ` David Miller
@ 2018-03-28 16:33     ` Eric Dumazet
  0 siblings, 0 replies; 5+ messages in thread
From: Eric Dumazet @ 2018-03-28 16:33 UTC (permalink / raw)
  To: David Miller; +Cc: krzysztof.blaszkowski, netdev, gregkh



On 03/28/2018 07:38 AM, David Miller wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Wed, 28 Mar 2018 06:38:21 -0700
> 
>> https://patchwork.ozlabs.org/patch/886324/
> 
> I have this in my current -stable submission set, and I'm working
> actively on this right now.


Thanks a lot David.

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-03-28 16:33 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-28 10:51 4.14.29 - tcp_push() - null skb's cb dereference Krzysztof Blaszkowski
2018-03-28 13:38 ` Eric Dumazet
2018-03-28 13:43   ` Krzysztof Blaszkowski
2018-03-28 14:38   ` David Miller
2018-03-28 16:33     ` Eric Dumazet

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.