linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dmitry Vyukov <dvyukov@google.com>
To: David Ahern <dsa@cumulusnetworks.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>,
	Mahesh Bandewar <maheshb@google.com>,
	Eric Dumazet <edumazet@google.com>,
	David Miller <davem@davemloft.net>,
	Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
	James Morris <jmorris@namei.org>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	Patrick McHardy <kaber@trash.net>,
	netdev <netdev@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Cong Wang <xiyou.wangcong@gmail.com>,
	syzkaller <syzkaller@googlegroups.com>
Subject: Re: net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone
Date: Tue, 7 Mar 2017 09:43:31 +0100	[thread overview]
Message-ID: <CACT4Y+aomj0W8M9JNr1GLsT8EFFHV9YsOTYzWkvwKZANE9hiBQ@mail.gmail.com> (raw)
In-Reply-To: <14c01aea-6c2f-6ba5-6aee-52c55f410da7@cumulusnetworks.com>

On Tue, Mar 7, 2017 at 12:41 AM, David Ahern <dsa@cumulusnetworks.com> wrote:
> On 3/6/17 11:51 AM, Dmitry Vyukov wrote:
>> We hit it several thousand times, but we get only several dozens of
>> crashes per day on ~80 VMs. So if you try to reproduce it on a single
>> machine it can take days for a single crash.
>> If you are ready to go that route, here are some instructions on
>> setting up syzkaller:
>> https://github.com/google/syzkaller
>> You also need kernel built with CONFIG_KASAN.
>
> ack and I have it setup on ubuntu 16.10 which has a fairly new compiler.
>
>> I am ready to help with resolving any issues.
>>
>> Another possible route is if you give me a patch with some additional
>> WARNINGs. Then I can deploy it to bots and collect stacks.
>
> try the attached.


This is on c1ae3cfa0e89fa1a7ecc4c99031f5e9ae99d9201. No other kernel
output from your patch (pr_err).

------------[ cut here ]------------
WARNING: CPU: 1 PID: 30179 at net/ipv6/ip6_fib.c:158
rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 30179 Comm: syz-executor3 Not tainted 4.11.0-rc1+ #310
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 panic+0x20f/0x426 kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
 rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
 fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
 fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
 ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
 inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
 rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 call_write_iter include/linux/fs.h:1733 [inline]
 do_iter_readv_writev fs/read_write.c:696 [inline]
 __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
 do_readv_writev+0x13f/0x200 fs/read_write.c:894
 vfs_writev+0x87/0xc0 fs/read_write.c:921
 do_writev+0x110/0x2c0 fs/read_write.c:954
 SYSC_writev fs/read_write.c:1027 [inline]
 SyS_writev+0x27/0x30 fs/read_write.c:1024
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007f31fcf33b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00000000004458d9
RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000005
RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 0000000020fad000 R14: 0000000000001000 R15: 0000000000000003



------------[ cut here ]------------
WARNING: CPU: 2 PID: 31175 at net/ipv6/ip6_fib.c:158
rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
Kernel panic - not syncing: panic_on_warn set ...

CPU: 2 PID: 31175 Comm: syz-executor1 Not tainted 4.11.0-rc1+ #310
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x2fb/0x3fd lib/dump_stack.c:52
 panic+0x20f/0x426 kernel/panic.c:180
 __warn+0x1c4/0x1e0 kernel/panic.c:541
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rt6_rcu_free+0x61/0x70 net/ipv6/ip6_fib.c:158
 rt6_release+0x1ee/0x290 net/ipv6/ip6_fib.c:189
 fib6_add_rt2node net/ipv6/ip6_fib.c:922 [inline]
 fib6_add+0x1d51/0x3290 net/ipv6/ip6_fib.c:1081
kvm_vm_ioctl_deassign_device: device hasn't been assigned before, so
cannot be deassigned
 __ip6_ins_rt+0x60/0x80 net/ipv6/route.c:948
 ip6_route_add+0x1a7/0x310 net/ipv6/route.c:2130
 inet6_rtm_newroute+0x191/0x1b0 net/ipv6/route.c:3294
 rtnetlink_rcv_msg+0x609/0x860 net/core/rtnetlink.c:4104
 netlink_rcv_skb+0x2ab/0x390 net/netlink/af_netlink.c:2298
 rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4110
 netlink_unicast_kernel net/netlink/af_netlink.c:1231 [inline]
 netlink_unicast+0x525/0x730 net/netlink/af_netlink.c:1257
 netlink_sendmsg+0xab3/0xe70 net/netlink/af_netlink.c:1803
 sock_sendmsg_nosec net/socket.c:633 [inline]
 sock_sendmsg+0xca/0x110 net/socket.c:643
 sock_write_iter+0x326/0x600 net/socket.c:846
 call_write_iter include/linux/fs.h:1733 [inline]
 do_iter_readv_writev fs/read_write.c:696 [inline]
 __do_readv_writev+0xbbc/0x10a0 fs/read_write.c:862
 do_readv_writev+0x13f/0x200 fs/read_write.c:894
 vfs_writev+0x87/0xc0 fs/read_write.c:921
 do_writev+0x110/0x2c0 fs/read_write.c:954
 SYSC_writev fs/read_write.c:1027 [inline]
 SyS_writev+0x27/0x30 fs/read_write.c:1024
 entry_SYSCALL_64_fastpath+0x1f/0xc2
RIP: 0033:0x4458d9
RSP: 002b:00007f1639006b58 EFLAGS: 00000292 ORIG_RAX: 0000000000000014
RAX: ffffffffffffffda RBX: 0000000000000019 RCX: 00000000004458d9
RDX: 0000000000000001 RSI: 00000000207cd000 RDI: 0000000000000019
RBP: 00000000006e30c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000292 R12: 0000000000708000
R13: 0000000000000010 R14: 0000000000000003 R15: 0000000000000000

  reply	other threads:[~2017-03-07  8:44 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-03 14:39 net: heap out-of-bounds in fib6_clean_node/rt6_fill_node/fib6_age/fib6_prune_clone Dmitry Vyukov
2017-03-03 19:12 ` David Ahern
2017-03-03 19:14   ` Dmitry Vyukov
2017-03-04 18:57   ` Dmitry Vyukov
2017-03-04 19:00     ` Dmitry Vyukov
2017-03-04 20:15     ` Eric Dumazet
2017-03-05 10:53       ` Dmitry Vyukov
2017-03-06 17:31       ` David Ahern
2017-03-06 18:51         ` Dmitry Vyukov
2017-03-06 23:41           ` David Ahern
2017-03-07  8:43             ` Dmitry Vyukov [this message]
2017-03-07  9:21               ` Dmitry Vyukov
2017-03-07 18:03                 ` David Ahern
2017-03-07 18:13                   ` Dmitry Vyukov
2017-03-07 18:43                     ` David Ahern
2017-03-07 19:02                       ` Dmitry Vyukov
2017-03-07 19:30                         ` Dmitry Vyukov
2017-03-07 20:00                           ` Dmitry Vyukov
2017-03-08 11:55                             ` Dmitry Vyukov
2017-03-27 12:42                               ` Dmitry Vyukov
2017-03-27 13:57                                 ` David Ahern
2017-03-27 14:23                                   ` Dmitry Vyukov
2017-04-18 20:43                                     ` Andrey Konovalov
2017-04-18 23:20                                       ` David Ahern
2017-04-19  1:09                                         ` Andrey Konovalov
2017-04-19 16:09                                       ` David Ahern
2017-04-19 16:12                                         ` Andrey Konovalov
2017-04-19 16:29                                           ` David Ahern
2017-04-19 23:47                                           ` Cong Wang
2017-04-19 23:51                                             ` David Ahern
2017-04-20  8:35                                               ` Dmitry Vyukov
2017-04-20 12:10                                                 ` Andrey Konovalov
2017-04-20 15:28                                                   ` Andrey Konovalov
2017-04-20 15:29                                                     ` Andrey Konovalov
2017-04-20 15:35                                                     ` David Ahern
2017-04-20 15:39                                                       ` Andrey Konovalov
2017-04-20 16:09                                                         ` Andrey Konovalov
2017-04-21 14:27                                                           ` David Ahern
2017-04-21 16:47                                                             ` Eric Dumazet
2017-04-21 18:25                                                               ` David Ahern
2017-04-25 15:51                                       ` David Ahern
2017-04-25 15:57                 ` David Ahern
2017-03-07 17:17               ` David Ahern
2017-03-07 17:45                 ` Dmitry Vyukov
2017-03-07 17:57               ` David Ahern
2017-04-25 15:56     ` David Ahern
2017-04-25 16:36       ` Andrey Konovalov
2017-04-25 16:38         ` Andrey Konovalov
2017-04-25 16:40           ` David Ahern

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CACT4Y+aomj0W8M9JNr1GLsT8EFFHV9YsOTYzWkvwKZANE9hiBQ@mail.gmail.com \
    --to=dvyukov@google.com \
    --cc=davem@davemloft.net \
    --cc=dsa@cumulusnetworks.com \
    --cc=edumazet@google.com \
    --cc=eric.dumazet@gmail.com \
    --cc=jmorris@namei.org \
    --cc=kaber@trash.net \
    --cc=kuznet@ms2.inr.ac.ru \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maheshb@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=syzkaller@googlegroups.com \
    --cc=xiyou.wangcong@gmail.com \
    --cc=yoshfuji@linux-ipv6.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).