Hi!

I've got the following error report while running the syzkaller fuzzer.

On commit d8e435f3ab6fea2ea324dce72b51dd7761747523 (Nov 26).

A reproducer is attached.

a.out: vmalloc: allocation failure, allocated 823562240 of 1427091456
bytes, mode:0x24000c2(GFP_KERNEL|__GFP_HIGHMEM)

oom_reaper: reaped process 3810 (a.out), now anon-rss:0kB,
file-rss:0kB, shmem-rss:0kB
a.out invoked oom-killer:
gfp_mask=0x24002c2(GFP_KERNEL|__GFP_HIGHMEM|__GFP_NOWARN), nodemask=0,
order=0, oom_score_adj=0
a.out cpuset=/ mems_allowed=0
CPU: 0 PID: 3814 Comm: a.out Not tainted 4.9.0-rc6+ #457
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
 ffff880068667380 ffffffff81c73b14 ffff880068667710 ffff88006b469018
 ffff880068667718 0000000000000000 ffff880068667400 ffffffff81641a87
 0000000000000000 0000000000000000 0000000000000297 ffffffff84d37280
Call Trace:
 [<     inline     >] __dump_stack lib/dump_stack.c:15
 [<ffffffff81c73b14>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
 [<ffffffff81641a87>] dump_header.isra.21+0x16f/0x5f5 mm/oom_kill.c:416
 [<ffffffff8154bad8>] oom_kill_process+0x4d8/0xab0 mm/oom_kill.c:835
 [<ffffffff8154c77c>] out_of_memory+0x2dc/0x1790 mm/oom_kill.c:1044
 [<     inline     >] __alloc_pages_may_oom mm/page_alloc.c:3086
 [<ffffffff8155afb6>] __alloc_pages_slowpath+0x1886/0x1bf0 mm/page_alloc.c:3683
 [<ffffffff8155b8e2>] __alloc_pages_nodemask+0x5c2/0x710 mm/page_alloc.c:3781
 [<ffffffff816236a4>] alloc_pages_current+0xf4/0x400 mm/mempolicy.c:2072
 [<     inline     >] alloc_pages ./include/linux/gfp.h:469
 [<     inline     >] __vmalloc_area_node mm/vmalloc.c:1631
 [<ffffffff815f8eab>] __vmalloc_node_range+0x33b/0x690 mm/vmalloc.c:1691
 [<     inline     >] __vmalloc_node mm/vmalloc.c:1734
 [<     inline     >] __vmalloc_node_flags mm/vmalloc.c:1748
 [<ffffffff815f92cb>] vmalloc+0x5b/0x70 mm/vmalloc.c:1763
 [<ffffffff82fd0893>] xt_alloc_table_info+0x83/0x120
net/netfilter/x_tables.c:961
 [<     inline     >] do_replace net/ipv4/netfilter/ip_tables.c:1140
 [<ffffffff8335b420>] do_ipt_set_ctl+0x210/0x420
net/ipv4/netfilter/ip_tables.c:1687
 [<     inline     >] nf_sockopt net/netfilter/nf_sockopt.c:105
 [<ffffffff82efdab7>] nf_setsockopt+0x67/0xc0 net/netfilter/nf_sockopt.c:114
 [<ffffffff831be741>] ip_setsockopt+0xa1/0xb0 net/ipv4/ip_sockglue.c:1231
 [<ffffffff832700d5>] udp_setsockopt+0x45/0x80 net/ipv4/udp.c:2085
 [<ffffffff8346b31f>] ipv6_setsockopt+0x11f/0x140 net/ipv6/ipv6_sockglue.c:892
 [<ffffffff83a6cd5d>] sctp_setsockopt+0x15d/0x3d70 net/sctp/socket.c:3788
 [<ffffffff82ca40e6>] sock_common_setsockopt+0x96/0xd0 net/core/sock.c:2690
 [<     inline     >] SYSC_setsockopt net/socket.c:1757
 [<ffffffff82ca10c4>] SyS_setsockopt+0x154/0x240 net/socket.c:1736
 [<ffffffff840f2c41>] entry_SYSCALL_64_fastpath+0x1f/0xc2
arch/x86/entry/entry_64.S:209
CPU: 1 PID: 3810 Comm: a.out Not tainted 4.9.0-rc6+ #457
Mem-Info:
active_anon:1938 inactive_anon:75 isolated_anon:0
 active_file:14 inactive_file:30 isolated_file:4
 unevictable:0 dirty:0 writeback:0 unstable:0
 slab_reclaimable:3316 slab_unreclaimable:9767
 mapped:21 shmem:81 pagetables:309 bounce:0
 free:1 free_pcp:75 free_cma:0
Node 0 active_anon:7752kB inactive_anon:300kB active_file:56kB
inactive_file:120kB unevictable:0kB isolated(anon):0kB
isolated(file):16kB mapped:84kB dirty:0kB writeback:0kB shmem:324kB
writeback_tmp:0kB unstable:0kB pages_scanned:134 all_unreclaimable? no
Node 0 DMA free:4kB min:48kB low:60kB high:72kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
writepending:0kB present:15992kB managed:15908kB mlocked:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1641 1641 1641
Node 0 DMA32 free:0kB min:5156kB low:6836kB high:8516kB
active_anon:7752kB inactive_anon:300kB active_file:56kB
inactive_file:120kB unevictable:0kB writepending:0kB present:2080760kB
managed:1684640kB mlocked:0kB slab_reclaimable:13264kB
slab_unreclaimable:39060kB kernel_stack:2944kB pagetables:1236kB
bounce:0kB free_pcp:300kB local_pcp:120kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB
0*1024kB 0*2048kB 0*4096kB = 0kB
Node 0 DMA32: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB
0*1024kB 0*2048kB 0*4096kB = 0kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
148 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
524188 pages RAM
0 pages HighMem/MovableOnly
99051 pages reserved
[ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents
oom_score_adj name
    0  1767     5346      133      16       3        0         -1000 udevd
    0  1876     5315      122      15       3        0         -1000 udevd
    0  1877     5315      122      15       3        0         -1000 udevd
    0  3541     2493      573       8       3        0             0 dhclient
    0  3676    13231      171      22       3        0             0 rsyslogd
    0  3725     4725       52      15       3        0             0 cron
    0  3751    12490      155      28       3        0         -1000 sshd
    0  3775     3694       43      13       3        0             0 getty
    0  3776     3694       43      13       3        0             0 getty
    0  3777     3694       42      13       3        0             0 getty
    0  3778     3694       41      13       3        0             0 getty
    0  3779     3694       44      13       3        0             0 getty
    0  3780     3694       43      13       3        0             0 getty
    0  3785     3649       44      12       3        0             0 getty
    0  3797    17818      205      39       3        0             0 sshd
    0  3800     4474      126      15       3        0             0 bash
    0  3804     2053       22       9       3        0             0 a.out
    0  3805     2053       26       9       3        0             0 a.out
    0  3806    18488        0      18       3        0             0 a.out