From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D6BCC282CB for ; Tue, 5 Feb 2019 19:34:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id ED5A7217F9 for ; Tue, 5 Feb 2019 19:34:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JZO8s3Bo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727249AbfBETe2 (ORCPT ); Tue, 5 Feb 2019 14:34:28 -0500 Received: from mail-yw1-f65.google.com ([209.85.161.65]:47033 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726422AbfBETe2 (ORCPT ); Tue, 5 Feb 2019 14:34:28 -0500 Received: by mail-yw1-f65.google.com with SMTP id n12so2119341ywn.13 for ; Tue, 05 Feb 2019 11:34:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=edT7P8aIpw4ThXSB8LnzI9rkwfCpTBmfoXTnVdgt9zQ=; b=JZO8s3BoF+UwCAfLIAD7qmvrxUtloKVnb9TscNc/Zf7hc4K+tHGRSdai7s3rNdv+f+ eMH4M6LzPi+cQ6J8t9FxTM65ZFxEQYnfFZoh3YgVHNRMDZWy7TgjtR/QNkQOX4mx4u23 41w9rf/YkH90ulyDyxkMPEh08My5hObipO3iPylDuJniuebVAOk4UFoCZDStktsrJ/gh aJlT7VF1L+MD6WDF3HeY5ANoVSWt9g/u5gBRNs+Rab31dxWelgPZjQOnBXhi60Qev/EX qpaKT+61bJ8HI5t1I9C7GwDHoPTh+2lPFg3ThpH/1Arcrp7nViuLp5hPgzTAjqQdIata sNOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=edT7P8aIpw4ThXSB8LnzI9rkwfCpTBmfoXTnVdgt9zQ=; b=MYY1m4ErOBHV2w03DBvmVt810d7GiHgIXhesMYJrC6wv5TmIfnShPwgtcLYGy/0vkd 0Vmb3uRnygKoywdnlVXM39yGXhda1k2SY4ZILhWk4ZzczuJgdJyyiVUB7YdM1ltTBa3M FN/RZPsMXcTxPp6KUOdvm576D9zMSh+KKgOyoB+07uPV9ABifHMGrmsUEt3XElcHTgfF zQuz+zgX4zSPP5cdVQCMU1zAtTl1AkzkPqxUjckCuB6w53tOfayZYjRqKgCe+F4xWThs /GKb7ob6fdZd3m38JVYY8ewl6IgHgSq6mLe4lahNUelyvrMgngMWTz6b056c/SKrPbzR zKng== X-Gm-Message-State: AHQUAuYR6xfK62PFYGzjL6BeWEuoMGklKn0ld0XjC3qcIgJGuSQuK7QQ er8Bi8xGYkrWsAUlkgej0OPRX+5U X-Google-Smtp-Source: AHgI3IZPcLYtA3b80hDk6VfE4oyH6G2eje78yDhHQo6kFBtQhIWW5vCIAgMXEOQCiCJ6T9IOPNqTZQ== X-Received: by 2002:a81:9384:: with SMTP id k126mr5510581ywg.24.1549395266413; Tue, 05 Feb 2019 11:34:26 -0800 (PST) Received: from [10.67.48.231] ([192.19.223.250]) by smtp.googlemail.com with ESMTPSA id x133sm2127995ywg.57.2019.02.05.11.34.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Feb 2019 11:34:25 -0800 (PST) Subject: Re: Kernel panic in eth_header To: Eric Dumazet , Andrew , Netdev References: <18c17dde-5963-4412-2e98-ba44953f0ddd@seti.kr.ua> <19716555-3522-cbdd-a128-e2ec672f89cd@gmail.com> From: Florian Fainelli Openpgp: preference=signencrypt Autocrypt: addr=f.fainelli@gmail.com; prefer-encrypt=mutual; keydata= mQGiBEjPuBIRBACW9MxSJU9fvEOCTnRNqG/13rAGsj+vJqontvoDSNxRgmafP8d3nesnqPyR xGlkaOSDuu09rxuW+69Y2f1TzjFuGpBk4ysWOR85O2Nx8AJ6fYGCoeTbovrNlGT1M9obSFGQ X3IzRnWoqlfudjTO5TKoqkbOgpYqIo5n1QbEjCCwCwCg3DOH/4ug2AUUlcIT9/l3pGvoRJ0E AICDzi3l7pmC5IWn2n1mvP5247urtHFs/uusE827DDj3K8Upn2vYiOFMBhGsxAk6YKV6IP0d ZdWX6fqkJJlu9cSDvWtO1hXeHIfQIE/xcqvlRH783KrihLcsmnBqOiS6rJDO2x1eAgC8meAX SAgsrBhcgGl2Rl5gh/jkeA5ykwbxA/9u1eEuL70Qzt5APJmqVXR+kWvrqdBVPoUNy/tQ8mYc nzJJ63ng3tHhnwHXZOu8hL4nqwlYHRa9eeglXYhBqja4ZvIvCEqSmEukfivk+DlIgVoOAJbh qIWgvr3SIEuR6ayY3f5j0f2ejUMYlYYnKdiHXFlF9uXm1ELrb0YX4GMHz7QnRmxvcmlhbiBG YWluZWxsaSA8Zi5mYWluZWxsaUBnbWFpbC5jb20+iGYEExECACYCGyMGCwkIBwMCBBUCCAME FgIDAQIeAQIXgAUCVF/S8QUJHlwd3wAKCRBhV5kVtWN2DvCVAJ4u4/bPF4P3jxb4qEY8I2gS 6hG0gACffNWlqJ2T4wSSn+3o7CCZNd7SLSC5BA0ESM+4EhAQAL/o09boR9D3Vk1Tt7+gpYr3 WQ6hgYVON905q2ndEoA2J0dQxJNRw3snabHDDzQBAcqOvdi7YidfBVdKi0wxHhSuRBfuOppu pdXkb7zxuPQuSveCLqqZWRQ+Cc2QgF7SBqgznbe6Ngout5qXY5Dcagk9LqFNGhJQzUGHAsIs hap1f0B1PoUyUNeEInV98D8Xd/edM3mhO9nRpUXRK9Bvt4iEZUXGuVtZLT52nK6Wv2EZ1TiT OiqZlf1P+vxYLBx9eKmabPdm3yjalhY8yr1S1vL0gSA/C6W1o/TowdieF1rWN/MYHlkpyj9c Rpc281gAO0AP3V1G00YzBEdYyi0gaJbCEQnq8Vz1vDXFxHzyhgGz7umBsVKmYwZgA8DrrB0M oaP35wuGR3RJcaG30AnJpEDkBYHznI2apxdcuTPOHZyEilIRrBGzDwGtAhldzlBoBwE3Z3MY 31TOpACu1ZpNOMysZ6xiE35pWkwc0KYm4hJA5GFfmWSN6DniimW3pmdDIiw4Ifcx8b3mFrRO BbDIW13E51j9RjbO/nAaK9ndZ5LRO1B/8Fwat7bLzmsCiEXOJY7NNpIEpkoNoEUfCcZwmLrU +eOTPzaF6drw6ayewEi5yzPg3TAT6FV3oBsNg3xlwU0gPK3v6gYPX5w9+ovPZ1/qqNfOrbsE FRuiSVsZQ5s3AAMFD/9XjlnnVDh9GX/r/6hjmr4U9tEsM+VQXaVXqZuHKaSmojOLUCP/YVQo 7IiYaNssCS4FCPe4yrL4FJJfJAsbeyDykMN7wAnBcOkbZ9BPJPNCbqU6dowLOiy8AuTYQ48m vIyQ4Ijnb6GTrtxIUDQeOBNuQC/gyyx3nbL/lVlHbxr4tb6YkhkO6shjXhQh7nQb33FjGO4P WU11Nr9i/qoV8QCo12MQEo244RRA6VMud06y/E449rWZFSTwGqb0FS0seTcYNvxt8PB2izX+ HZA8SL54j479ubxhfuoTu5nXdtFYFj5Lj5x34LKPx7MpgAmj0H7SDhpFWF2FzcC1bjiW9mjW HaKaX23Awt97AqQZXegbfkJwX2Y53ufq8Np3e1542lh3/mpiGSilCsaTahEGrHK+lIusl6mz Joil+u3k01ofvJMK0ZdzGUZ/aPMZ16LofjFA+MNxWrZFrkYmiGdv+LG45zSlZyIvzSiG2lKy kuVag+IijCIom78P9jRtB1q1Q5lwZp2TLAJlz92DmFwBg1hyFzwDADjZ2nrDxKUiybXIgZp9 aU2d++ptEGCVJOfEW4qpWCCLPbOT7XBr+g/4H3qWbs3j/cDDq7LuVYIe+wchy/iXEJaQVeTC y5arMQorqTFWlEOgRA8OP47L9knl9i4xuR0euV6DChDrguup2aJVU4hPBBgRAgAPAhsMBQJU X9LxBQkeXB3fAAoJEGFXmRW1Y3YOj4UAn3nrFLPZekMeqX5aD/aq/dsbXSfyAKC45Go0YyxV HGuUuzv+GKZ6nsysJ7kCDQRXG8fwARAA6q/pqBi5PjHcOAUgk2/2LR5LjjesK50bCaD4JuNc YDhFR7Vs108diBtsho3w8WRd9viOqDrhLJTroVckkk74OY8r+3t1E0Dd4wHWHQZsAeUvOwDM PQMqTUBFuMi6ydzTZpFA2wBR9x6ofl8Ax+zaGBcFrRlQnhsuXLnM1uuvS39+pmzIjasZBP2H UPk5ifigXcpelKmj6iskP3c8QN6x6GjUSmYx+xUfs/GNVSU1XOZn61wgPDbgINJd/THGdqiO iJxCLuTMqlSsmh1+E1dSdfYkCb93R/0ZHvMKWlAx7MnaFgBfsG8FqNtZu3PCLfizyVYYjXbV WO1A23riZKqwrSJAATo5iTS65BuYxrFsFNPrf7TitM8E76BEBZk0OZBvZxMuOs6Z1qI8YKVK UrHVGFq3NbuPWCdRul9SX3VfOunr9Gv0GABnJ0ET+K7nspax0xqq7zgnM71QEaiaH17IFYGS sG34V7Wo3vyQzsk7qLf9Ajno0DhJ+VX43g8+AjxOMNVrGCt9RNXSBVpyv2AMTlWCdJ5KI6V4 KEzWM4HJm7QlNKE6RPoBxJVbSQLPd9St3h7mxLcne4l7NK9eNgNnneT7QZL8fL//s9K8Ns1W t60uQNYvbhKDG7+/yLcmJgjF74XkGvxCmTA1rW2bsUriM533nG9gAOUFQjURkwI8jvMAEQEA AYkCaAQYEQIACQUCVxvH8AIbAgIpCRBhV5kVtWN2DsFdIAQZAQIABgUCVxvH8AAKCRCH0Jac RAcHBIkHD/9nmfog7X2ZXMzL9ktT++7x+W/QBrSTCTmq8PK+69+INN1ZDOrY8uz6htfTLV9+ e2W6G8/7zIvODuHk7r+yQ585XbplgP0V5Xc8iBHdBgXbqnY5zBrcH+Q/oQ2STalEvaGHqNoD UGyLQ/fiKoLZTPMur57Fy1c9rTuKiSdMgnT0FPfWVDfpR2Ds0gpqWePlRuRGOoCln5GnREA/ 2MW2rWf+CO9kbIR+66j8b4RUJqIK3dWn9xbENh/aqxfonGTCZQ2zC4sLd25DQA4w1itPo+f5 V/SQxuhnlQkTOCdJ7b/mby/pNRz1lsLkjnXueLILj7gNjwTabZXYtL16z24qkDTI1x3g98R/ xunb3/fQwR8FY5/zRvXJq5us/nLvIvOmVwZFkwXc+AF+LSIajqQz9XbXeIP/BDjlBNXRZNdo dVuSU51ENcMcilPr2EUnqEAqeczsCGpnvRCLfVQeSZr2L9N4svNhhfPOEscYhhpHTh0VPyxI pPBNKq+byuYPMyk3nj814NKhImK0O4gTyCK9b+gZAVvQcYAXvSouCnTZeJRrNHJFTgTgu6E0 caxTGgc5zzQHeX67eMzrGomG3ZnIxmd1sAbgvJUDaD2GrYlulfwGWwWyTNbWRvMighVdPkSF 6XFgQaosWxkV0OELLy2N485YrTr2Uq64VKyxpncLh50e2RnyAJ9Za0Dx0yyp44iD1OvHtkEI M5kY0ACeNhCZJvZ5g4C2Lc9fcTHu8jxmEkI= Message-ID: <3ed6d5b9-f2ef-8bd1-f7b4-c4e1d8a540fd@gmail.com> Date: Tue, 5 Feb 2019 11:34:23 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <19716555-3522-cbdd-a128-e2ec672f89cd@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 2/5/19 8:57 AM, Eric Dumazet wrote: > > > On 02/05/2019 08:29 AM, Andrew wrote: >> Hi all. >> >> After upgrade on PPPoE BRAS to kernel 4.9.153 I've got an kernel panic after a 3 days of uptime. >> >> Unfortunately kernel is compiled w/o debug info; I rebuilt kernel with debug info enabled (kernel is compiled with same function addresses - I compare vmlinux symbol maps) - it says that panic is in net/ethernet/eth.c:88 >> >> Below there is a kernel panic trace. igb is from vendor, ver. 5.3.5.4. What extra info is needed? >> >> [263565.106441] BUG: unable to handle kernel paging request at ffff88015a4d2dd4 >> [263565.113527] IP: [] eth_header+0x3b/0xc0 >> [263565.119030] PGD 1e8f067 [263565.121474] PUD 0 >> [263565.123580] >> [263565.125166] Oops: 0002 [#1] SMP >> [263565.128398] Modules linked in: xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_filter xt_length xt_TCPMSS xt_tcpudp xt_mark xt_dscp iptable_mangle ip_tables x_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack sch_sfq sch_htb cls_u32 sch_ingress sch_prio sch_tbf cls_flow cls_fw act_police ifb 8021q mrp garp stp llc softdog pppoe pppox ppp_generic slhc i2c_nforce2 i2c_core igb(O) parport_pc dca parport thermal asus_atk0110 fan ptp k10temp hwmon pps_core nv_tco >> [263565.176083] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G           O    4.9.153-x86_64 #1 >> [263565.183996] Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010 >> [263565.195289] task: ffff88007d0f5200 task.stack: ffffc9000006c000 >> [263565.201295] RIP: 0010:[] [] eth_header+0x3b/0xc0 >> [263565.209225] RSP: 0018:ffff88007fa83c58  EFLAGS: 00010286 >> [263565.214622] RAX: ffff88015a4d2dc8 RBX: 0000000000000008 RCX: ffff8800682434a0 >> [263565.221843] RDX: ffff88015a4d2dc8 RSI: ffff88015a4d2dc8 RDI: ffff880077aab000 >> [263565.229062] RBP: ffff88007b663d90 R08: ffff88007b663d90 R09: 0000000000000574 >> [263565.236281] R10: ffff88007d1fa000 R11: 0000000000000000 R12: ffff8800682434a0 >> [263565.243501] R13: ffff88007d1fa000 R14: 0000000000000574 R15: 0000000000000008 >> [263565.250719] FS:  0000000000000000(0000) GS:ffff88007fa80000(0000) knlGS:0000000000000000 >> [263565.258894] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [263565.264725] CR2: ffff88015a4d2dd4 CR3: 000000007ad73000 CR4: 00000000000006f0 >> [263565.271944] Stack: >> [263565.274041]  ffff880077aab000 ffff880068243400 ffff88007a745000 ffff8800682434a0 >> [263565.281582]  0000000000000002 ffffffff81571d09 ffff880068243400 ffff88007fa83d00 >> [263565.289121]  ffff88007a745000 ffff880077aab000 ffff88007a712000 ffffffff815a8c61 >> [263565.296661] Call Trace: >> [263565.299193]  [263565.301205] [] ? neigh_connected_output+0xa9/0x100 >> [263565.307740]  [] ? ip_finish_output2+0x221/0x400 >> [263565.313920]  [] ? nf_iterate+0x54/0x60 >> [263565.319319]  [] ? ip_output+0x6a/0xf0 >> [263565.324631]  [] ? nf_iterate+0x12/0x60 >> [263565.330030]  [] ? ip_fragment.constprop.5+0x80/0x80 >> [263565.336556]  [] ? ip_forward+0x396/0x480 >> [263565.342128]  [] ? ip_check_defrag+0x1e0/0x1e0 >> [263565.348134]  [] ? ip_rcv+0x2ae/0x370 >> [263565.353361]  [] ? pppoe_rcv_core+0xd2/0x160 [pppoe] >> [263565.359888]  [] ? ip_local_deliver_finish+0x1d0/0x1d0 >> [263565.366586]  [] ? __netif_receive_skb_core+0x527/0xa80 >> [263565.373373]  [] ? process_backlog+0x92/0x130 >> [263565.379291]  [] ? net_rx_action+0x24d/0x390 >> [263565.385124]  [] ? __do_softirq+0xf4/0x2a0 >> [263565.390784]  [] ? irq_exit+0xbc/0xd0 >> [263565.396008]  [] ? call_function_single_interrupt+0x96/0xa0 >> [263565.403141]  [263565.405153] [] ? __sched_text_end+0x2/0x2 >> [263565.410907]  [] ? native_safe_halt+0x2/0x10 >> [263565.416741]  [] ? default_idle+0x18/0xd0 >> [263565.422314]  [] ? cpu_startup_entry+0x126/0x220 >> [263565.428492]  [] ? start_secondary+0x161/0x180 >> [263565.434496] Code: 0e 00 00 00 53 89 d3 49 89 cc 4c 89 c5 45 89 ce e8 bb 8a fc ff 66 83 fb 01 48 89 c6 74 44 66 83 fb 04 74 3e 66 c1 c3 08 48 85 ed <66> 89 58 0c 74 40 8b 45 00 4d 85 e4 89 46 06 0f b7 45 04 66 89 >> [263565.454534] RIP  [] eth_header+0x3b/0xc0 >> [263565.460124]  RSP >> [263565.463696] CR2: ffff88015a4d2dd4 >> [263565.467104] ---[ end trace a1bcaf3618724adf ]--- >> [263565.471807] Kernel panic - not syncing: Fatal exception in interrupt >> [263565.478245] Kernel Offset: disabled >> [263565.481818] Rebooting in 5 seconds.. >> > > > This is a well known issue, a fix should come shortly in stable branches Is Peter or yourself doing the backport? David would only take care of the most two recent stable kernels. Sorry about missing that change as part of the fragmenstack backport to 4.9... > > diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c > index f8bbd693c19c247e41839c2d0b5318ca51b23ee8..d95b32af4a0e3f552405c9e61cc372729834160c 100644 > --- a/net/ipv4/ip_fragment.c > +++ b/net/ipv4/ip_fragment.c > @@ -425,6 +425,7 @@ static int ip_frag_queue(struct ipq *qp, struct sk_buff *skb) > * fragment. > */ > > + err = -EINVAL; > /* Find out where to put this fragment. */ > prev_tail = qp->q.fragments_tail; > if (!prev_tail) > @@ -501,7 +502,6 @@ static int ip_frag_queue(struct ipq *qp, struct sk_buff *skb) > > discard_qp: > inet_frag_kill(&qp->q); > - err = -EINVAL; > __IP_INC_STATS(net, IPSTATS_MIB_REASM_OVERLAPS); > err: > kfree_skb(skb); > > > -- Florian