From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22D5CC282CB for ; Tue, 5 Feb 2019 20:13:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DBF86217D6 for ; Tue, 5 Feb 2019 20:13:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lgotvuj3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730239AbfBEUNK (ORCPT ); Tue, 5 Feb 2019 15:13:10 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:33480 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726654AbfBEUNJ (ORCPT ); Tue, 5 Feb 2019 15:13:09 -0500 Received: by mail-pl1-f195.google.com with SMTP id z23so2016276plo.0 for ; Tue, 05 Feb 2019 12:13:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=WkBIdwdxbaCoU04aHrPOUf/LrotT/cLtz83XFWxuD0M=; b=lgotvuj3cTZpeCmafDCrM2OWUb4roC3Q6EviC27cpyo68sM+Qh92IbFvxuM3Hv1QBQ OBGkvHrUG5YVQ0Ysf7Vy4ZrdlPeK/U1USxWV/Re5zBRYnvVs1mB9VAQ/TmX+5vkNZCJr gfoiM2o3jT9yPxKHudimV8Mj3Mkn7KVsV4HDqDK9XADuteYy6pZY5IFLdzk0xRDx23F/ C2oHiJfdqfQxFSzLa1axmcqT90NsEHMTi1T4hVC3vWYoXLEnu1i8tG9oFf9/cAzRvr0U lcXQRpJHCRKQaliGBv/MUkYcwmPMDulbs0ZE0YkIxzjw3/Y4kQsUjad/1oIT5Qy/bB3Q Yj0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=WkBIdwdxbaCoU04aHrPOUf/LrotT/cLtz83XFWxuD0M=; b=NQE6B4aZNtB+z1o6sA6jri3xvUO+0baXtshHJ1QYSd23M4LVFyY+LJ0q7n8B8Ntfss BrH69wyERYlQ+2lkURF8WL9yCjW5MF6XvyE3ieQ/16rzycZx19+veKMsPjVtZVNcDfvT YXdMNmoipGYAR8Rt8g4ad+u0sK+DthfDEpGjJIcG5tOJ4AndNEw5oS4wNDdWaMaQkimW XeiGATCZ0tjQct0iEKPvimjKgd4WUZOzzFWuAWJkSTpiRh3oTBKtyUoyrMg8tvwhsnW3 fFDnRgKtk9MD+c5DEu+XCUB9wzInOl77GefauN1muK06KOTf4046RdY7j0Rvnpfy5qxG XAZg== X-Gm-Message-State: AHQUAubAdp/IUaGanL/gLNfW9JPNAlgdBTnK17c6h8eIXZDQ7F+nZgD3 0jNlPB+NGjz8/vLnjME3sbX3+LWv X-Google-Smtp-Source: AHgI3IZyE1VHZJ/kazfAcCR+bkTmoHuhvkZGq2sEsWwtkqxsZk9KLa0bZtdnjIaQyKgGpsKfk2Xjcw== X-Received: by 2002:a17:902:981:: with SMTP id 1mr6782574pln.142.1549397588063; Tue, 05 Feb 2019 12:13:08 -0800 (PST) Received: from ?IPv6:2620:15c:2c1:200:55c7:81e6:c7d8:94b? ([2620:15c:2c1:200:55c7:81e6:c7d8:94b]) by smtp.gmail.com with ESMTPSA id 19sm9740814pfs.108.2019.02.05.12.13.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Feb 2019 12:13:07 -0800 (PST) Subject: Re: Kernel panic in eth_header To: Florian Fainelli , Eric Dumazet , Andrew , Netdev References: <18c17dde-5963-4412-2e98-ba44953f0ddd@seti.kr.ua> <19716555-3522-cbdd-a128-e2ec672f89cd@gmail.com> <3ed6d5b9-f2ef-8bd1-f7b4-c4e1d8a540fd@gmail.com> From: Eric Dumazet Message-ID: Date: Tue, 5 Feb 2019 12:13:06 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <3ed6d5b9-f2ef-8bd1-f7b4-c4e1d8a540fd@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 02/05/2019 11:34 AM, Florian Fainelli wrote: > On 2/5/19 8:57 AM, Eric Dumazet wrote: >> >> >> On 02/05/2019 08:29 AM, Andrew wrote: >>> Hi all. >>> >>> After upgrade on PPPoE BRAS to kernel 4.9.153 I've got an kernel panic after a 3 days of uptime. >>> >>> Unfortunately kernel is compiled w/o debug info; I rebuilt kernel with debug info enabled (kernel is compiled with same function addresses - I compare vmlinux symbol maps) - it says that panic is in net/ethernet/eth.c:88 >>> >>> Below there is a kernel panic trace. igb is from vendor, ver. 5.3.5.4. What extra info is needed? >>> >>> [263565.106441] BUG: unable to handle kernel paging request at ffff88015a4d2dd4 >>> [263565.113527] IP: [] eth_header+0x3b/0xc0 >>> [263565.119030] PGD 1e8f067 [263565.121474] PUD 0 >>> [263565.123580] >>> [263565.125166] Oops: 0002 [#1] SMP >>> [263565.128398] Modules linked in: xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_filter xt_length xt_TCPMSS xt_tcpudp xt_mark xt_dscp iptable_mangle ip_tables x_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre nf_nat nf_conntrack sch_sfq sch_htb cls_u32 sch_ingress sch_prio sch_tbf cls_flow cls_fw act_police ifb 8021q mrp garp stp llc softdog pppoe pppox ppp_generic slhc i2c_nforce2 i2c_core igb(O) parport_pc dca parport thermal asus_atk0110 fan ptp k10temp hwmon pps_core nv_tco >>> [263565.176083] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G           O    4.9.153-x86_64 #1 >>> [263565.183996] Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010 >>> [263565.195289] task: ffff88007d0f5200 task.stack: ffffc9000006c000 >>> [263565.201295] RIP: 0010:[] [] eth_header+0x3b/0xc0 >>> [263565.209225] RSP: 0018:ffff88007fa83c58  EFLAGS: 00010286 >>> [263565.214622] RAX: ffff88015a4d2dc8 RBX: 0000000000000008 RCX: ffff8800682434a0 >>> [263565.221843] RDX: ffff88015a4d2dc8 RSI: ffff88015a4d2dc8 RDI: ffff880077aab000 >>> [263565.229062] RBP: ffff88007b663d90 R08: ffff88007b663d90 R09: 0000000000000574 >>> [263565.236281] R10: ffff88007d1fa000 R11: 0000000000000000 R12: ffff8800682434a0 >>> [263565.243501] R13: ffff88007d1fa000 R14: 0000000000000574 R15: 0000000000000008 >>> [263565.250719] FS:  0000000000000000(0000) GS:ffff88007fa80000(0000) knlGS:0000000000000000 >>> [263565.258894] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> [263565.264725] CR2: ffff88015a4d2dd4 CR3: 000000007ad73000 CR4: 00000000000006f0 >>> [263565.271944] Stack: >>> [263565.274041]  ffff880077aab000 ffff880068243400 ffff88007a745000 ffff8800682434a0 >>> [263565.281582]  0000000000000002 ffffffff81571d09 ffff880068243400 ffff88007fa83d00 >>> [263565.289121]  ffff88007a745000 ffff880077aab000 ffff88007a712000 ffffffff815a8c61 >>> [263565.296661] Call Trace: >>> [263565.299193]  [263565.301205] [] ? neigh_connected_output+0xa9/0x100 >>> [263565.307740]  [] ? ip_finish_output2+0x221/0x400 >>> [263565.313920]  [] ? nf_iterate+0x54/0x60 >>> [263565.319319]  [] ? ip_output+0x6a/0xf0 >>> [263565.324631]  [] ? nf_iterate+0x12/0x60 >>> [263565.330030]  [] ? ip_fragment.constprop.5+0x80/0x80 >>> [263565.336556]  [] ? ip_forward+0x396/0x480 >>> [263565.342128]  [] ? ip_check_defrag+0x1e0/0x1e0 >>> [263565.348134]  [] ? ip_rcv+0x2ae/0x370 >>> [263565.353361]  [] ? pppoe_rcv_core+0xd2/0x160 [pppoe] >>> [263565.359888]  [] ? ip_local_deliver_finish+0x1d0/0x1d0 >>> [263565.366586]  [] ? __netif_receive_skb_core+0x527/0xa80 >>> [263565.373373]  [] ? process_backlog+0x92/0x130 >>> [263565.379291]  [] ? net_rx_action+0x24d/0x390 >>> [263565.385124]  [] ? __do_softirq+0xf4/0x2a0 >>> [263565.390784]  [] ? irq_exit+0xbc/0xd0 >>> [263565.396008]  [] ? call_function_single_interrupt+0x96/0xa0 >>> [263565.403141]  [263565.405153] [] ? __sched_text_end+0x2/0x2 >>> [263565.410907]  [] ? native_safe_halt+0x2/0x10 >>> [263565.416741]  [] ? default_idle+0x18/0xd0 >>> [263565.422314]  [] ? cpu_startup_entry+0x126/0x220 >>> [263565.428492]  [] ? start_secondary+0x161/0x180 >>> [263565.434496] Code: 0e 00 00 00 53 89 d3 49 89 cc 4c 89 c5 45 89 ce e8 bb 8a fc ff 66 83 fb 01 48 89 c6 74 44 66 83 fb 04 74 3e 66 c1 c3 08 48 85 ed <66> 89 58 0c 74 40 8b 45 00 4d 85 e4 89 46 06 0f b7 45 04 66 89 >>> [263565.454534] RIP  [] eth_header+0x3b/0xc0 >>> [263565.460124]  RSP >>> [263565.463696] CR2: ffff88015a4d2dd4 >>> [263565.467104] ---[ end trace a1bcaf3618724adf ]--- >>> [263565.471807] Kernel panic - not syncing: Fatal exception in interrupt >>> [263565.478245] Kernel Offset: disabled >>> [263565.481818] Rebooting in 5 seconds.. >>> >> >> >> This is a well known issue, a fix should come shortly in stable branches > > Is Peter or yourself doing the backport? David would only take care of > the most two recent stable kernels. > > Sorry about missing that change as part of the fragmenstack backport to > 4.9... Greg took care of this for the trees he manages.