Re: [BUG] mveta: mvneta_txq_bufs_free NULL pointer dereference

[Thomas Petazzoni <thomas.petazzoni@free-electrons.com>]

Hello,

+Grégory Clement, Antoine Ténart, Simon Guinot, Marcin Wojtas.

On Mon, 27 Nov 2017 09:47:10 +0100, Sean Nyekjær wrote:

> I see you are the maintainer on mvneta :-)
> I have an Espressobin board, i'm currently running with archlinux for
> arm. I have been running with 4.13.x mainline, 4.13.x with arch
> patches, 4.14.0 mainline and 4.14.1 with arch patches.
> 
> You can see what patches that are applied here:
> https://archlinuxarm.org/packages/aarch64/linux-espressobin/files
> 
> To the issue (same with all the kernels) :-)
> Every 10-14 days, sometimes faster the ethernet stops working. I have
> a serial debug connected so i can check the logs. The kernel logs
> contains nothing :-(
> If i then try to down the interface nasty things happen.

I'm not sure at all, but could you try to apply
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0d63785c6b94b5d2f095f90755825f90eea791f5
and see if the problem is resolved ?

I'm leaving the full kernel dump below for others to look at.

Thanks!

Thomas

> Here is my kernel dump:
> 
> [root@espressobin ~]# ip link set dev eth0 down
> [ 1339.493220] mvneta d0030000.ethernet eth0: TIMEOUT for TX stopped
> status=0x0000ffff
> [root@espressobin ~]#
> [ 1339.802218] br0: port 1(lan1) entered disabled state
> [ 1339.874861] br0: port 2(lan0) entered disabled state
> [ 1339.927740] alloc_contig_range: [7c198, 7c19d) PFNs busy
> [ 1339.933991] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
> [ 1339.974840] br0: bridge flag offload is not supported 1(lan1)
> [ 1340.033767] br0: bridge flag offload is not supported 2(lan0)
> [ 1340.532339] mv88e6085 d0032004.mdio-mii:01 lan1: Link is Down
> [ 1340.537957] mv88e6085 d0032004.mdio-mii:01 lan0: Link is Down
> [ 1341.012314] mvneta d0030000.ethernet eth0: Link is Up - 1Gbps/Full
> - flow control off
> [ 1341.020267] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [ 1341.035094] br0: port 1(lan1) entered blocking state
> [ 1341.040148] br0: port 1(lan1) entered forwarding state
> [ 1341.056721] br0: port 1(lan1) entered disabled state
> [ 1341.077884] br0: port 2(lan0) entered blocking state
> [ 1341.082795] br0: port 2(lan0) entered forwarding state
> [ 1341.091679] br0: port 2(lan0) entered disabled state
> [ 1341.107766] IPv6: ADDRCONF(NETDEV_UP): wan: link is not ready
> [ 1342.058742] mv88e6085 d0032004.mdio-mii:01 lan1: Link is Down
> [ 1342.143820] mv88e6085 d0032004.mdio-mii:01 lan0: Link is Down
> [ 1344.139466] mv88e6085 d0032004.mdio-mii:01 lan1: Link is Up -
> 100Mbps/Full - flow control off
> [ 1344.148358] br0: port 1(lan1) entered blocking state
> [ 1344.153309] br0: port 1(lan1) entered forwarding state
> [ 1344.202470] Unable to handle kernel NULL pointer dereference at
> virtual address 00000081
> [ 1344.210490] Mem abort info:
> [ 1344.213332]   Exception class = DABT (current EL), IL = 32 bits
> [ 1344.219833]   SET = 0, FnV = 0
> [ 1344.222466]   EA = 0, S1PTW = 0
> [ 1344.226149] Data abort info:
> [ 1344.229201]   ISV = 0, ISS = 0x00000006
> [ 1344.233099]   CM = 0, WnR = 0
> [ 1344.236131] user pgtable: 4k pages, 48-bit VAs, pgd = ffff80006ca11000
> [ 1344.242788] [0000000000000081] *pgd=000000006ca31003,
> *pud=000000006ca75003, *pmd=0000000000000000
> [ 1344.252122] Internal error: Oops: 96000006 [#1] SMP
> [ 1344.256794] Modules linked in: tun xt_nat veth ipt_MASQUERADE
> nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype
> iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter overlay
> bridge stp llc
> aes_ce_blk crypto_simd aes_ce_cipher crc32_ce crct10dif_ce ghash_ce
> aes_arm64 sha2_ce sha256_arm64 sha1_ce sch_fq_codel ip_tables ipv6
> [ 1344.293629] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.1-1-ARCH #1
> [ 1344.300542] Hardware name: Globalscale Marvell ESPRESSOBin Board (DT)
> [ 1344.307103] task: ffff000008d70580 task.stack: ffff000008d60000
> [ 1344.313311] PC is at mvneta_txq_bufs_free.isra.24+0x68/0x170
> [ 1344.318968] LR is at mvneta_txq_bufs_free.isra.24+0xd8/0x170
> [ 1344.324537] pc : [<ffff000008685198>] lr : [<ffff000008685208>]
> pstate: 80000145
> [ 1344.332089] sp : ffff000008003d10
> [ 1344.335684] x29: ffff000008003d10 x28: ffff000008d45000
> [ 1344.341164] x27: ffff800077cb0028 x26: 0000000000000003
> [ 1344.346737] x25: 0000000000000001 x24: ffff800077d36a00
> [ 1344.352219] x23: ffff800077d32938 x22: 0000000000000005
> [ 1344.357713] x21: ffff000008926110 x20: 00000000000000b6
> [ 1344.363281] x19: ffff800077cb0208 x18: 000000000000002e
> [ 1344.368947] x17: 0000ffff9127b8d0 x16: ffff0000080ce160
> [ 1344.374340] x15: 0000000000000008 x14: 0000000000000000
> [ 1344.379824] x13: 0000000000000001 x12: 0000000000000000
> [ 1344.384859] x11: 0000000000000000 x10: 0000000000000000
> [ 1344.390699] x9 : ffff000008d67000 x8 : 00000001000197df
> [ 1344.395825] x7 : 0000000000000000 x6 : 0000000000000000
> [ 1344.401663] x5 : 0000000000000001 x4 : 0000000000000000
> [ 1344.407234] x3 : ffff800078b3a000 x2 : ffff00000a43d060
> [ 1344.412715] x1 : 0000000000000003 x0 : 0000000000000003
> [ 1344.418110] Process swapper/0 (pid: 0, stack limit = 0xffff000008d60000)
> [ 1344.425026] Call trace:
> [ 1344.427277] Exception stack(0xffff000008003bd0 to 0xffff000008003d10)
> [ 1344.434372] 3bc0:
> 0000000000000003 0000000000000003
> [ 1344.442550] 3be0: ffff00000a43d060 ffff800078b3a000
> 0000000000000000 0000000000000001
> [ 1344.450374] 3c00: 0000000000000000 0000000000000000
> 00000001000197df ffff000008d67000
> [ 1344.458460] 3c20: 0000000000000000 0000000000000000
> 0000000000000000 0000000000000001
> [ 1344.466189] 3c40: 0000000000000000 0000000000000008
> ffff0000080ce160 0000ffff9127b8d0
> [ 1344.474633] 3c60: 000000000000002e ffff800077cb0208
> 00000000000000b6 ffff000008926110
> [ 1344.482808] 3c80: 0000000000000005 ffff800077d32938
> ffff800077d36a00 0000000000000001
> [ 1344.490805] 3ca0: 0000000000000003 ffff800077cb0028
> ffff000008d45000 ffff000008003d10
> [ 1344.498897] 3cc0: ffff000008685208 ffff000008003d10
> ffff000008685198 0000000080000145
> [ 1344.506896] 3ce0: ffff800077cb0208 00000000000000b6
> 0001000000000000 0000000000000005
> [ 1344.515255] 3d00: ffff000008003d10 ffff000008685198
> [ 1344.520207] [<ffff000008685198>] mvneta_txq_bufs_free.isra.24+0x68/0x170
> [ 1344.527142] [<ffff0000086873c8>] mvneta_poll+0x4f0/0xad8
> [ 1344.532528] [<ffff0000087d04fc>] net_rx_action+0x184/0x418
> [ 1344.538461] [<ffff000008081798>] __do_softirq+0x130/0x32c
> [ 1344.543594] [<ffff0000080cee58>] irq_exit+0xc8/0x100
> [ 1344.548812] [<ffff00000812a52c>] __handle_domain_irq+0x6c/0xc0
> [ 1344.554651] [<ffff000008081560>] gic_handle_irq+0x80/0x184
> [ 1344.560492] Exception stack(0xffff000008d63db0 to 0xffff000008d63ef0)
> [ 1344.567233] 3da0:
> ffff000008d45000 0000000000000000
> [ 1344.575317] 3dc0: ffff000008d63ef0 0000000000784718
> 0000800073275000 ffff000008d63f00
> [ 1344.583403] 3de0: 0000800073275000 0000000000000001
> ffff000008d70fe0 ffff000008d63e80
> [ 1344.591403] 3e00: 0000000000000a00 0000000000000000
> 0000000000000000 0000000000000001
> [ 1344.599666] 3e20: 0000000000000000 0000000000000008
> ffff0000080ce160 0000ffff9127b8d0
> [ 1344.607843] 3e40: 000000000000002e ffff000008d45000
> ffff000008d69000 ffff000008d69000
> [ 1344.615839] 3e60: ffff000008d4f148 ffff000008d69bec
> 0000000000000000 0000000000000000
> [ 1344.623836] 3e80: ffff000008d70580 000000007ff963f8
> 0000000000c80018 ffff000008d63ef0
> [ 1344.631922] 3ea0: ffff00000808521c ffff000008d63ef0
> ffff000008085220 0000000000000145
> [ 1344.640097] 3ec0: ffff80007bfffb00 ffff000008cea028
> ffffffffffffffff 0000000000000000
> [ 1344.648181] 3ee0: ffff000008d63ef0 ffff000008085220
> [ 1344.653037] [<ffff000008082fb0>] el1_irq+0xb0/0x140
> [ 1344.657891] [<ffff000008085220>] arch_cpu_idle+0x30/0x188
> [ 1344.663911] [<ffff00000810eed0>] do_idle+0x128/0x1e8
> [ 1344.669041] [<ffff00000810f13c>] cpu_startup_entry+0x2c/0x30
> [ 1344.674972] [<ffff00000890516c>] rest_init+0xb4/0xc0
> [ 1344.679945] [<ffff000008c80cf0>] start_kernel+0x394/0x3a8
> [ 1344.685594] Code: 93407c01 8b011442 f8617879 b4000079 (b9408321)
> [ 1344.691523] ---[ end trace 0e5abdfc76ee83e5 ]---
> [ 1344.696733] Kernel panic - not syncing: Fatal exception in interrupt
> [ 1344.703310] SMP: stopping secondary CPUs
> [ 1344.707369] Kernel Offset: disabled
> [ 1344.710613] CPU features: 0x002008
> [ 1344.714294] Memory Limit: none
> [ 1344.717086] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
> 
> I you want more logs or some other details about my setup i'll be
> happy to help :-)
> Also with testing a possible fix.
> 
> Thanks,
> Sean Nyekjaer



-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com