All of lore.kernel.org
 help / color / mirror / Atom feed
* fake rtable dst patch applied but kernel keeps panicing
@ 2012-04-18 10:08 Massimo Cetra
  2012-04-18 10:31 ` Eric Dumazet
  2012-04-19  8:22 ` Peter Huang(Peng)
  0 siblings, 2 replies; 10+ messages in thread
From: Massimo Cetra @ 2012-04-18 10:08 UTC (permalink / raw)
  To: netdev; +Cc: Eric Dumazet, peter.huangpeng

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

Hi Eric,
Hi Peter,

I applied the 1st patch by Peter to a 3.2.14 kernel, compiled, installed 
and rebooted.

I have had another panic and i'm attaching the oops as i received it 
through netcnsole.

I don't think the patch is enough to prevent the OOPS.

Can i be helpful in some way ?

Thanks,
  Massimo

[-- Attachment #2: BUG3.txt --]
[-- Type: text/plain, Size: 17253 bytes --]

Apr 18 11:53:31 lamu [ 1749.735232] BUG: unable to handle kernel 
Apr 18 11:53:31 NULL pointer dereference
Apr 18 11:53:31 lamu  at 0000000000000018
Apr 18 11:53:31 lamu [ 1749.750893] IP:
Apr 18 11:53:31 lamu  [<ffffffffa02d233d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 11:53:31 lamu [ 1749.764978] PGD 0 
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1749.768992] Oops: 0000 [#1] 
Apr 18 11:53:31 SMP  
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1749.775450] CPU 0 
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1749.779105] Modules linked in:
Apr 18 11:53:31 lamu  ipt_MASQUERADE
Apr 18 11:53:31 lamu  iptable_nat
Apr 18 11:53:31 lamu  nf_nat
Apr 18 11:53:31 lamu  nf_conntrack_ipv4
Apr 18 11:53:31 lamu  nf_defrag_ipv4
Apr 18 11:53:31 lamu  ip_vs_rr
Apr 18 11:53:31 lamu  ip_vs
Apr 18 11:53:31 lamu  nf_conntrack
Apr 18 11:53:31 lamu  libcrc32c
Apr 18 11:53:31 lamu  ip6table_filter
Apr 18 11:53:31 lamu  ip6_tables
Apr 18 11:53:31 lamu  iptable_filter
Apr 18 11:53:31 lamu  ip_tables
Apr 18 11:53:31 lamu  ebtable_nat
Apr 18 11:53:31 lamu  ebtables
Apr 18 11:53:31 lamu  x_tables
Apr 18 11:53:31 lamu  crc32c
Apr 18 11:53:31 lamu  drbd
Apr 18 11:53:31 lamu  lru_cache
Apr 18 11:53:31 lamu  cn
Apr 18 11:53:31 lamu  sit
Apr 18 11:53:31 lamu  tunnel4
Apr 18 11:53:31 lamu  tun
Apr 18 11:53:31 lamu  bridge
Apr 18 11:53:31 lamu  stp
Apr 18 11:53:31 lamu  virtio_net
Apr 18 11:53:31 lamu  virtio_blk
Apr 18 11:53:31 lamu  virtio_rng
Apr 18 11:53:31 lamu  rng_core
Apr 18 11:53:31 lamu  virtio_pci
Apr 18 11:53:31 lamu  virtio_ring
Apr 18 11:53:31 lamu  virtio
Apr 18 11:53:31 lamu  kvm_intel
Apr 18 11:53:31 lamu  kvm
Apr 18 11:53:31 lamu  ipmi_devintf
Apr 18 11:53:31 lamu  ipmi_poweroff
Apr 18 11:53:31 lamu  ipmi_si
Apr 18 11:53:31 lamu  ipmi_watchdog
Apr 18 11:53:31 lamu  ipmi_msghandler
Apr 18 11:53:31 lamu  netconsole
Apr 18 11:53:31 lamu  configfs
Apr 18 11:53:31 lamu  loop
Apr 18 11:53:31 lamu  snd_pcm
Apr 18 11:53:31 lamu  snd_page_alloc
Apr 18 11:53:31 lamu  option
Apr 18 11:53:31 lamu  snd_timer
Apr 18 11:53:31 lamu  snd
Apr 18 11:53:31 lamu  usb_wwan
Apr 18 11:53:31 lamu  usbserial
Apr 18 11:53:31 lamu  iTCO_wdt
Apr 18 11:53:31 lamu  i7core_edac
Apr 18 11:53:31 lamu  edac_core
Apr 18 11:53:31 lamu  processor
Apr 18 11:53:31 lamu  joydev
Apr 18 11:53:31 lamu  soundcore
Apr 18 11:53:31 lamu  iTCO_vendor_support
Apr 18 11:53:31 lamu  psmouse
Apr 18 11:53:31 lamu  button
Apr 18 11:53:31 lamu  dcdbas
Apr 18 11:53:31 lamu  serio_raw
Apr 18 11:53:31 lamu  evdev
Apr 18 11:53:31 lamu  pcspkr
Apr 18 11:53:31 lamu  thermal_sys
Apr 18 11:53:31 lamu  ext3
Apr 18 11:53:31 lamu  mbcache
Apr 18 11:53:31 lamu  jbd
Apr 18 11:53:31 lamu  dm_mod
Apr 18 11:53:31 lamu  sr_mod
Apr 18 11:53:31 lamu  cdrom
Apr 18 11:53:31 lamu  ses
Apr 18 11:53:31 lamu  sd_mod
Apr 18 11:53:31 lamu  usbhid
Apr 18 11:53:31 lamu  hid
Apr 18 11:53:31 lamu  usb_storage
Apr 18 11:53:31 lamu  uas
Apr 18 11:53:31 lamu  crc_t10dif
Apr 18 11:53:31 lamu  enclosure
Apr 18 11:53:31 lamu  ata_generic
Apr 18 11:53:31 lamu  uhci_hcd
Apr 18 11:53:31 lamu  ata_piix
Apr 18 11:53:31 lamu  ehci_hcd
Apr 18 11:53:31 lamu  libata
Apr 18 11:53:31 lamu  usbcore
Apr 18 11:53:31 lamu  usb_common
Apr 18 11:53:31 lamu  megaraid_sas
Apr 18 11:53:31 lamu  scsi_mod
Apr 18 11:53:31 lamu  bnx2
Apr 18 11:53:31 lamu  [last unloaded: scsi_wait_scan]
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1749.931103] 
Apr 18 11:53:31 lamu [ 1749.934064] Pid: 5936, comm: kvm Not tainted 3.2.0-2-amd64 #1
Apr 18 11:53:31 lamu  Dell Inc. PowerEdge R410
Apr 18 11:53:31 lamu /0N051F 
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1749.951108] RIP: 0010:[<ffffffffa02d233d>] 
Apr 18 11:53:31 lamu  [<ffffffffa02d233d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 11:53:31 lamu [ 1749.970058] RSP: 0018:ffff88042fc03b18  EFLAGS: 00010293
Apr 18 11:53:31 lamu [ 1749.980667] RAX: 0000000000000000 RBX: ffff880419b10880 RCX: 0000000100058883
Apr 18 11:53:31 lamu [ 1749.994915] RDX: ffffffffa02d230f RSI: 0000000000000282 RDI: ffff880419b10880
Apr 18 11:53:31 lamu [ 1750.009164] RBP: ffff880225216000 R08: 0000000000000000 R09: ffff88042fc03ad0
Apr 18 11:53:31 lamu [ 1750.023414] R10: ffffffff8165aac0 R11: ffffffff8165aac0 R12: 0000000000000000
Apr 18 11:53:31 lamu [ 1750.037662] R13: ffff880225afc002 R14: ffff8803ec883c00 R15: ffff880225afc000
Apr 18 11:53:31 lamu [ 1750.051911] FS:  00007f4355786900(0000) GS:ffff88042fc00000(0000) knlGS:0000000000000000
Apr 18 11:53:31 lamu [ 1750.068085] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 18 11:53:31 lamu [ 1750.079560] CR2: 0000000000000018 CR3: 0000000417220000 CR4: 00000000000026e0
Apr 18 11:53:31 lamu [ 1750.093809] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 18 11:53:31 lamu [ 1750.108060] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr 18 11:53:31 lamu [ 1750.122309] Process kvm (pid: 5936, threadinfo ffff88041b750000, task ffff880419bce930)
Apr 18 11:53:31 lamu [ 1750.138306] Stack:
Apr 18 11:53:31 lamu [ 1750.142323]  ffffffff80000000
Apr 18 11:53:31 lamu  ffffffffa02d2714
Apr 18 11:53:31 lamu  ffff880419b10880
Apr 18 11:53:31 lamu  ffff880225216000
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1750.157156]  ffff8802279a5000
Apr 18 11:53:31 lamu  ffffffffa02d2aa0
Apr 18 11:53:31 lamu  ffff880380000000
Apr 18 11:53:31 lamu  0000000225216740
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1750.171986]  ffff880419b10880
Apr 18 11:53:31 lamu  ffffffffa02d8cd0
Apr 18 11:53:31 lamu  ffffffff81691190
Apr 18 11:53:31 lamu  0000000000000002
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1750.186819] Call Trace:
Apr 18 11:53:31 lamu [ 1750.191703]  <IRQ> 
Apr 18 11:53:31 lamu  
Apr 18 11:53:31 lamu [ 1750.195909]  [<ffffffffa02d2714>] ? br_parse_ip_options+0x3d/0x19a [bridge]
Apr 18 11:53:31 lamu [ 1750.209816]  [<ffffffffa02d2aa0>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
Apr 18 11:53:31 lamu [ 1750.223375]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 11:53:31 lamu [ 1750.233986]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:31 lamu [ 1750.246504]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:31 lamu [ 1750.259020]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 11:53:31 lamu [ 1750.270152]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:31 lamu [ 1750.282670]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:31 lamu [ 1750.296403]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:31 lamu [ 1750.308921]  [<ffffffffa02cd887>] ? NF_HOOK.constprop.10+0x3c/0x56 [bridge]
Apr 18 11:53:31 ataru ntpd[14048]: Listen normally on 56 br0:0 77.238.6.181 UDP 123
Apr 18 11:53:31 ataru ntpd[14048]: Listen normally on 57 br1:0 172.16.1.39 UDP 123
Apr 18 11:53:31 ataru ntpd[14048]: peers refreshed
Apr 18 11:53:31 lamu [ 1750.322826]  [<ffffffffa02cda1b>] ? br_forward+0x16/0x5a [bridge]
Apr 18 11:53:31 lamu [ 1750.334998]  [<ffffffffa02ce543>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
Apr 18 11:53:31 lamu [ 1750.349613]  [<ffffffffa02d2638>] ? br_nf_pre_routing_finish+0x1ee/0x1fb [bridge]
Apr 18 11:53:31 lamu [ 1750.364575]  [<ffffffffa02d1ff7>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
Apr 18 11:53:31 lamu [ 1750.377440]  [<ffffffffa02d2f91>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
Apr 18 11:53:31 lamu [ 1750.391171]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 11:53:31 lamu [ 1750.401783]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:31 lamu [ 1750.415514]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 11:53:31 lamu [ 1750.426644]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:31 lamu [ 1750.440375]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1750.454106]  [<ffffffffa02ce388>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1750.467839]  [<ffffffff810135ad>] ? paravirt_read_tsc+0x5/0x8
Apr 18 11:53:32 lamu [ 1750.479314]  [<ffffffff81013622>] ? read_tsc+0x5/0x14
Apr 18 11:53:32 lamu [ 1750.489403]  [<ffffffffa02ce764>] ? br_handle_frame+0x1b3/0x1cb [bridge]
Apr 18 11:53:32 lamu [ 1750.502788]  [<ffffffffa02ce5b1>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
Apr 18 11:53:32 lamu [ 1750.517406]  [<ffffffff81289222>] ? __netif_receive_skb+0x324/0x41f
Apr 18 11:53:32 lamu [ 1750.529922]  [<ffffffff81289389>] ? process_backlog+0x6c/0x123
Apr 18 11:53:32 lamu [ 1750.541573]  [<ffffffff8128b26f>] ? net_rx_action+0xa1/0x1af
Apr 18 11:53:32 lamu [ 1750.552878]  [<ffffffff81036faf>] ? test_tsk_need_resched+0xa/0x13
Apr 18 11:53:32 lamu [ 1750.565222]  [<ffffffff8104be34>] ? __do_softirq+0xb9/0x177
Apr 18 11:53:32 lamu [ 1750.576355]  [<ffffffff813503ec>] ? call_softirq+0x1c/0x30
Apr 18 11:53:32 lamu [ 1750.587307]  <EOI> 
Apr 18 11:53:32 lamu  
Apr 18 11:53:32 lamu [ 1750.591512]  [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
Apr 18 11:53:32 lamu [ 1750.602120]  [<ffffffff8128b55f>] ? netif_rx_ni+0x1e/0x27
Apr 18 11:53:32 lamu [ 1750.612904]  [<ffffffffa0309721>] ? tun_get_user+0x39a/0x3c2 [tun]
Apr 18 11:53:32 lamu [ 1750.625249]  [<ffffffffa0309a66>] ? tun_chr_poll+0xcd/0xcd [tun]
Apr 18 11:53:32 lamu [ 1750.637245]  [<ffffffffa0309ac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
Apr 18 11:53:32 lamu [ 1750.650111]  [<ffffffff810f95e4>] ? do_sync_readv_writev+0x9a/0xd7
Apr 18 11:53:32 lamu [ 1750.662455]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 11:53:32 lamu [ 1750.673583]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 11:53:32 lamu [ 1750.684718]  [<ffffffff81162649>] ? security_file_permission+0x16/0x2d
Apr 18 11:53:32 lamu [ 1750.697755]  [<ffffffff810f9848>] ? do_readv_writev+0xaf/0x11c
Apr 18 11:53:32 lamu [ 1750.709407]  [<ffffffff8112abce>] ? eventfd_ctx_read+0x162/0x174
Apr 18 11:53:32 lamu [ 1750.721405]  [<ffffffff8103f403>] ? try_to_wake_up+0x197/0x197
Apr 18 11:53:32 lamu [ 1750.733054]  [<ffffffff810f9a1d>] ? sys_writev+0x45/0x90
Apr 18 11:53:32 lamu [ 1750.743664]  [<ffffffff8134e192>] ? system_call_fastpath+0x16/0x1b
Apr 18 11:53:32 lamu [ 1750.756004] Code: 
Apr 18 11:53:32 53  
Apr 18 11:53:32 48  
Apr 18 11:53:32 89  
Apr 18 11:53:32 fb  
Apr 18 11:53:32 48  
Apr 18 11:53:32 83  
Apr 18 11:53:32 ec  
Apr 18 11:53:32 10  
Apr 18 11:53:32 66  
Apr 18 11:53:32 81  
Apr 18 11:53:32 7f  
Apr 18 11:53:32 7e  
Apr 18 11:53:32 08  
Apr 18 11:53:32 06  
Apr 18 11:53:32 4c  
Apr 18 11:53:32 8b  
Apr 18 11:53:32 a7  
Apr 18 11:53:32 98  
Apr 18 11:53:32 00  
Apr 18 11:53:32 00  
Apr 18 11:53:32 00  
Apr 18 11:53:32 74  
Apr 18 11:53:32 3d  
Apr 18 11:53:32 e8  
Apr 18 11:53:32 07  
Apr 18 11:53:32 fe  
Apr 18 11:53:32 ff  
Apr 18 11:53:32 ff  
Apr 18 11:53:32 66  
Apr 18 11:53:32 3d  
Apr 18 11:53:32 08  
Apr 18 11:53:32 06  
Apr 18 11:53:32 75  
Apr 18 11:53:32 09  
Apr 18 11:53:32 83  
Apr 18 11:53:32 3d  
Apr 18 11:53:32 91  
Apr 18 11:53:32 6a  
Apr 18 11:53:32 00  
Apr 18 11:53:32 00  
Apr 18 11:53:32 00  
Apr 18 11:53:32 75  
Apr 18 11:53:32 29  
Apr 18 11:53:32 lamu  
Apr 18 11:53:32 f6  
Apr 18 11:53:32 44  
Apr 18 11:53:32 24  
Apr 18 11:53:32 18  
Apr 18 11:53:32 01  
Apr 18 11:53:32 49  
Apr 18 11:53:32 8b  
Apr 18 11:53:32 6c  
Apr 18 11:53:32 24  
Apr 18 11:53:32 08  
Apr 18 11:53:32 74  
Apr 18 11:53:32 12  
Apr 18 11:53:32 8a  
Apr 18 11:53:32 43  
Apr 18 11:53:32 7d  
Apr 18 11:53:32 83  
Apr 18 11:53:32 e0  
Apr 18 11:53:32 f8  
Apr 18 11:53:32 83  
Apr 18 11:53:32 c8  
Apr 18 11:53:32 lamu  
Apr 18 11:53:32 lamu [ 1750.794643] RIP 
Apr 18 11:53:32 lamu  [<ffffffffa02d233d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 11:53:32 lamu [ 1750.808909]  RSP <ffff88042fc03b18>
Apr 18 11:53:32 lamu [ 1750.815875] CR2: 0000000000000018
Apr 18 11:53:32 lamu [ 1750.823229] ---[ end trace edf4c83afa62cdb9 ]---
Apr 18 11:53:32 lamu [ 1750.832693] Kernel panic - not syncing: Fatal exception in interrupt
Apr 18 11:53:32 lamu [ 1750.845601] Pid: 5936, comm: kvm Tainted: G      D      3.2.0-2-amd64 #1
Apr 18 11:53:32 lamu [ 1750.859399] Call Trace:
Apr 18 11:53:32 lamu [ 1750.864741]  <IRQ> 
Apr 18 11:53:32 lamu  [<ffffffff81342aa0>] ? panic+0x95/0x1a5
Apr 18 11:53:32 lamu [ 1750.876729]  [<ffffffff8134a006>] ? oops_end+0xa9/0xb6
Apr 18 11:53:32 lamu [ 1750.887462]  [<ffffffff813423ec>] ? no_context+0x1ff/0x20e
Apr 18 11:53:32 lamu [ 1750.898989]  [<ffffffff81052266>] ? __mod_timer+0x139/0x14b
Apr 18 11:53:32 lamu [ 1750.910619]  [<ffffffff8134c019>] ? do_page_fault+0x1a8/0x337
Apr 18 11:53:32 lamu [ 1750.922490]  [<ffffffffa03d9f06>] ? ip_vs_conn_put+0x28/0x32 [ip_vs]
Apr 18 11:53:32 lamu [ 1750.935665]  [<ffffffffa03dc0e0>] ? ip_vs_out+0x2bd/0x432 [ip_vs]
Apr 18 11:53:32 lamu [ 1750.948343]  [<ffffffffa02cd847>] ? br_dev_queue_push_xmit+0x9b/0x9f [bridge]
Apr 18 11:53:32 lamu [ 1750.963100]  [<ffffffff81349775>] ? page_fault+0x25/0x30
Apr 18 11:53:32 lamu [ 1750.974217]  [<ffffffffa02d230f>] ? nf_bridge_update_protocol+0x20/0x20 [bridge]
Apr 18 11:53:32 lamu [ 1750.989475]  [<ffffffffa02d233d>] ? br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 11:53:32 lamu [ 1751.003874]  [<ffffffffa02d232e>] ? br_nf_forward_finish+0x1f/0x95 [bridge]
Apr 18 11:53:32 lamu [ 1751.018288]  [<ffffffffa02d2714>] ? br_parse_ip_options+0x3d/0x19a [bridge]
Apr 18 11:53:32 lamu [ 1751.032665]  [<ffffffffa02d2aa0>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
Apr 18 11:53:32 lamu [ 1751.046726]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 11:53:32 lamu [ 1751.057748]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:32 lamu [ 1751.070699]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:32 lamu [ 1751.083635]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 11:53:32 lamu [ 1751.095178]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:32 lamu [ 1751.108133]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.122284]  [<ffffffffa02cd941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 11:53:32 lamu [ 1751.135191]  [<ffffffffa02cd887>] ? NF_HOOK.constprop.10+0x3c/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.149495]  [<ffffffffa02cda1b>] ? br_forward+0x16/0x5a [bridge]
Apr 18 11:53:32 lamu [ 1751.162178]  [<ffffffffa02ce543>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
Apr 18 11:53:32 lamu [ 1751.177209]  [<ffffffffa02d2638>] ? br_nf_pre_routing_finish+0x1ee/0x1fb [bridge]
Apr 18 11:53:32 lamu [ 1751.192580]  [<ffffffffa02d1ff7>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
Apr 18 11:53:32 lamu [ 1751.205860]  [<ffffffffa02d2f91>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
Apr 18 11:53:32 lamu [ 1751.219995]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 11:53:32 lamu [ 1751.231061]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.245296]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 11:53:32 lamu [ 1751.256882]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.271011]  [<ffffffffa02ce3a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.285168]  [<ffffffffa02ce388>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
Apr 18 11:53:32 lamu [ 1751.299280]  [<ffffffff810135ad>] ? paravirt_read_tsc+0x5/0x8
Apr 18 11:53:32 lamu [ 1751.311154]  [<ffffffff81013622>] ? read_tsc+0x5/0x14
Apr 18 11:53:32 lamu [ 1751.321748]  [<ffffffffa02ce764>] ? br_handle_frame+0x1b3/0x1cb [bridge]
Apr 18 11:53:32 lamu [ 1751.335512]  [<ffffffffa02ce5b1>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
Apr 18 11:53:32 lamu [ 1751.350390]  [<ffffffff81289222>] ? __netif_receive_skb+0x324/0x41f
Apr 18 11:53:32 lamu [ 1751.363287]  [<ffffffff81289389>] ? process_backlog+0x6c/0x123
Apr 18 11:53:32 lamu [ 1751.375344]  [<ffffffff8128b26f>] ? net_rx_action+0xa1/0x1af
Apr 18 11:53:32 lamu [ 1751.387056]  [<ffffffff81036faf>] ? test_tsk_need_resched+0xa/0x13
Apr 18 11:53:32 lamu [ 1751.399819]  [<ffffffff8104be34>] ? __do_softirq+0xb9/0x177
Apr 18 11:53:32 lamu [ 1751.411354]  [<ffffffff813503ec>] ? call_softirq+0x1c/0x30
Apr 18 11:53:32 lamu [ 1751.422774]  <EOI> 
Apr 18 11:53:32 lamu  [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
Apr 18 11:53:32 lamu [ 1751.435471]  [<ffffffff8128b55f>] ? netif_rx_ni+0x1e/0x27
Apr 18 11:53:32 lamu [ 1751.446679]  [<ffffffffa0309721>] ? tun_get_user+0x39a/0x3c2 [tun]
Apr 18 11:53:33 lamu [ 1751.459426]  [<ffffffffa0309a66>] ? tun_chr_poll+0xcd/0xcd [tun]
Apr 18 11:53:33 lamu [ 1751.471861]  [<ffffffffa0309ac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
Apr 18 11:53:33 lamu [ 1751.485116]  [<ffffffff810f95e4>] ? do_sync_readv_writev+0x9a/0xd7
Apr 18 11:53:33 lamu [ 1751.497903]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 11:53:33 lamu [ 1751.509414]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 11:53:33 lamu [ 1751.520933]  [<ffffffff81162649>] ? security_file_permission+0x16/0x2d
Apr 18 11:53:33 lamu [ 1751.534358]  [<ffffffff810f9848>] ? do_readv_writev+0xaf/0x11c
Apr 18 11:53:33 lamu [ 1751.546397]  [<ffffffff8112abce>] ? eventfd_ctx_read+0x162/0x174
Apr 18 11:53:33 lamu [ 1751.558815]  [<ffffffff8103f403>] ? try_to_wake_up+0x197/0x197
Apr 18 11:53:33 lamu [ 1751.570869]  [<ffffffff810f9a1d>] ? sys_writev+0x45/0x90
Apr 18 11:53:33 lamu [ 1751.581979]  [<ffffffff8134e192>] ? system_call_fastpath+0x16/0x1b

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-18 10:08 fake rtable dst patch applied but kernel keeps panicing Massimo Cetra
@ 2012-04-18 10:31 ` Eric Dumazet
  2012-04-19  8:01   ` Massimo Cetra
  2012-04-19  8:22 ` Peter Huang(Peng)
  1 sibling, 1 reply; 10+ messages in thread
From: Eric Dumazet @ 2012-04-18 10:31 UTC (permalink / raw)
  To: Massimo Cetra; +Cc: netdev, peter.huangpeng

On Wed, 2012-04-18 at 12:08 +0200, Massimo Cetra wrote:
> Hi Eric,
> Hi Peter,
> 
> I applied the 1st patch by Peter to a 3.2.14 kernel, compiled, installed 
> and rebooted.
> 
> I have had another panic and i'm attaching the oops as i received it 
> through netcnsole.
> 
> I don't think the patch is enough to prevent the OOPS.
> 
> Can i be helpful in some way ?
> 
> Thanks,
>   Massimo

Seems a different issue, skb->nf_bridge seems to be NULL

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-18 10:31 ` Eric Dumazet
@ 2012-04-19  8:01   ` Massimo Cetra
  2012-04-19  8:14     ` Eric Dumazet
  0 siblings, 1 reply; 10+ messages in thread
From: Massimo Cetra @ 2012-04-19  8:01 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Massimo Cetra, netdev, peter.huangpeng

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

On 18/04/2012 12:31, Eric Dumazet wrote:
>
> Seems a different issue, skb->nf_bridge seems to be NULL
>

This is another trace of another panic.
I hope it may be useful.
Please notice that it is not related to adding/removing br interfaces or 
tun/vic interfaces from a bridge set.
It happened suddenly during a normal workload.

When were those bridge-related bugs introduced ?
What's the latest release that seems to work ?
I'm asking so that i can restore some servers to a proper workload.


Massimo

[-- Attachment #2: BUG4.txt --]
[-- Type: text/plain, Size: 25550 bytes --]

Apr 18 16:22:05 lamu [15868.069196] BUG: unable to handle kernel 
Apr 18 16:22:05 NULL pointer dereference
Apr 18 16:22:05 lamu  at 0000000000000018
Apr 18 16:22:05 lamu [15868.084886] IP:
Apr 18 16:22:05 lamu  [<ffffffffa02dc33d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 16:22:05 lamu [15868.098984] PGD 0 
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.103014] Oops: 0000 [#1] 
Apr 18 16:22:05 SMP  
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.109487] CPU 0 
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.113142] Modules linked in:
Apr 18 16:22:05 lamu  option
Apr 18 16:22:05 lamu  usb_wwan
Apr 18 16:22:05 lamu  usbserial
Apr 18 16:22:05 lamu  uas
Apr 18 16:22:05 lamu  usb_storage
Apr 18 16:22:05 lamu  ipt_MASQUERADE
Apr 18 16:22:05 lamu  iptable_nat
Apr 18 16:22:05 lamu  nf_nat
Apr 18 16:22:05 lamu  nf_conntrack_ipv4
Apr 18 16:22:05 lamu  nf_defrag_ipv4
Apr 18 16:22:05 lamu  ip_vs_rr
Apr 18 16:22:05 lamu  ip_vs
Apr 18 16:22:05 lamu  nf_conntrack
Apr 18 16:22:05 lamu  libcrc32c
Apr 18 16:22:05 lamu  ip6table_filter
Apr 18 16:22:05 lamu  ip6_tables
Apr 18 16:22:05 lamu  iptable_filter
Apr 18 16:22:05 lamu  ip_tables
Apr 18 16:22:05 lamu  ebtable_nat
Apr 18 16:22:05 lamu  ebtables
Apr 18 16:22:05 lamu  x_tables
Apr 18 16:22:05 lamu  crc32c
Apr 18 16:22:05 lamu  drbd
Apr 18 16:22:05 lamu  lru_cache
Apr 18 16:22:05 lamu  cn
Apr 18 16:22:05 lamu  sit
Apr 18 16:22:05 lamu  tunnel4
Apr 18 16:22:05 lamu  tun
Apr 18 16:22:05 lamu  bridge
Apr 18 16:22:05 lamu  stp
Apr 18 16:22:05 lamu  virtio_net
Apr 18 16:22:05 lamu  virtio_blk
Apr 18 16:22:05 lamu  virtio_rng
Apr 18 16:22:05 lamu  rng_core
Apr 18 16:22:05 lamu  virtio_pci
Apr 18 16:22:05 lamu  virtio_ring
Apr 18 16:22:05 lamu  virtio
Apr 18 16:22:05 lamu  kvm_intel
Apr 18 16:22:05 lamu  kvm
Apr 18 16:22:05 lamu  ipmi_devintf
Apr 18 16:22:05 lamu  ipmi_poweroff
Apr 18 16:22:05 lamu  ipmi_si
Apr 18 16:22:05 lamu  ipmi_watchdog
Apr 18 16:22:05 lamu  ipmi_msghandler
Apr 18 16:22:05 lamu  netconsole
Apr 18 16:22:05 lamu  configfs
Apr 18 16:22:05 lamu  loop
Apr 18 16:22:05 lamu  snd_pcm
Apr 18 16:22:05 lamu  snd_page_alloc
Apr 18 16:22:05 lamu  snd_timer
Apr 18 16:22:05 lamu  snd
Apr 18 16:22:05 lamu  dcdbas
Apr 18 16:22:05 lamu  iTCO_wdt
Apr 18 16:22:05 lamu  soundcore
Apr 18 16:22:05 lamu  i7core_edac
Apr 18 16:22:05 lamu  edac_core
Apr 18 16:22:05 lamu  joydev
Apr 18 16:22:05 lamu  iTCO_vendor_support
Apr 18 16:22:05 lamu  psmouse
Apr 18 16:22:05 lamu  evdev
Apr 18 16:22:05 lamu  pcspkr
Apr 18 16:22:05 lamu  serio_raw
Apr 18 16:22:05 lamu  processor
Apr 18 16:22:05 lamu  button
Apr 18 16:22:05 lamu  thermal_sys
Apr 18 16:22:05 lamu  ext3
Apr 18 16:22:05 lamu  mbcache
Apr 18 16:22:05 lamu  jbd
Apr 18 16:22:05 lamu  dm_mod
Apr 18 16:22:05 lamu  usbhid
Apr 18 16:22:05 lamu  sr_mod
Apr 18 16:22:05 lamu  hid
Apr 18 16:22:05 lamu  cdrom
Apr 18 16:22:05 lamu  ses
Apr 18 16:22:05 lamu  sd_mod
Apr 18 16:22:05 lamu  enclosure
Apr 18 16:22:05 lamu  crc_t10dif
Apr 18 16:22:05 lamu  ata_generic
Apr 18 16:22:05 lamu  uhci_hcd
Apr 18 16:22:05 lamu  ata_piix
Apr 18 16:22:05 lamu  ehci_hcd
Apr 18 16:22:05 lamu  libata
Apr 18 16:22:05 lamu  megaraid_sas
Apr 18 16:22:05 lamu  scsi_mod
Apr 18 16:22:05 lamu  usbcore
Apr 18 16:22:05 lamu  usb_common
Apr 18 16:22:05 lamu  bnx2
Apr 18 16:22:05 lamu  [last unloaded: scsi_wait_scan]
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.265328] 
Apr 18 16:22:05 lamu [15868.268305] Pid: 10572, comm: kvm Not tainted 3.2.0-2-amd64 #1
Apr 18 16:22:05 lamu  Dell Inc. PowerEdge R410
Apr 18 16:22:05 lamu /0N051F 
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.285556] RIP: 0010:[<ffffffffa02dc33d>] 
Apr 18 16:22:05 lamu  [<ffffffffa02dc33d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 16:22:05 lamu [15868.304520] RSP: 0018:ffff88042fc03b18  EFLAGS: 00010293
Apr 18 16:22:05 lamu [15868.315127] RAX: 0000000000000000 RBX: ffff8804251565c0 RCX: 0000000000000007
Apr 18 16:22:05 lamu [15868.329377] RDX: ffffffffa02dc30f RSI: 00000001003ba680 RDI: ffff8804251565c0
Apr 18 16:22:05 lamu [15868.343625] RBP: ffff880226b0e000 R08: 0000000000000000 R09: ffff88042fc03ad0
Apr 18 16:22:05 lamu [15868.357874] R10: ffffffff8165aac0 R11: ffffffff8165aac0 R12: 0000000000000000
Apr 18 16:22:05 lamu [15868.372122] R13: ffff880426414002 R14: ffff8803e0e02180 R15: ffff880426414000
Apr 18 16:22:05 lamu [15868.386372] FS:  00007f2d79be2900(0000) GS:ffff88042fc00000(0000) knlGS:0000000000000000
Apr 18 16:22:05 lamu [15868.402545] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 18 16:22:05 lamu [15868.414019] CR2: 0000000000000018 CR3: 00000001c8133000 CR4: 00000000000026e0
Apr 18 16:22:05 lamu [15868.428268] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 18 16:22:05 lamu [15868.442517] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr 18 16:22:05 lamu [15868.456766] Process kvm (pid: 10572, threadinfo ffff8801c8194000, task ffff880225a77120)
Apr 18 16:22:05 lamu [15868.472938] Stack:
Apr 18 16:22:05 lamu [15868.476955]  ffffffff80000000
Apr 18 16:22:05 lamu  ffffffffa02dc714
Apr 18 16:22:05 lamu  ffff8804251565c0
Apr 18 16:22:05 lamu  ffff880226b0e000
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.491788]  ffff880424e25000
Apr 18 16:22:05 lamu  ffffffffa02dcaa0
Apr 18 16:22:05 lamu  ffff880380000000
Apr 18 16:22:05 lamu  0000000200000000
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.506620]  ffff8804251565c0
Apr 18 16:22:05 lamu  ffffffffa02e2cd0
Apr 18 16:22:05 lamu  ffffffff81691190
Apr 18 16:22:05 lamu  0000000000000002
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.521450] Call Trace:
Apr 18 16:22:05 lamu [15868.526334]  <IRQ> 
Apr 18 16:22:05 lamu  
Apr 18 16:22:05 lamu [15868.530541]  [<ffffffffa02dc714>] ? br_parse_ip_options+0x3d/0x19a [bridge]
Apr 18 16:22:05 lamu [15868.544447]  [<ffffffffa02dcaa0>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
Apr 18 16:22:05 lamu [15868.558008]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:05 lamu [15868.568620]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:05 lamu [15868.581139]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:06 lamu [15868.593656]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:06 lamu [15868.604788]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:06 lamu [15868.617307]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.631041]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:06 lamu [15868.643559]  [<ffffffffa02d7887>] ? NF_HOOK.constprop.10+0x3c/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.657465]  [<ffffffffa02d7a1b>] ? br_forward+0x16/0x5a [bridge]
Apr 18 16:22:06 lamu [15868.669637]  [<ffffffffa02d8543>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
Apr 18 16:22:06 lamu [15868.684254]  [<ffffffffa02dc638>] ? br_nf_pre_routing_finish+0x1ee/0x1fb [bridge]
Apr 18 16:22:06 lamu [15868.699217]  [<ffffffffa02dbff7>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
Apr 18 16:22:06 lamu [15868.712088]  [<ffffffffa02dcf91>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
Apr 18 16:22:06 lamu [15868.725820]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:06 lamu [15868.736432]  [<ffffffff8128b151>] ? napi_gro_receive+0x1d/0x2b
Apr 18 16:22:06 lamu [15868.748081]  [<ffffffff8128acfb>] ? napi_skb_finish+0x1c/0x31
Apr 18 16:22:06 lamu [15868.759561]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.773292]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:06 lamu [15868.784426]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.798157]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.811891]  [<ffffffffa02d8388>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
Apr 18 16:22:06 lamu [15868.825621]  [<ffffffff8128b1bf>] ? napi_complete+0x28/0x37
Apr 18 16:22:06 lamu [15868.836751]  [<ffffffffa02d8764>] ? br_handle_frame+0x1b3/0x1cb [bridge]
Apr 18 16:22:06 lamu [15868.850136]  [<ffffffffa02d85b1>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
Apr 18 16:22:06 lamu [15868.864748]  [<ffffffff81289222>] ? __netif_receive_skb+0x324/0x41f
Apr 18 16:22:06 lamu [15868.877266]  [<ffffffff81289389>] ? process_backlog+0x6c/0x123
Apr 18 16:22:06 lamu [15868.888916]  [<ffffffff8128b26f>] ? net_rx_action+0xa1/0x1af
Apr 18 16:22:06 lamu [15868.900221]  [<ffffffff81036faf>] ? test_tsk_need_resched+0xa/0x13
Apr 18 16:22:06 lamu [15868.912565]  [<ffffffff8104be34>] ? __do_softirq+0xb9/0x177
Apr 18 16:22:06 lamu [15868.923698]  [<ffffffff813503ec>] ? call_softirq+0x1c/0x30
Apr 18 16:22:06 lamu [15868.934652]  <EOI> 
Apr 18 16:22:06 lamu  
Apr 18 16:22:06 lamu [15868.938860]  [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
Apr 18 16:22:06 lamu [15868.949468]  [<ffffffff8128b55f>] ? netif_rx_ni+0x1e/0x27
Apr 18 16:22:06 lamu [15868.960252]  [<ffffffffa02bd721>] ? tun_get_user+0x39a/0x3c2 [tun]
Apr 18 16:22:06 lamu [15868.972597]  [<ffffffffa02bda66>] ? tun_chr_poll+0xcd/0xcd [tun]
Apr 18 16:22:06 lamu [15868.984596]  [<ffffffffa02bdac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
Apr 18 16:22:06 lamu [15868.997461]  [<ffffffff810f95e4>] ? do_sync_readv_writev+0x9a/0xd7
Apr 18 16:22:06 lamu [15869.009804]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:06 lamu [15869.020932]  [<ffffffff810f8c66>] ? do_sync_read+0xab/0xe3
Apr 18 16:22:06 lamu [15869.031888]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:06 lamu [15869.043021]  [<ffffffff81162649>] ? security_file_permission+0x16/0x2d
Apr 18 16:22:06 lamu [15869.056059]  [<ffffffff810f9848>] ? do_readv_writev+0xaf/0x11c
Apr 18 16:22:06 lamu [15869.067710]  [<ffffffff8112abce>] ? eventfd_ctx_read+0x162/0x174
Apr 18 16:22:06 lamu [15869.079708]  [<ffffffff8103f403>] ? try_to_wake_up+0x197/0x197
Apr 18 16:22:06 lamu [15869.091357]  [<ffffffff810f9a1d>] ? sys_writev+0x45/0x90
Apr 18 16:22:06 lamu [15869.101967]  [<ffffffff8134e192>] ? system_call_fastpath+0x16/0x1b
Apr 18 16:22:06 lamu [15869.114309] Code: 
Apr 18 16:22:06 53  
Apr 18 16:22:06 48  
Apr 18 16:22:06 89  
Apr 18 16:22:06 fb  
Apr 18 16:22:06 48  
Apr 18 16:22:06 83  
Apr 18 16:22:06 ec  
Apr 18 16:22:06 10  
Apr 18 16:22:06 66  
Apr 18 16:22:06 81  
Apr 18 16:22:06 7f  
Apr 18 16:22:06 7e  
Apr 18 16:22:06 08  
Apr 18 16:22:06 06  
Apr 18 16:22:06 4c  
Apr 18 16:22:06 8b  
Apr 18 16:22:06 a7  
Apr 18 16:22:06 98  
Apr 18 16:22:06 00  
Apr 18 16:22:06 00  
Apr 18 16:22:06 00  
Apr 18 16:22:06 74  
Apr 18 16:22:06 3d  
Apr 18 16:22:06 e8  
Apr 18 16:22:06 07  
Apr 18 16:22:06 fe  
Apr 18 16:22:06 ff  
Apr 18 16:22:06 ff  
Apr 18 16:22:06 66  
Apr 18 16:22:06 3d  
Apr 18 16:22:06 08  
Apr 18 16:22:06 06  
Apr 18 16:22:06 75  
Apr 18 16:22:06 09  
Apr 18 16:22:06 83  
Apr 18 16:22:06 3d  
Apr 18 16:22:06 91  
Apr 18 16:22:06 6a  
Apr 18 16:22:06 00  
Apr 18 16:22:06 00  
Apr 18 16:22:06 00  
Apr 18 16:22:06 75  
Apr 18 16:22:06 29  
Apr 18 16:22:06 lamu  
Apr 18 16:22:06 f6  
Apr 18 16:22:06 44  
Apr 18 16:22:06 24  
Apr 18 16:22:06 lamu  
Apr 18 16:22:06 lamu [15869.143402] ------------[ cut here ]------------
Apr 18 16:22:06 lamu [15869.143405] WARNING: at /tmp/buildd/linux-2.6-3.2.14/debian/build/source_amd64_none/kernel/softirq.c:159 _local_bh_enable_ip.isra.11+0x3d/0x88()
Apr 18 16:22:06 lamu [15869.143408] Hardware name: PowerEdge R410
Apr 18 16:22:06 lamu [15869.143409] Modules linked in: option usb_wwan usbserial uas usb_storage ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_rr ip_vs nf_conntrack libcrc32c ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables crc32c drbd lru_cache cn sit tunnel4 tun bridge stp virtio_net virtio_blk virtio_rng rng_core virtio_pci virtio_ring virtio kvm_intel kvm ipmi_devintf ipmi_poweroff ipmi_si ipmi_watchdog ipmi_msghandler netconsole configfs loop snd_pcm snd_page_alloc snd_timer snd dcdbas iTCO_wdt soundcore i7core_edac edac_core joydev iTCO_vendor_support psmouse evdev pcspkr serio_raw processor button thermal_sys ext3 mbcache jbd dm_mod usbhid sr_mod hid cdrom ses sd_mod enclosure crc_t10dif ata_generic uhci_hcd ata_piix ehci_hcd libata megaraid_sas scsi_mod usbcore usb_common bnx2 [last unloaded: scsi_wait_scan]
Apr 18 16:22:06 lamu [15869.143447] Pid: 10572, comm: kvm Not tainted 3.2.0-2-amd64 #1
Apr 18 16:22:06 lamu [15869.143449] Call Trace:
Apr 18 16:22:06 lamu [15869.143450]  <IRQ>  [<ffffffff81046815>] ? warn_slowpath_common+0x78/0x8c
Apr 18 16:22:06 lamu [15869.143457]  [<ffffffff8104bd26>] ? _local_bh_enable_ip.isra.11+0x3d/0x88
Apr 18 16:22:06 lamu [15869.143463]  [<ffffffffa0000748>] ? bnx2_reg_rd_ind+0x31/0x38 [bnx2]
Apr 18 16:22:06 lamu [15869.143470]  [<ffffffffa00097d7>] ? bnx2_poll+0x1b7/0x1c4 [bnx2]
Apr 18 16:22:06 lamu [15869.143476]  [<ffffffff8129b0bd>] ? netpoll_poll_dev.part.16+0x9b/0x499
Apr 18 16:22:06 lamu [15869.143480]  [<ffffffff8129b5d5>] ? netpoll_send_skb_on_dev+0x11a/0x201
Apr 18 16:22:06 lamu [15869.143485]  [<ffffffffa02d631a>] ? br_dev_xmit+0x12e/0x142 [bridge]
Apr 18 16:22:06 lamu [15869.143489]  [<ffffffff8129b586>] ? netpoll_send_skb_on_dev+0xcb/0x201
Apr 18 16:22:06 lamu [15869.143493]  [<ffffffffa01b125c>] ? write_msg+0x98/0xf3 [netconsole]
Apr 18 16:22:06 lamu [15869.143497]  [<ffffffff8104695e>] ? __call_console_drivers+0x72/0x83
Apr 18 16:22:06 lamu [15869.143501]  [<ffffffff8104702a>] ? console_unlock+0x144/0x1e8
Apr 18 16:22:06 lamu [15869.143504]  [<ffffffff8104754d>] ? vprintk+0x396/0x3d9
Apr 18 16:22:06 lamu [15869.143509]  [<ffffffffa02dc341>] ? br_nf_forward_finish+0x32/0x95 [bridge]
Apr 18 16:22:06 lamu [15869.143515]  [<ffffffffa02dc312>] ? br_nf_forward_finish+0x3/0x95 [bridge]
Apr 18 16:22:06 lamu [15869.143521]  [<ffffffff81342bf3>] ? printk+0x43/0x48
Apr 18 16:22:06 lamu [15869.143524]  [<ffffffff8100fe6a>] ? show_registers+0x1de/0x20a
Apr 18 16:22:06 lamu [15869.143530]  [<ffffffff8134a09e>] ? __die+0x8b/0xc8
Apr 18 16:22:06 lamu [15869.143533]  [<ffffffff813423c3>] ? no_context+0x1d6/0x20e
Apr 18 16:22:06 lamu [15869.143538]  [<ffffffff810e9c80>] ? virt_to_slab+0x6/0x16
Apr 18 16:22:07 lamu [15869.143541]  [<ffffffff8134c019>] ? do_page_fault+0x1a8/0x337
Apr 18 16:22:07 lamu [15869.143546]  [<ffffffffa039df06>] ? ip_vs_conn_put+0x28/0x32 [ip_vs]
Apr 18 16:22:07 lamu [15869.143551]  [<ffffffffa03a00e0>] ? ip_vs_out+0x2bd/0x432 [ip_vs]
Apr 18 16:22:07 lamu [15869.143554]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:07 lamu [15869.143558]  [<ffffffff81349775>] ? page_fault+0x25/0x30
Apr 18 16:22:07 lamu [15869.143564]  [<ffffffffa02dc30f>] ? nf_bridge_update_protocol+0x20/0x20 [bridge]
Apr 18 16:22:07 lamu [15869.143570]  [<ffffffffa02dc33d>] ? br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 16:22:07 lamu [15869.143575]  [<ffffffffa02dc32e>] ? br_nf_forward_finish+0x1f/0x95 [bridge]
Apr 18 16:22:07 lamu [15869.143581]  [<ffffffffa02dc714>] ? br_parse_ip_options+0x3d/0x19a [bridge]
Apr 18 16:22:07 lamu [15869.143586]  [<ffffffffa02dcaa0>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
Apr 18 16:22:07 lamu [15869.143590]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:07 lamu [15869.143595]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15869.143599]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15869.143602]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:07 lamu [15869.143607]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15869.143612]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143617]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15869.143622]  [<ffffffffa02d7887>] ? NF_HOOK.constprop.10+0x3c/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143627]  [<ffffffffa02d7a1b>] ? br_forward+0x16/0x5a [bridge]
Apr 18 16:22:07 lamu [15869.143632]  [<ffffffffa02d8543>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
Apr 18 16:22:07 lamu [15869.143638]  [<ffffffffa02dc638>] ? br_nf_pre_routing_finish+0x1ee/0x1fb [bridge]
Apr 18 16:22:07 lamu [15869.143644]  [<ffffffffa02dbff7>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
Apr 18 16:22:07 lamu [15869.143649]  [<ffffffffa02dcf91>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
Apr 18 16:22:07 lamu [15869.143653]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:07 lamu [15869.143656]  [<ffffffff8128b151>] ? napi_gro_receive+0x1d/0x2b
Apr 18 16:22:07 lamu [15869.143659]  [<ffffffff8128acfb>] ? napi_skb_finish+0x1c/0x31
Apr 18 16:22:07 lamu [15869.143664]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143667]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:07 lamu [15869.143672]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143677]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143683]  [<ffffffffa02d8388>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
Apr 18 16:22:07 lamu [15869.143686]  [<ffffffff8128b1bf>] ? napi_complete+0x28/0x37
Apr 18 16:22:07 lamu [15869.143691]  [<ffffffffa02d8764>] ? br_handle_frame+0x1b3/0x1cb [bridge]
Apr 18 16:22:07 lamu [15869.143696]  [<ffffffffa02d85b1>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
Apr 18 16:22:07 lamu [15869.143700]  [<ffffffff81289222>] ? __netif_receive_skb+0x324/0x41f
Apr 18 16:22:07 lamu [15869.143703]  [<ffffffff81289389>] ? process_backlog+0x6c/0x123
Apr 18 16:22:07 lamu [15869.143706]  [<ffffffff8128b26f>] ? net_rx_action+0xa1/0x1af
Apr 18 16:22:07 lamu [15869.143709]  [<ffffffff81036faf>] ? test_tsk_need_resched+0xa/0x13
Apr 18 16:22:07 lamu [15869.143712]  [<ffffffff8104be34>] ? __do_softirq+0xb9/0x177
Apr 18 16:22:07 lamu [15869.143715]  [<ffffffff813503ec>] ? call_softirq+0x1c/0x30
Apr 18 16:22:07 lamu [15869.143716]  <EOI>  [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
Apr 18 16:22:07 lamu [15869.143722]  [<ffffffff8128b55f>] ? netif_rx_ni+0x1e/0x27
Apr 18 16:22:07 lamu [15869.143725]  [<ffffffffa02bd721>] ? tun_get_user+0x39a/0x3c2 [tun]
Apr 18 16:22:07 lamu [15869.143729]  [<ffffffffa02bda66>] ? tun_chr_poll+0xcd/0xcd [tun]
Apr 18 16:22:07 lamu [15869.143733]  [<ffffffffa02bdac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
Apr 18 16:22:07 lamu [15869.143736]  [<ffffffff810f95e4>] ? do_sync_readv_writev+0x9a/0xd7
Apr 18 16:22:07 lamu [15869.143739]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:07 lamu [15869.143742]  [<ffffffff810f8c66>] ? do_sync_read+0xab/0xe3
Apr 18 16:22:07 lamu [15869.143744]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:07 lamu [15869.143747]  [<ffffffff81162649>] ? security_file_permission+0x16/0x2d
Apr 18 16:22:07 lamu [15869.143750]  [<ffffffff810f9848>] ? do_readv_writev+0xaf/0x11c
Apr 18 16:22:07 lamu [15869.143753]  [<ffffffff8112abce>] ? eventfd_ctx_read+0x162/0x174
Apr 18 16:22:07 lamu [15869.143756]  [<ffffffff8103f403>] ? try_to_wake_up+0x197/0x197
Apr 18 16:22:07 lamu [15869.143759]  [<ffffffff810f9a1d>] ? sys_writev+0x45/0x90
Apr 18 16:22:07 lamu [15869.143762]  [<ffffffff8134e192>] ? system_call_fastpath+0x16/0x1b
Apr 18 16:22:07 lamu [15869.143765] ---[ end trace aba9f7cf33914c5d ]---
Apr 18 16:22:07 lamu [15870.282276] 18 
Apr 18 16:22:07 01  
Apr 18 16:22:07 49  
Apr 18 16:22:07 8b  
Apr 18 16:22:07 6c  
Apr 18 16:22:07 24  
Apr 18 16:22:07 08  
Apr 18 16:22:07 74  
Apr 18 16:22:07 12  
Apr 18 16:22:07 8a  
Apr 18 16:22:07 43  
Apr 18 16:22:07 7d  
Apr 18 16:22:07 83  
Apr 18 16:22:07 e0  
Apr 18 16:22:07 f8  
Apr 18 16:22:07 83  
Apr 18 16:22:07 c8  
Apr 18 16:22:07 lamu  
Apr 18 16:22:07 lamu [15870.294307] RIP 
Apr 18 16:22:07 lamu  [<ffffffffa02dc33d>] br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 16:22:07 lamu [15870.308573]  RSP <ffff88042fc03b18>
Apr 18 16:22:07 lamu [15870.315539] CR2: 0000000000000018
Apr 18 16:22:07 lamu [15870.322840] ---[ end trace aba9f7cf33914c5e ]---
Apr 18 16:22:07 lamu [15870.332281] Kernel panic - not syncing: Fatal exception in interrupt
Apr 18 16:22:07 lamu [15870.345225] Pid: 10572, comm: kvm Tainted: G      D W    3.2.0-2-amd64 #1
Apr 18 16:22:07 lamu [15870.359065] Call Trace:
Apr 18 16:22:07 lamu [15870.364320]  <IRQ> 
Apr 18 16:22:07 lamu  [<ffffffff81342aa0>] ? panic+0x95/0x1a5
Apr 18 16:22:07 lamu [15870.376264]  [<ffffffff8134a006>] ? oops_end+0xa9/0xb6
Apr 18 16:22:07 lamu [15870.386917]  [<ffffffff813423ec>] ? no_context+0x1ff/0x20e
Apr 18 16:22:07 lamu [15870.398274]  [<ffffffff810e9c80>] ? virt_to_slab+0x6/0x16
Apr 18 16:22:07 lamu [15870.409348]  [<ffffffff8134c019>] ? do_page_fault+0x1a8/0x337
Apr 18 16:22:07 lamu [15870.421135]  [<ffffffffa039df06>] ? ip_vs_conn_put+0x28/0x32 [ip_vs]
Apr 18 16:22:07 lamu [15870.434043]  [<ffffffffa03a00e0>] ? ip_vs_out+0x2bd/0x432 [ip_vs]
Apr 18 16:22:07 lamu [15870.446570]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:07 lamu [15870.457981]  [<ffffffff81349775>] ? page_fault+0x25/0x30
Apr 18 16:22:07 lamu [15870.468869]  [<ffffffffa02dc30f>] ? nf_bridge_update_protocol+0x20/0x20 [bridge]
Apr 18 16:22:07 lamu [15870.483977]  [<ffffffffa02dc33d>] ? br_nf_forward_finish+0x2e/0x95 [bridge]
Apr 18 16:22:07 lamu [15870.498164]  [<ffffffffa02dc32e>] ? br_nf_forward_finish+0x1f/0x95 [bridge]
Apr 18 16:22:07 lamu [15870.512405]  [<ffffffffa02dc714>] ? br_parse_ip_options+0x3d/0x19a [bridge]
Apr 18 16:22:07 lamu [15870.526664]  [<ffffffffa02dcaa0>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
Apr 18 16:22:07 lamu [15870.540676]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:07 lamu [15870.551680]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15870.564408]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:07 lamu [15870.577207]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:08 lamu [15870.588572]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:08 lamu [15870.601300]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.615222]  [<ffffffffa02d7941>] ? __br_deliver+0xa0/0xa0 [bridge]
Apr 18 16:22:08 lamu [15870.628089]  [<ffffffffa02d7887>] ? NF_HOOK.constprop.10+0x3c/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.642400]  [<ffffffffa02d7a1b>] ? br_forward+0x16/0x5a [bridge]
Apr 18 16:22:08 lamu [15870.654796]  [<ffffffffa02d8543>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
Apr 18 16:22:08 lamu [15870.669621]  [<ffffffffa02dc638>] ? br_nf_pre_routing_finish+0x1ee/0x1fb [bridge]
Apr 18 16:22:08 lamu [15870.684772]  [<ffffffffa02dbff7>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
Apr 18 16:22:08 lamu [15870.698003]  [<ffffffffa02dcf91>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
Apr 18 16:22:08 lamu [15870.711896]  [<ffffffff812abfa1>] ? nf_iterate+0x41/0x77
Apr 18 16:22:08 lamu [15870.722763]  [<ffffffff8128b151>] ? napi_gro_receive+0x1d/0x2b
Apr 18 16:22:08 lamu [15870.734852]  [<ffffffff8128acfb>] ? napi_skb_finish+0x1c/0x31
Apr 18 16:22:08 lamu [15870.746586]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.760677]  [<ffffffff812ac03f>] ? nf_hook_slow+0x68/0x101
Apr 18 16:22:08 lamu [15870.772150]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.786239]  [<ffffffffa02d83a2>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.800153]  [<ffffffffa02d8388>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
Apr 18 16:22:08 lamu [15870.814178]  [<ffffffff8128b1bf>] ? napi_complete+0x28/0x37
Apr 18 16:22:08 lamu [15870.825710]  [<ffffffffa02d8764>] ? br_handle_frame+0x1b3/0x1cb [bridge]
Apr 18 16:22:08 lamu [15870.839289]  [<ffffffffa02d85b1>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
Apr 18 16:22:08 lamu [15870.854163]  [<ffffffff81289222>] ? __netif_receive_skb+0x324/0x41f
Apr 18 16:22:08 lamu [15870.866863]  [<ffffffff81289389>] ? process_backlog+0x6c/0x123
Apr 18 16:22:08 lamu [15870.878810]  [<ffffffff8128b26f>] ? net_rx_action+0xa1/0x1af
Apr 18 16:22:08 lamu [15870.890414]  [<ffffffff81036faf>] ? test_tsk_need_resched+0xa/0x13
Apr 18 16:22:08 lamu [15870.902969]  [<ffffffff8104be34>] ? __do_softirq+0xb9/0x177
Apr 18 16:22:08 lamu [15870.914361]  [<ffffffff813503ec>] ? call_softirq+0x1c/0x30
Apr 18 16:22:08 lamu [15870.925616]  <EOI> 
Apr 18 16:22:08 lamu  [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
Apr 18 16:22:08 lamu [15870.938023]  [<ffffffff8128b55f>] ? netif_rx_ni+0x1e/0x27
Apr 18 16:22:08 lamu [15870.949046]  [<ffffffffa02bd721>] ? tun_get_user+0x39a/0x3c2 [tun]
Apr 18 16:22:08 lamu [15870.961716]  [<ffffffffa02bda66>] ? tun_chr_poll+0xcd/0xcd [tun]
Apr 18 16:22:08 lamu [15870.973982]  [<ffffffffa02bdac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
Apr 18 16:22:08 lamu [15870.987111]  [<ffffffff810f95e4>] ? do_sync_readv_writev+0x9a/0xd7
Apr 18 16:22:08 lamu [15870.999717]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:08 lamu [15871.011058]  [<ffffffff810f8c66>] ? do_sync_read+0xab/0xe3
Apr 18 16:22:08 lamu [15871.022382]  [<ffffffff810363cb>] ? should_resched+0x5/0x23
Apr 18 16:22:08 lamu [15871.033786]  [<ffffffff81162649>] ? security_file_permission+0x16/0x2d
Apr 18 16:22:08 lamu [15871.047087]  [<ffffffff810f9848>] ? do_readv_writev+0xaf/0x11c
Apr 18 16:22:08 lamu [15871.058948]  [<ffffffff8112abce>] ? eventfd_ctx_read+0x162/0x174
Apr 18 16:22:08 lamu [15871.071307]  [<ffffffff8103f403>] ? try_to_wake_up+0x197/0x197
Apr 18 16:22:08 lamu [15871.083164]  [<ffffffff810f9a1d>] ? sys_writev+0x45/0x90
Apr 18 16:22:08 lamu [15871.094031]  [<ffffffff8134e192>] ? system_call_fastpath+0x16/0x1b

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-19  8:01   ` Massimo Cetra
@ 2012-04-19  8:14     ` Eric Dumazet
  2012-04-19  8:45       ` Eric Dumazet
  0 siblings, 1 reply; 10+ messages in thread
From: Eric Dumazet @ 2012-04-19  8:14 UTC (permalink / raw)
  To: Massimo Cetra; +Cc: Massimo Cetra, netdev, peter.huangpeng

On Thu, 2012-04-19 at 10:01 +0200, Massimo Cetra wrote:
> On 18/04/2012 12:31, Eric Dumazet wrote:
> >
> > Seems a different issue, skb->nf_bridge seems to be NULL
> >
> 
> This is another trace of another panic.
> I hope it may be useful.
> Please notice that it is not related to adding/removing br interfaces or 
> tun/vic interfaces from a bridge set.
> It happened suddenly during a normal workload.
> 
> When were those bridge-related bugs introduced ?
> What's the latest release that seems to work ?
> I'm asking so that i can restore some servers to a proper workload.
> 
> 
> Massimo

Maybe you should try latest kernel, because we fixed some bugs lately.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-18 10:08 fake rtable dst patch applied but kernel keeps panicing Massimo Cetra
  2012-04-18 10:31 ` Eric Dumazet
@ 2012-04-19  8:22 ` Peter Huang(Peng)
  2012-04-19  8:37   ` Massimo Cetra
  1 sibling, 1 reply; 10+ messages in thread
From: Peter Huang(Peng) @ 2012-04-19  8:22 UTC (permalink / raw)
  To: Massimo Cetra; +Cc: netdev, Eric Dumazet

Hi, Massimo

can this be reproduce easily or depend on some HW or SW environment?

This patch already verified on 2.6.32.36/54,3.3.1, 3.4.rc[23] for over
6 hours, but didn't encounter this problem.

On 2012/4/18 18:08, Massimo Cetra wrote:
> Hi Eric,
> Hi Peter,
>
> I applied the 1st patch by Peter to a 3.2.14 kernel, compiled, installed
> and rebooted.
>
> I have had another panic and i'm attaching the oops as i received it
> through netcnsole.
>
> I don't think the patch is enough to prevent the OOPS.
>
> Can i be helpful in some way ?
>
> Thanks,
> Massimo

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-19  8:22 ` Peter Huang(Peng)
@ 2012-04-19  8:37   ` Massimo Cetra
  0 siblings, 0 replies; 10+ messages in thread
From: Massimo Cetra @ 2012-04-19  8:37 UTC (permalink / raw)
  To: Peter Huang(Peng); +Cc: Massimo Cetra, netdev, Eric Dumazet

On 19/04/2012 10:22, Peter Huang(Peng) wrote:
> Hi, Massimo
>
> can this be reproduce easily or depend on some HW or SW environment?
>
> This patch already verified on 2.6.32.36/54,3.3.1, 3.4.rc[23] for over
> 6 hours, but didn't encounter this problem.

The server is a Dell PowerEdge R410.

The network cards are:
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 
Gigabit Ethernet (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 
Gigabit Ethernet (rev 20)

The server is doing nothing except for routing packets and running 
virtual servers.

This server is guesting about 10 KVM virtual servers in a bridged 
configuration.


lamu:~# brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.0024e869f8eb       yes             eth1
                                                         kvmnet-db0
                                                         kvmnet-devel0
                                                         kvmnet-kermit0
                                                         kvmnet-mail0
                                                         kvmnet-max0
                                                         kvmnet-repo0
                                                         kvmnet-voip0
                                                         kvmnet-wper0
                                                         kvmnet-wsos0


kvmnet-* interfaces are tun devices.

-------------------------

Whenever i add a couple of "virtual" bridges assigning an ip address to, 
for example, br0:1 br0:2, the panic happens a couple of times a day.

If i leave only br0 and br1 it's very hard to trigger (but this may be 
related to network traffic which, in this case, is very low).

By the way, i'm using this kernel (with and without the previous patch) 
on my workstation which has a br0 interface and some VM attached in the 
same way.
I am not experiencing crashes and the only difference is that i don't 
have, on my desktop, any br0:1 br0:2 interfaces.

Hope it helps.

Max

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-19  8:14     ` Eric Dumazet
@ 2012-04-19  8:45       ` Eric Dumazet
  2012-04-19 10:52         ` Massimo Cetra
  2012-04-20 10:02         ` Massimo Cetra
  0 siblings, 2 replies; 10+ messages in thread
From: Eric Dumazet @ 2012-04-19  8:45 UTC (permalink / raw)
  To: Massimo Cetra; +Cc: Massimo Cetra, netdev, peter.huangpeng

On Thu, 2012-04-19 at 10:14 +0200, Eric Dumazet wrote:
> On Thu, 2012-04-19 at 10:01 +0200, Massimo Cetra wrote:
> > On 18/04/2012 12:31, Eric Dumazet wrote:
> > >
> > > Seems a different issue, skb->nf_bridge seems to be NULL
> > >
> > 
> > This is another trace of another panic.
> > I hope it may be useful.
> > Please notice that it is not related to adding/removing br interfaces or 
> > tun/vic interfaces from a bridge set.
> > It happened suddenly during a normal workload.
> > 
> > When were those bridge-related bugs introduced ?
> > What's the latest release that seems to work ?
> > I'm asking so that i can restore some servers to a proper workload.
> > 
> > 
> > Massimo
> 
> Maybe you should try latest kernel, because we fixed some bugs lately.
> 
> 

Oh well, at first glance nf_bridge_unshare() is buggy, not sure if this
can help your bug, but its another step.


[PATCH] bridge: fix nf_bridge_unshare()

If memory allocation failed, return an error.

If not, skb->nf_bridge should be updated to point to the copy, not old
info, or bad things can happen.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
 net/bridge/br_netfilter.c |   24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/net/bridge/br_netfilter.c b/net/bridge/br_netfilter.c
index dec4f38..b7c2cec 100644
--- a/net/bridge/br_netfilter.c
+++ b/net/bridge/br_netfilter.c
@@ -185,21 +185,20 @@ static inline struct nf_bridge_info *nf_bridge_alloc(struct sk_buff *skb)
 	return skb->nf_bridge;
 }
 
-static inline struct nf_bridge_info *nf_bridge_unshare(struct sk_buff *skb)
+
+static inline int nf_bridge_unshare(struct sk_buff *skb)
 {
-	struct nf_bridge_info *nf_bridge = skb->nf_bridge;
+	struct nf_bridge_info *copy, *nf_bridge = skb->nf_bridge;
 
 	if (atomic_read(&nf_bridge->use) > 1) {
-		struct nf_bridge_info *tmp = nf_bridge_alloc(skb);
-
-		if (tmp) {
-			memcpy(tmp, nf_bridge, sizeof(struct nf_bridge_info));
-			atomic_set(&tmp->use, 1);
-		}
+		copy = kmemdup(nf_bridge, sizeof(*nf_bridge), GFP_ATOMIC);
+		if (!copy)
+			return -ENOMEM;
+		atomic_set(&copy->use, 1);
 		nf_bridge_put(nf_bridge);
-		nf_bridge = tmp;
+		skb->nf_bridge = copy;
 	}
-	return nf_bridge;
+	return 0;
 }
 
 static inline void nf_bridge_push_encap_header(struct sk_buff *skb)
@@ -744,8 +743,9 @@ static unsigned int br_nf_forward_ip(unsigned int hook, struct sk_buff *skb,
 		return NF_ACCEPT;
 
 	/* Need exclusive nf_bridge_info since we might have multiple
-	 * different physoutdevs. */
-	if (!nf_bridge_unshare(skb))
+	 * different physoutdevs.
+	 */
+	if (nf_bridge_unshare(skb))
 		return NF_DROP;
 
 	parent = bridge_parent(out);

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-19  8:45       ` Eric Dumazet
@ 2012-04-19 10:52         ` Massimo Cetra
  2012-04-20 10:02         ` Massimo Cetra
  1 sibling, 0 replies; 10+ messages in thread
From: Massimo Cetra @ 2012-04-19 10:52 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Massimo Cetra, netdev, peter.huangpeng

On 19/04/2012 10:45, Eric Dumazet wrote:

>> Maybe you should try latest kernel, because we fixed some bugs lately.
>
> Oh well, at first glance nf_bridge_unshare() is buggy, not sure if this
> can help your bug, but its another step.
>

I'm recompiling 3.3.2 with this new patch.
I'll let you know.

Thanks you,

  Massimo

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-19  8:45       ` Eric Dumazet
  2012-04-19 10:52         ` Massimo Cetra
@ 2012-04-20 10:02         ` Massimo Cetra
  2012-04-20 13:22           ` Eric Dumazet
  1 sibling, 1 reply; 10+ messages in thread
From: Massimo Cetra @ 2012-04-20 10:02 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Massimo Cetra, netdev, peter.huangpeng

On 19/04/2012 10:45, Eric Dumazet wrote:
> Oh well, at first glance nf_bridge_unshare() is buggy, not sure if this
> can help your bug, but its another step.
>
>
> [PATCH] bridge: fix nf_bridge_unshare()
>
> If memory allocation failed, return an error.
>
> If not, skb->nf_bridge should be updated to point to the copy, not old
> info, or bad things can happen.
>
> Signed-off-by: Eric Dumazet<eric.dumazet@gmail.com>
> ---
>   net/bridge/br_netfilter.c |   24 ++++++++++++------------
>   1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/net/bridge/br_netfilter.c b/net/bridge/br_netfilter.c
> index dec4f38..b7c2cec 100644
> --- a/net/bridge/br_netfilter.c
> +++ b/net/bridge/br_netfilter.c
> @@ -185,21 +185,20 @@ static inline struct nf_bridge_info *nf_bridge_alloc(struct sk_buff *skb)
>   	return skb->nf_bridge;
>   }
>
> -static inline struct nf_bridge_info *nf_bridge_unshare(struct sk_buff *skb)
> +
> +static inline int nf_bridge_unshare(struct sk_buff *skb)
>   {
> -	struct nf_bridge_info *nf_bridge = skb->nf_bridge;
> +	struct nf_bridge_info *copy, *nf_bridge = skb->nf_bridge;
>
>   	if (atomic_read(&nf_bridge->use)>  1) {
> -		struct nf_bridge_info *tmp = nf_bridge_alloc(skb);
> -
> -		if (tmp) {
> -			memcpy(tmp, nf_bridge, sizeof(struct nf_bridge_info));
> -			atomic_set(&tmp->use, 1);
> -		}
> +		copy = kmemdup(nf_bridge, sizeof(*nf_bridge), GFP_ATOMIC);
> +		if (!copy)
> +			return -ENOMEM;
> +		atomic_set(&copy->use, 1);
>   		nf_bridge_put(nf_bridge);
> -		nf_bridge = tmp;
> +		skb->nf_bridge = copy;
>   	}
> -	return nf_bridge;
> +	return 0;
>   }
>
>   static inline void nf_bridge_push_encap_header(struct sk_buff *skb)
> @@ -744,8 +743,9 @@ static unsigned int br_nf_forward_ip(unsigned int hook, struct sk_buff *skb,
>   		return NF_ACCEPT;
>
>   	/* Need exclusive nf_bridge_info since we might have multiple
> -	 * different physoutdevs. */
> -	if (!nf_bridge_unshare(skb))
> +	 * different physoutdevs.
> +	 */
> +	if (nf_bridge_unshare(skb))
>   		return NF_DROP;
>
>   	parent = bridge_parent(out);
>
>

Hello, Eric,

i applied this patch and Peters last patch to a 3.3.2 kernel.
The result was a bit disappointing because the step was backwards.

Locally, from the same machine, i could ping each IP of each tun 
interface used by any virtual server.

 From the LAN such addresses were not pingable while the ip address of 
the bridge was reachable.

Max

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: fake rtable dst patch applied but kernel keeps panicing
  2012-04-20 10:02         ` Massimo Cetra
@ 2012-04-20 13:22           ` Eric Dumazet
  0 siblings, 0 replies; 10+ messages in thread
From: Eric Dumazet @ 2012-04-20 13:22 UTC (permalink / raw)
  To: Massimo Cetra; +Cc: netdev, peter.huangpeng

On Fri, 2012-04-20 at 12:02 +0200, Massimo Cetra wrote:

> 
> i applied this patch and Peters last patch to a 3.3.2 kernel.
> The result was a bit disappointing because the step was backwards.
> 
> Locally, from the same machine, i could ping each IP of each tun 
> interface used by any virtual server.
> 
>  From the LAN such addresses were not pingable while the ip address of 
> the bridge was reachable.
> 
> Max

I dont know, this code is crap and should be fixed.

This nf_bridge_unshare() is obviously buggy as hell.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2012-04-20 13:22 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-04-18 10:08 fake rtable dst patch applied but kernel keeps panicing Massimo Cetra
2012-04-18 10:31 ` Eric Dumazet
2012-04-19  8:01   ` Massimo Cetra
2012-04-19  8:14     ` Eric Dumazet
2012-04-19  8:45       ` Eric Dumazet
2012-04-19 10:52         ` Massimo Cetra
2012-04-20 10:02         ` Massimo Cetra
2012-04-20 13:22           ` Eric Dumazet
2012-04-19  8:22 ` Peter Huang(Peng)
2012-04-19  8:37   ` Massimo Cetra

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.