All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] xend: disallow multiple destroy() call
@ 2013-02-04  1:54 Joe Jin
  2013-02-04  9:52 ` Roger Pau Monné
  2013-03-19 17:41 ` Konrad Rzeszutek Wilk
  0 siblings, 2 replies; 10+ messages in thread
From: Joe Jin @ 2013-02-04  1:54 UTC (permalink / raw)
  To: Ian Campbell, Konrad Rzeszutek Wilk; +Cc: xen-devel

Current Xend allowing multiple call destroy() for same domain, this lead
multiple hard resets(FLR) for pci pass-through, and some controller might
failed.

In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after
guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and
we had to reboot the server to recovery it.

Signed-off-by: Joe Jin <joe.jin@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
---
 tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/python/xen/xend/XendDomainInfo.py b/tools/python/xen/xend/XendDomainInfo.py
index e9d3e7e..658c3b6 100644
--- a/tools/python/xen/xend/XendDomainInfo.py
+++ b/tools/python/xen/xend/XendDomainInfo.py
@@ -367,6 +367,8 @@ class XendDomainInfo:
     @type refresh_shutdown_lock: threading.Condition
     @ivar _deviceControllers: device controller cache for this domain
     @type _deviceControllers: dict 'string' to DevControllers
+    @ivar destroying: Is this domain destroying
+    @type destroying: bool
     """
     
     def __init__(self, info, domid = None, dompath = None, augment = False,
@@ -455,6 +457,8 @@ class XendDomainInfo:
         self._checkName(self.info['name_label'])
 
         self.metrics = XendVMMetrics(uuid.createString(), self)
+
+        self.destroying = False
             
 
     #
@@ -3073,6 +3077,12 @@ class XendDomainInfo:
 
         if self.domid is None:
             return
+
+        if self.destroying == False:
+            self.destroying = True
+        else:
+            raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid))
+
         from xen.xend import XendDomain
         log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid))
 
@@ -3088,6 +3098,7 @@ class XendDomainInfo:
                     self.info[state] = 0
                 self._stateSet(DOM_STATE_HALTED)
             except:
+                self.destroying = False
                 log.exception("XendDomainInfo.destroy: domain destruction failed.")
 
             XendDomain.instance().remove_domain(self)
-- 
1.8.1

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-04  1:54 [PATCH] xend: disallow multiple destroy() call Joe Jin
@ 2013-02-04  9:52 ` Roger Pau Monné
  2013-02-05  6:47   ` Joe Jin
  2013-03-19 17:41 ` Konrad Rzeszutek Wilk
  1 sibling, 1 reply; 10+ messages in thread
From: Roger Pau Monné @ 2013-02-04  9:52 UTC (permalink / raw)
  To: Joe Jin; +Cc: xen-devel, Ian Campbell, Konrad Rzeszutek Wilk

On 04/02/13 02:54, Joe Jin wrote:
> Current Xend allowing multiple call destroy() for same domain, this lead
> multiple hard resets(FLR) for pci pass-through, and some controller might
> failed.
> 
> In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after
> guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and
> we had to reboot the server to recovery it.

Does the same problem happen with libxl/xl?

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-04  9:52 ` Roger Pau Monné
@ 2013-02-05  6:47   ` Joe Jin
  2013-02-05  9:42     ` Ian Campbell
  0 siblings, 1 reply; 10+ messages in thread
From: Joe Jin @ 2013-02-05  6:47 UTC (permalink / raw)
  To: Roger Pau Monné; +Cc: xen-devel, Ian Campbell, Konrad Rzeszutek Wilk

On 02/04/13 17:52, Roger Pau Monné wrote:
> On 04/02/13 02:54, Joe Jin wrote:
>> Current Xend allowing multiple call destroy() for same domain, this lead
>> multiple hard resets(FLR) for pci pass-through, and some controller might
>> failed.
>>
>> In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after
>> guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and
>> we had to reboot the server to recovery it.
> 
> Does the same problem happen with libxl/xl?

execute xl-destroy twice crashed my server!

Thanks,
Joe

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-05  6:47   ` Joe Jin
@ 2013-02-05  9:42     ` Ian Campbell
  2013-02-16  5:38       ` Joe Jin
  2013-02-17  4:04       ` Joe Jin
  0 siblings, 2 replies; 10+ messages in thread
From: Ian Campbell @ 2013-02-05  9:42 UTC (permalink / raw)
  To: Joe Jin; +Cc: Konrad Rzeszutek Wilk, xen-devel, Roger Pau Monne

On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:
> execute xl-destroy twice crashed my server!

Can you give more details please.

Ian.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-05  9:42     ` Ian Campbell
@ 2013-02-16  5:38       ` Joe Jin
  2013-02-18 10:33         ` Ian Campbell
  2013-02-17  4:04       ` Joe Jin
  1 sibling, 1 reply; 10+ messages in thread
From: Joe Jin @ 2013-02-16  5:38 UTC (permalink / raw)
  To: Ian Campbell; +Cc: xen-devel, Roger Pau Monne, Konrad Rzeszutek Wilk

On 02/05/13 17:42, Ian Campbell wrote:
> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:
>> > execute xl-destroy twice crashed my server!
> Can you give more details please.
> 
Hi Ian,

Sorry the server not available for test now.

Would you please help to review my patch for xm-destroy?

Thanks,
Joe

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-05  9:42     ` Ian Campbell
  2013-02-16  5:38       ` Joe Jin
@ 2013-02-17  4:04       ` Joe Jin
  2013-02-17  5:07         ` Joe Jin
  1 sibling, 1 reply; 10+ messages in thread
From: Joe Jin @ 2013-02-17  4:04 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Konrad Rzeszutek Wilk, xen-devel, Roger Pau Monne

I got serial console output now and mostly like panic happened by "xl create":

(XEN) HVM1: Detected Xen v4.1.2-OVM
(XEN) HVM1: CPU speed is 2926 MHz
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13
(XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 05:0 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000
(XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40
(XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40
(XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1
(XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1
(XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004
(XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101
(XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100
(XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201
(XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1:  - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc00287c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80
(XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1:  - Manufacturer: LSI Corporation
(XEN) HVM1:  - Product name: LSI MPI Boot Support
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80
(XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1:  - Lo data: 000ea020-000ea04f
(XEN) HVM1:  - Hi data: fc002c00-fc00eb4f
(XEN) HVM1: vm86 TSS at fc00ec00
(XEN) HVM1: BIOS map:
(XEN) HVM1:  c0000-c8fff: VGA BIOS
(XEN) HVM1:  c9000-d4fff: PCI Option ROMs
(XEN) HVM1:  eb000-eb302: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM1:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:  [04]: 00000000:00100000 - 00000000:f0000000: RAM
(XEN) HVM1:  HOLE: 00000000:f0000000 - 00000000:fc000000
(XEN) HVM1:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1:  [06]: 00000001:00000000 - 00000004:0f800000: RAM
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: Options: apmbios pcibios eltorito PMM 
(XEN) HVM1: 
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes)
(XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata0  slave: QEMU HARDDISK ATA-7 Hard-Disk (  97 GBytes)
(XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes)
(XEN) HVM1: IDE time out
(XEN) HVM1: 
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1: PCI device 1000:0070 not found at index 0
(XEN) HVM1: PCI device 1000:0072 not found at index 2
(XEN) HVM1: PCI device 1000:0074 not found at index 0
(XEN) HVM1: PCI device 1000:0076 not found at index 0
(XEN) HVM1: PCI device 1000:0077 not found at index 0
(XEN) HVM1: PCI device 1000:0064 not found at index 0
(XEN) HVM1: PCI device 1000:0065 not found at index 0
(XEN) HVM1: PCI device 1000:0080 not found at index 0
(XEN) HVM1: PCI device 1000:0081 not found at index 0
(XEN) HVM1: PCI device 1000:0082 not found at index 0
(XEN) HVM1: PCI device 1000:0083 not found at index 0
(XEN) HVM1: PCI device 1000:0084 not found at index 0
(XEN) HVM1: PCI device 1000:0085 not found at index 0
(XEN) HVM1: PCI device 1000:0086 not found at index 0
(XEN) HVM1: PCI device 1000:0087 not found at index 0
(XEN) HVM1: 
(XEN) HVM1: 
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1: 
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported!
(XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported!
(XEN) HVM1: KBD: unsupported int 16h function 03
(XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported!
(XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9
(XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100
(XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100
(XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100
(XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100
(XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0
------------[ cut here ]------------
WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
Hardware name: SUN FIRE X4370 M2 SERVER       
remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]
Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1
Call Trace:
 [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240
 [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0
 [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70
 [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4
 [<ffffffff8125cf98>] ? sprintf+0x68/0x70
 [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0
 [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20
 [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240
 [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400
 [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0
 [<ffffffff810d571a>] free_desc+0x2a/0x70
 [<ffffffff810d579e>] irq_free_descs+0x3e/0x80
 [<ffffffff812f7822>] xen_free_irq+0x52/0x70
 [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
---[ end trace fa61ba514bac93c7 ]---
BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
PGD 6a2cb067 PUD 71d8c067 PMD 0 
Oops: 0000 [#1] SMP 
CPU 0 
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Pid: 19329, comm: qemu-dm Tainted: G        W   2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER       /ASSY,SC_BD,T4         
RIP: e030:[<ffffffff812f6a76>]  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
RSP: e02b:ffff88006cfc5dd8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48
RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70
RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218
R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701
FS:  00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200)
Stack:
 ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000
 ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138
 ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000
Call Trace:
 [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180
 [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff812f837f>] evtchn_put+0x1f/0x40
 [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
 [<ffffffff812640e1>] ? list_del+0x11/0x40
 [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
 [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
 [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
 [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
 [<ffffffff8113a740>] ? do_munmap+0x240/0x280
 [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 
RIP  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
 RSP <ffff88006cfc5dd8>
CR2: 000000000000001c

Thanks,
Joe
On 02/05/13 17:42, Ian Campbell wrote:
> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:
>> execute xl-destroy twice crashed my server!
> 
> Can you give more details please.
> 
> Ian.
> 
> 
> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-17  4:04       ` Joe Jin
@ 2013-02-17  5:07         ` Joe Jin
  2013-02-18 10:39           ` Ian Campbell
  0 siblings, 1 reply; 10+ messages in thread
From: Joe Jin @ 2013-02-17  5:07 UTC (permalink / raw)
  To: Joe Jin; +Cc: Roger Pau Monne, xen-devel, Ian Campbell, Konrad Rzeszutek Wilk

Execute "xl-destroy" twice lead the dom0 panic:
config file as below:

vncunused = 1
kernel = '/usr/lib/xen/boot/hvmloader'
vnc = 1
name = 'test'
memory = '16384'
#memory = '1024'
timer_mode = 0
device_model = '/usr/lib64/xen/bin/qemu-dm'
builder = 'hvm'
vnclisten = '0.0.0.0'
cpus = '0,2,4,6,8,11,1,3,5,7,9,10'
#on_crash = 'coredump-restart'
#on_crash = 'preserve'
on_crash = 'destroy'
on_reboot = 'restart'
vcpus = 12
pci = ['0d:00.0', '1f:00.0']
#pci = ['0d:00.0']
pae = 1
apic = 1
vif = ['type=netfront,bridge=priv1', 'type=netfront,bridge=net1', 'type=netfront,bridge=net2', 'type=netfront,bridge=net3', 'type=netfront,bridge=net4']
serial = 'pty'
disk = ['file:/path/System.img,xvda,w', 'file:/path/u01.img,xvdb,w', 'file:/path/swap.img,xvdc,w']
acpi = 1

Crash info from serial console:

(XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3180 mfn=df03c nr_mfns=4
(XEN) p2m.c:2797:d0 clear_mmio_p2m_entry: gfn_to_mfn failed! gfn=000f3182
(XEN) domctl.c:995:d0 memory_map:remove: gfn=f3100 mfn=df040 nr_mfns=40
------------[ cut here ]------------
kernel BUG at drivers/pci/access.c:423!
invalid opcode: 0000 [#1] SMP 
CPU 18 
Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport igb ses enclosure ixgbe mdio e1000e snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc ghes iTCO_wdt i2c_core pcspkr iTCO_vendor_support i7core_edac ioatdma edac_core hed dca dm_region_hash dm_log dm_mod usb_storage s
 hpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]

Pid: 20399, comm: xl Not tainted 2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER       /ASSY,SC_BD,T4         
RIP: e030:[<ffffffff812704bf>]  [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60
RSP: e02b:ffff88006e609dd8  EFLAGS: 00010202
RAX: 0000000000000024 RBX: 0000000000000001 RCX: 00000000800d0004
RDX: 0000000000000400 RSI: 0000000000000200 RDI: 0000000000000200
RBP: ffff88006e609de8 R08: 0000000000000002 R09: 0000000000000400
R10: 0000000000000000 R11: 0000000000000202 R12: ffff8800731e4000
R13: ffff8800731e4000 R14: ffffffff81565e20 R15: ffff8800731e40a0
FS:  00007fb7b5dfa730(0000) GS:ffff88007f66e000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003340613000 CR3: 0000000068440000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process xl (pid: 20399, threadinfo ffff88006e608000, task ffff880071fe0500)
Stack:
 0000000000000000 0000000000000000 ffff88006e609e18 ffffffff812749dc
 ffff8800731e4000 0000000000000000 ffff8800731e4000 00000000ffffffed
 ffff88006e609e38 ffffffff812764c4 ffff8800731e4090 0000000000000001
Call Trace:
 [<ffffffff812749dc>] pci_dev_reset+0x6c/0xd0
 [<ffffffff812764c4>] pci_reset_function+0x54/0x70
 [<ffffffff8127a6ba>] reset_store+0x5a/0x70
 [<ffffffff81342070>] dev_attr_store+0x20/0x30
 [<ffffffff811db48f>] flush_write_buffer+0x5f/0x90
 [<ffffffff811db856>] sysfs_write_file+0x66/0xa0
 [<ffffffff8116dd8e>] vfs_write+0xce/0x190
 [<ffffffff8116e3e5>] sys_write+0x55/0x90
 [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
Code: 07 00 00 48 c7 c7 e0 73 d7 81 89 c3 83 c8 20 c0 eb 05 41 88 84 24 80 07 00 00 83 e3 01 e8 6a 17 29 00 85 db 75 05 5b 41 5c c9 c3 <0f> 0b eb fe 66 0f 1f 44 00 00 0f 1f 80 00 00 00 00 55 48 89 e5 
RIP  [<ffffffff812704bf>] pci_block_user_cfg_access+0x4f/0x60
 RSP <ffff88006e609dd8>
pciback 0000:0d:


BTW: xm-create work fine but xl-create trigger the panic as below:

On 02/17/13 12:04, Joe Jin wrote:
> I got serial console output now and mostly like panic happened by "xl create":
> 
> (XEN) HVM1: Detected Xen v4.1.2-OVM
> (XEN) HVM1: CPU speed is 2926 MHz
> (XEN) HVM1: Xenbus rings @0xfeffc000, event channel 13
> (XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10
> (XEN) HVM1: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11
> (XEN) HVM1: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5
> (XEN) HVM1: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM1: pci dev 01:3 INTA->IRQ10
> (XEN) HVM1: pci dev 03:0 INTA->IRQ5
> (XEN) HVM1: pci dev 04:0 INTA->IRQ5
> (XEN) HVM1: pci dev 05:0 INTA->IRQ10
> (XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM1: pci dev 04:0 bar 30 size 00080000: f3000000
> (XEN) HVM1: pci dev 05:0 bar 30 size 00080000: f3080000
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3100 mfn=df040 nr_mfns=40
> (XEN) HVM1: pci dev 04:0 bar 1c size 00040000: f3100004
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3140 mfn=df640 nr_mfns=40
> (XEN) HVM1: pci dev 05:0 bar 1c size 00040000: f3140004
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3180 mfn=df03c nr_mfns=4
> (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3182 mfn=df03e nr_mfns=1
> (XEN) HVM1: pci dev 04:0 bar 14 size 00004000: f3180004
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3184 mfn=df63c nr_mfns=4
> (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3186 mfn=df63e nr_mfns=1
> (XEN) HVM1: pci dev 05:0 bar 14 size 00004000: f3184004
> (XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3188000
> (XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) HVM1: pci dev 04:0 bar 10 size 00000100: 0000c101
> (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100
> (XEN) HVM1: pci dev 05:0 bar 10 size 00000100: 0000c201
> (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100
> (XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c301
> (XEN) HVM1: Multiprocessor initialisation:
> (XEN) HVM1:  - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU1 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU2 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU3 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU4 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU5 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU6 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU7 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU8 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU9 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU10 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1:  - CPU11 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
> (XEN) HVM1: Writing SMBIOS tables ...
> (XEN) HVM1: Loading ROMBIOS ...
> (XEN) HVM1: 10364 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM1:   Relocating to 0xfc000000-0xfc00287c ... done
> (XEN) HVM1: Creating MP tables ...
> (XEN) HVM1: Loading Cirrus VGABIOS ...
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3000 mfn=df080 nr_mfns=80
> (XEN) HVM1: Loading PCI Option ROM ...
> (XEN) HVM1:  - Manufacturer: LSI Corporation
> (XEN) HVM1:  - Product name: LSI MPI Boot Support
> (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3000 mfn=df080 nr_mfns=80
> (XEN) domctl.c:985:d0 memory_map:add: gfn=f3080 mfn=df680 nr_mfns=80
> (XEN) domctl.c:995:d0 memory_map:remove: gfn=f3080 mfn=df680 nr_mfns=80
> (XEN) HVM1: Loading ACPI ...
> (XEN) HVM1:  - Lo data: 000ea020-000ea04f
> (XEN) HVM1:  - Hi data: fc002c00-fc00eb4f
> (XEN) HVM1: vm86 TSS at fc00ec00
> (XEN) HVM1: BIOS map:
> (XEN) HVM1:  c0000-c8fff: VGA BIOS
> (XEN) HVM1:  c9000-d4fff: PCI Option ROMs
> (XEN) HVM1:  eb000-eb302: SMBIOS tables
> (XEN) HVM1:  f0000-fffff: Main BIOS
> (XEN) HVM1: E820 table:
> (XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
> (XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
> (XEN) HVM1:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
> (XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
> (XEN) HVM1:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
> (XEN) HVM1:  [04]: 00000000:00100000 - 00000000:f0000000: RAM
> (XEN) HVM1:  HOLE: 00000000:f0000000 - 00000000:fc000000
> (XEN) HVM1:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
> (XEN) HVM1:  [06]: 00000001:00000000 - 00000004:0f800000: RAM
> (XEN) HVM1: Invoking ROMBIOS ...
> (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) stdvga.c:147:d1 entering stdvga and caching modes
> (XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
> (XEN) HVM1: Bochs BIOS - build: 06/23/99
> (XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) HVM1: Options: apmbios pcibios eltorito PMM 
> (XEN) HVM1: 
> (XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (60000 MBytes)
> (XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM1: ata0  slave: QEMU HARDDISK ATA-7 Hard-Disk (  97 GBytes)
> (XEN) HVM1: ata1-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM1: ata1 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes)
> (XEN) HVM1: IDE time out
> (XEN) HVM1: 
> (XEN) HVM1: PCI device 1000:0070 not found at index 0
> (XEN) HVM1: PCI device 1000:0072 not found at index 2
> (XEN) HVM1: PCI device 1000:0074 not found at index 0
> (XEN) HVM1: PCI device 1000:0076 not found at index 0
> (XEN) HVM1: PCI device 1000:0077 not found at index 0
> (XEN) HVM1: PCI device 1000:0064 not found at index 0
> (XEN) HVM1: PCI device 1000:0065 not found at index 0
> (XEN) HVM1: PCI device 1000:0080 not found at index 0
> (XEN) HVM1: PCI device 1000:0081 not found at index 0
> (XEN) HVM1: PCI device 1000:0082 not found at index 0
> (XEN) HVM1: PCI device 1000:0083 not found at index 0
> (XEN) HVM1: PCI device 1000:0084 not found at index 0
> (XEN) HVM1: PCI device 1000:0085 not found at index 0
> (XEN) HVM1: PCI device 1000:0086 not found at index 0
> (XEN) HVM1: PCI device 1000:0087 not found at index 0
> (XEN) HVM1: PCI device 1000:0070 not found at index 0
> (XEN) HVM1: PCI device 1000:0072 not found at index 2
> (XEN) HVM1: PCI device 1000:0074 not found at index 0
> (XEN) HVM1: PCI device 1000:0076 not found at index 0
> (XEN) HVM1: PCI device 1000:0077 not found at index 0
> (XEN) HVM1: PCI device 1000:0064 not found at index 0
> (XEN) HVM1: PCI device 1000:0065 not found at index 0
> (XEN) HVM1: PCI device 1000:0080 not found at index 0
> (XEN) HVM1: PCI device 1000:0081 not found at index 0
> (XEN) HVM1: PCI device 1000:0082 not found at index 0
> (XEN) HVM1: PCI device 1000:0083 not found at index 0
> (XEN) HVM1: PCI device 1000:0084 not found at index 0
> (XEN) HVM1: PCI device 1000:0085 not found at index 0
> (XEN) HVM1: PCI device 1000:0086 not found at index 0
> (XEN) HVM1: PCI device 1000:0087 not found at index 0
> (XEN) HVM1: 
> (XEN) HVM1: 
> (XEN) HVM1: Press F12 for boot menu.
> (XEN) HVM1: 
> (XEN) HVM1: Booting from Hard Disk...
> (XEN) HVM1: Booting from 0000:7c00
> (XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported!
> (XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported!
> (XEN) HVM1: KBD: unsupported int 16h function 03
> (XEN) HVM1: *** int 15h function AX=e980, BX=0000 not yet supported!
> (XEN) irq.c:330: Dom1 callback via changed to Direct Vector 0xe9
> (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c100 f_mport=9000 np=100
> (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c100 f_mport=9000 np=100
> (XEN) domctl.c:1065:d0 ioport_map:remove f_gport=c200 f_mport=d000 np=100
> (XEN) domctl.c:1041:d0 ioport_map:add f_gport=c200 f_mport=d000 np=100
> (XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0
> (XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0
> (XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0
> (XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0
> ------------[ cut here ]------------
> WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
> Hardware name: SUN FIRE X4370 M2 SERVER       
> remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'
> Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage
  shpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]
> Pid: 19329, comm: qemu-dm Not tainted 2.6.39-200.1.15.el5uek #1
> Call Trace:
>  [<ffffffff811d10dd>] ? remove_proc_entry+0x22d/0x240
>  [<ffffffff8106de10>] warn_slowpath_common+0x90/0xc0
>  [<ffffffff8106df3e>] warn_slowpath_fmt+0x6e/0x70
>  [<ffffffff8100a7cf>] ? xen_restore_fl_direct_reloc+0x4/0x4
>  [<ffffffff8125cf98>] ? sprintf+0x68/0x70
>  [<ffffffff811cfef1>] ? __xlate_proc_name+0x41/0xc0
>  [<ffffffff81501b9e>] ? _raw_spin_lock+0xe/0x20
>  [<ffffffff811d10dd>] remove_proc_entry+0x22d/0x240
>  [<ffffffff8113232c>] ? zap_pte_range+0x1ec/0x400
>  [<ffffffff810d9430>] unregister_irq_proc+0xd0/0xf0
>  [<ffffffff810d571a>] free_desc+0x2a/0x70
>  [<ffffffff810d579e>] irq_free_descs+0x3e/0x80
>  [<ffffffff812f7822>] xen_free_irq+0x52/0x70
>  [<ffffffff812f82cb>] unbind_from_irq+0xeb/0x180
>  [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
>  [<ffffffff812f837f>] evtchn_put+0x1f/0x40
>  [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
>  [<ffffffff812640e1>] ? list_del+0x11/0x40
>  [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
>  [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
>  [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
>  [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
>  [<ffffffff8113a740>] ? do_munmap+0x240/0x280
>  [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
>  [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
> ---[ end trace fa61ba514bac93c7 ]---
> BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
> IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
> PGD 6a2cb067 PUD 71d8c067 PMD 0 
> Oops: 0000 [#1] SMP 
> CPU 0 
> Modules linked in: xt_physdev iptable_filter ip_tables xen_blkback xen_netback xen_pciback xen_gntdev xen_evtchn ipmi_devintf ipmi_si lockd sunrpc bridge stp llc bonding dm_round_robin dm_multipath be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 libiscsi_tcp libiscsi scsi_transport_iscsi xenfs xen_privcmd dm_mirror video sbs sbshc acpi_memhotplug acpi_ipmi ipmi_msghandler parport_pc lp parport ses enclosure ixgbe igb e1000e mdio snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801 i2c_core i7core_edac iTCO_wdt pcspkr ioatdma iTCO_vendor_support edac_core dca ghes hed dm_region_hash dm_log dm_mod usb_storage
  shpchp sg ahci libahci sd_mod crc_t10dif raid1 ext3 jbd mbcache [last unloaded: raid_class]
> 
> Pid: 19329, comm: qemu-dm Tainted: G        W   2.6.39-200.1.15.el5uek #1 ORACLE CORPORATION SUN FIRE X4370 M2 SERVER       /ASSY,SC_BD,T4         
> RIP: e030:[<ffffffff812f6a76>]  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
> RSP: e02b:ffff88006cfc5dd8  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88007e38be48
> RDX: 0000000000000000 RSI: 0000000000000218 RDI: ffff88007e38bd70
> RBP: ffff88006cfc5dd8 R08: 0000000000000001 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000218
> R13: 0000000000000000 R14: 0000000000000017 R15: 0000000000104701
> FS:  00007ff36f4046e0(0000) GS:ffff88007f49a000(0000) knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 000000000000001c CR3: 000000005f445000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process qemu-dm (pid: 19329, threadinfo ffff88006cfc4000, task ffff880067084200)
> Stack:
>  ffff88006cfc5e38 ffffffff812f81fc 00007ff36e620000 0000003e38e10000
>  ffff88006cfc5e38 000000000000014f 0000000000000206 ffff88006b2e8138
>  ffffffff81009f0d ffff88006b951cc0 ffff88006b951cc0 0000000000000000
> Call Trace:
>  [<ffffffff812f81fc>] unbind_from_irq+0x1c/0x180
>  [<ffffffff81009f0d>] ? xen_force_evtchn_callback+0xd/0x10
>  [<ffffffff812f837f>] evtchn_put+0x1f/0x40
>  [<ffffffffa055b4e9>] gntdev_put_map+0x39/0x110 [xen_gntdev]
>  [<ffffffff812640e1>] ? list_del+0x11/0x40
>  [<ffffffffa055c38c>] gntdev_ioctl_unmap_grant_ref+0xac/0xd0 [xen_gntdev]
>  [<ffffffffa055c708>] gntdev_ioctl+0x98/0xa0 [xen_gntdev]
>  [<ffffffff8117eb8d>] vfs_ioctl+0x1d/0x50
>  [<ffffffff8117f623>] do_vfs_ioctl+0x63/0x1b0
>  [<ffffffff8113a740>] ? do_munmap+0x240/0x280
>  [<ffffffff8117f804>] sys_ioctl+0x94/0xa0
>  [<ffffffff8150a242>] system_call_fastpath+0x16/0x1b
> Code: 8b 64 24 08 c9 c3 0f 1f 80 00 00 00 00 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 39 3d f1 16 4c 00 76 0b e8 ea fa ff ff <0f> b7 40 1c c9 c3 89 f9 31 c0 48 c7 c2 2f 72 70 81 be d1 00 00 
> RIP  [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40
>  RSP <ffff88006cfc5dd8>
> CR2: 000000000000001c
> 
> Thanks,
> Joe
> On 02/05/13 17:42, Ian Campbell wrote:
>> On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:
>>> execute xl-destroy twice crashed my server!
>>
>> Can you give more details please.
>>
>> Ian.
>>
>>
>>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-16  5:38       ` Joe Jin
@ 2013-02-18 10:33         ` Ian Campbell
  0 siblings, 0 replies; 10+ messages in thread
From: Ian Campbell @ 2013-02-18 10:33 UTC (permalink / raw)
  To: Joe Jin; +Cc: xen-devel, Roger Pau Monne, Konrad Rzeszutek Wilk

On Sat, 2013-02-16 at 05:38 +0000, Joe Jin wrote:
> On 02/05/13 17:42, Ian Campbell wrote:
> > On Tue, 2013-02-05 at 06:47 +0000, Joe Jin wrote:
> >> > execute xl-destroy twice crashed my server!
> > Can you give more details please.
> > 
> Hi Ian,
> 
> Sorry the server not available for test now.

OK, please let us know if/when you can gather details. Oh I see you
already have, thanks!

> Would you please help to review my patch for xm-destroy?

xend is deprecated and I'm afraid I don't have many spare cycles to
spend on it. I think really it needs to be down to those who want to
keep xend alive to review each others patches.

Ian.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-17  5:07         ` Joe Jin
@ 2013-02-18 10:39           ` Ian Campbell
  0 siblings, 0 replies; 10+ messages in thread
From: Ian Campbell @ 2013-02-18 10:39 UTC (permalink / raw)
  To: Joe Jin; +Cc: xen-devel, Roger Pau Monne, Konrad Rzeszutek Wilk

On Sun, 2013-02-17 at 05:07 +0000, Joe Jin wrote:
> Execute "xl-destroy" twice lead the dom0 panic:
> config file as below:

Thanks.

> kernel BUG at drivers/pci/access.c:423!
[...]
> BTW: xm-create work fine but xl-create trigger the panic as below:

Both the panic above and the warn+panic below seem like kernel bugs
which are simply exposed by xl rather than xl bugs as such. Presumably
xl just tickles things different to xm.

The dom0 kernel you are using is a bit dated and appears (from the
revision) to be rather heavily patched.

> > WARNING: at fs/proc/generic.c:850 remove_proc_entry+0x22d/0x240()
> > Hardware name: SUN FIRE X4370 M2 SERVER
> > remove_proc_entry: removing non-empty directory 'irq/536', leaking at least 'eth7'
[..]
> > BUG: unable to handle kernel NULL pointer dereference at 000000000000001c
> > IP: [<ffffffff812f6a76>] evtchn_from_irq+0x16/0x40

Wei recently posted a bug fix for an issue which looked a bit like this
last one. That was something do with error handling on allocation
failure though so I doubt it represents the actual root cause though.

Ian.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] xend: disallow multiple destroy() call
  2013-02-04  1:54 [PATCH] xend: disallow multiple destroy() call Joe Jin
  2013-02-04  9:52 ` Roger Pau Monné
@ 2013-03-19 17:41 ` Konrad Rzeszutek Wilk
  1 sibling, 0 replies; 10+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-03-19 17:41 UTC (permalink / raw)
  To: Joe Jin; +Cc: Ian Campbell, xen-devel

On Mon, Feb 04, 2013 at 09:54:23AM +0800, Joe Jin wrote:
> Current Xend allowing multiple call destroy() for same domain, this lead
> multiple hard resets(FLR) for pci pass-through, and some controller might
> failed.
> 
> In our test, we pass through 2 LSI HAB controllers to the PVHVM guest, after
> guest brought up, call xm-destroy twice, the adapters's BIOS will hung, and
> we had to reboot the server to recovery it.
> 
> Signed-off-by: Joe Jin <joe.jin@oracle.com>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

The other bugs (the kernel ones) are a nice find and should also
be fixed - but I think with the upstream kernel they have been
mostly fixed.

> Cc: Ian Campbell <ian.campbell@citrix.com>
> ---
>  tools/python/xen/xend/XendDomainInfo.py | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/tools/python/xen/xend/XendDomainInfo.py b/tools/python/xen/xend/XendDomainInfo.py
> index e9d3e7e..658c3b6 100644
> --- a/tools/python/xen/xend/XendDomainInfo.py
> +++ b/tools/python/xen/xend/XendDomainInfo.py
> @@ -367,6 +367,8 @@ class XendDomainInfo:
>      @type refresh_shutdown_lock: threading.Condition
>      @ivar _deviceControllers: device controller cache for this domain
>      @type _deviceControllers: dict 'string' to DevControllers
> +    @ivar destroying: Is this domain destroying
> +    @type destroying: bool
>      """
>      
>      def __init__(self, info, domid = None, dompath = None, augment = False,
> @@ -455,6 +457,8 @@ class XendDomainInfo:
>          self._checkName(self.info['name_label'])
>  
>          self.metrics = XendVMMetrics(uuid.createString(), self)
> +
> +        self.destroying = False
>              
>  
>      #
> @@ -3073,6 +3077,12 @@ class XendDomainInfo:
>  
>          if self.domid is None:
>              return
> +
> +        if self.destroying == False:
> +            self.destroying = True
> +        else:
> +            raise VmError("Domain (domid=%s) is destroying, please wait!", str(self.domid))
> +
>          from xen.xend import XendDomain
>          log.debug("XendDomainInfo.destroy: domid=%s", str(self.domid))
>  
> @@ -3088,6 +3098,7 @@ class XendDomainInfo:
>                      self.info[state] = 0
>                  self._stateSet(DOM_STATE_HALTED)
>              except:
> +                self.destroying = False
>                  log.exception("XendDomainInfo.destroy: domain destruction failed.")
>  
>              XendDomain.instance().remove_domain(self)
> -- 
> 1.8.1
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2013-03-19 17:41 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-02-04  1:54 [PATCH] xend: disallow multiple destroy() call Joe Jin
2013-02-04  9:52 ` Roger Pau Monné
2013-02-05  6:47   ` Joe Jin
2013-02-05  9:42     ` Ian Campbell
2013-02-16  5:38       ` Joe Jin
2013-02-18 10:33         ` Ian Campbell
2013-02-17  4:04       ` Joe Jin
2013-02-17  5:07         ` Joe Jin
2013-02-18 10:39           ` Ian Campbell
2013-03-19 17:41 ` Konrad Rzeszutek Wilk

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.