From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758415Ab2ECUD0 (ORCPT ); Thu, 3 May 2012 16:03:26 -0400 Received: from mail-we0-f174.google.com ([74.125.82.174]:35205 "EHLO mail-we0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754586Ab2ECUDY (ORCPT ); Thu, 3 May 2012 16:03:24 -0400 Date: Thu, 3 May 2012 23:02:59 +0300 From: Sergey Senozhatsky To: Avi Kivity Cc: Marcelo Tosatti , "Paul E. McKenney" , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: possible circular locking dependency Message-ID: <20120503200259.GA3524@swordfish.minsk.epam.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, 3.4-rc5 [32881.212463] kvm: VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL does not work properly. Using workaround [32882.360505] [32882.360509] ====================================================== [32882.360511] [ INFO: possible circular locking dependency detected ] [32882.360515] 3.4.0-rc5-dbg-00932-gfabccd4-dirty #1107 Not tainted [32882.360517] ------------------------------------------------------- [32882.360519] qemu-system-x86/15168 is trying to acquire lock: [32882.360521] (cpu_hotplug.lock){+.+.+.}, at: [] get_online_cpus+0x41/0x55 [32882.360532] [32882.360532] but task is already holding lock: [32882.360534] (&sp->mutex){+.+...}, at: [] __synchronize_srcu+0x9a/0x104 [32882.360542] [32882.360542] which lock already depends on the new lock. [32882.360543] [32882.360545] [32882.360545] the existing dependency chain (in reverse order) is: [32882.360547] [32882.360547] -> #3 (&sp->mutex){+.+...}: [32882.360552] [] lock_acquire+0x148/0x1c3 [32882.360557] [] mutex_lock_nested+0x6e/0x2d1 [32882.360562] [] __synchronize_srcu+0x9a/0x104 [32882.360566] [] synchronize_srcu+0x15/0x17 [32882.360569] [] srcu_notifier_chain_unregister+0x5b/0x69 [32882.360573] [] cpufreq_unregister_notifier+0x22/0x3c [32882.360580] [] cpufreq_governor_dbs+0x322/0x3ac [32882.360584] [] __cpufreq_governor+0x6b/0xa8 [32882.360587] [] __cpufreq_set_policy+0xf3/0x145 [32882.360591] [] store_scaling_governor+0x173/0x1a9 [32882.360594] [] store+0x5a/0x86 [32882.360597] [] sysfs_write_file+0xee/0x126 [32882.360603] [] vfs_write+0xa3/0x14c [32882.360607] [] sys_write+0x43/0x73 [32882.360610] [] system_call_fastpath+0x1a/0x1f [32882.360614] [32882.360615] -> #2 (dbs_mutex){+.+.+.}: [32882.360619] [] lock_acquire+0x148/0x1c3 [32882.360622] [] mutex_lock_nested+0x6e/0x2d1 [32882.360625] [] cpufreq_governor_dbs+0x7c/0x3ac [32882.360629] [] __cpufreq_governor+0x6b/0xa8 [32882.360632] [] __cpufreq_set_policy+0x109/0x145 [32882.360636] [] cpufreq_add_dev_interface+0x257/0x288 [32882.360639] [] cpufreq_add_dev+0x40a/0x42a [32882.360643] [] subsys_interface_register+0x9b/0xdc [32882.360648] [] cpufreq_register_driver+0xa0/0x14b [32882.360652] [] store_up_threshold+0x3a/0x50 [cpufreq_ondemand] [32882.360657] [] do_one_initcall+0x7f/0x140 [32882.360663] [] sys_init_module+0x1818/0x1aec [32882.360667] [] system_call_fastpath+0x1a/0x1f [32882.360671] [32882.360671] -> #1 (&per_cpu(cpu_policy_rwsem, cpu)){+++++.}: [32882.360675] [] lock_acquire+0x148/0x1c3 [32882.360679] [] down_write+0x49/0x6c [32882.360682] [] lock_policy_rwsem_write+0x47/0x78 [32882.360685] [] cpufreq_cpu_callback+0x57/0x81 [32882.360692] [] notifier_call_chain+0xac/0xd9 [32882.360697] [] __raw_notifier_call_chain+0xe/0x10 [32882.360701] [] __cpu_notify+0x20/0x37 [32882.360705] [] _cpu_down+0x7b/0x25d [32882.360709] [] disable_nonboot_cpus+0x5f/0x10b [32882.360712] [] suspend_devices_and_enter+0x197/0x401 [32882.360719] [] pm_suspend+0x104/0x1bd [32882.360722] [] state_store+0xa0/0xc9 [32882.360726] [] kobj_attr_store+0xf/0x1b [32882.360730] [] sysfs_write_file+0xee/0x126 [32882.360733] [] vfs_write+0xa3/0x14c [32882.360736] [] sys_write+0x43/0x73 [32882.360739] [] system_call_fastpath+0x1a/0x1f [32882.360743] [32882.360743] -> #0 (cpu_hotplug.lock){+.+.+.}: [32882.360747] [] __lock_acquire+0xf6b/0x1612 [32882.360751] [] lock_acquire+0x148/0x1c3 [32882.360754] [] mutex_lock_nested+0x6e/0x2d1 [32882.360757] [] get_online_cpus+0x41/0x55 [32882.360760] [] synchronize_sched_expedited+0x26/0xfa [32882.360766] [] __synchronize_srcu+0xa8/0x104 [32882.360769] [] synchronize_srcu_expedited+0x15/0x17 [32882.360773] [] __kvm_set_memory_region+0x3d8/0x46a [kvm] [32882.360789] [] kvm_set_memory_region+0x37/0x50 [kvm] [32882.360798] [] vmx_set_tss_addr+0x4c/0x200 [kvm_intel] [32882.360803] [] kvm_arch_vm_ioctl+0x160/0x9df [kvm] [32882.360816] [] kvm_vm_ioctl+0x36a/0x39c [kvm] [32882.360825] [] vfs_ioctl+0x24/0x2f [32882.360829] [] do_vfs_ioctl+0x412/0x455 [32882.360832] [] sys_ioctl+0x56/0x7b [32882.360835] [] system_call_fastpath+0x1a/0x1f [32882.360839] [32882.360839] other info that might help us debug this: [32882.360840] [32882.360842] Chain exists of: [32882.360842] cpu_hotplug.lock --> dbs_mutex --> &sp->mutex [32882.360847] [32882.360848] Possible unsafe locking scenario: [32882.360849] [32882.360851] CPU0 CPU1 [32882.360852] ---- ---- [32882.360854] lock(&sp->mutex); [32882.360856] lock(dbs_mutex); [32882.360859] lock(&sp->mutex); [32882.360862] lock(cpu_hotplug.lock); [32882.360865] [32882.360865] *** DEADLOCK *** [32882.360866] [32882.360868] 2 locks held by qemu-system-x86/15168: [32882.360870] #0: (&kvm->slots_lock){+.+.+.}, at: [] kvm_set_memory_region+0x29/0x50 [kvm] [32882.360882] #1: (&sp->mutex){+.+...}, at: [] __synchronize_srcu+0x9a/0x104 [32882.360888] [32882.360889] stack backtrace: [32882.360892] Pid: 15168, comm: qemu-system-x86 Not tainted 3.4.0-rc5-dbg-00932-gfabccd4-dirty #1107 [32882.360894] Call Trace: [32882.360898] [] print_circular_bug+0x29f/0x2b0 [32882.360901] [] __lock_acquire+0xf6b/0x1612 [32882.360905] [] lock_acquire+0x148/0x1c3 [32882.360908] [] ? get_online_cpus+0x41/0x55 [32882.360911] [] mutex_lock_nested+0x6e/0x2d1 [32882.360914] [] ? get_online_cpus+0x41/0x55 [32882.360917] [] ? synchronize_sched+0xa8/0xa8 [32882.360921] [] get_online_cpus+0x41/0x55 [32882.360923] [] synchronize_sched_expedited+0x26/0xfa [32882.360927] [] ? synchronize_sched+0xa8/0xa8 [32882.360930] [] __synchronize_srcu+0xa8/0x104 [32882.360933] [] synchronize_srcu_expedited+0x15/0x17 [32882.360942] [] __kvm_set_memory_region+0x3d8/0x46a [kvm] [32882.360945] [] ? mark_held_locks+0xbe/0xea [32882.360954] [] kvm_set_memory_region+0x37/0x50 [kvm] [32882.360959] [] vmx_set_tss_addr+0x4c/0x200 [kvm_intel] [32882.360971] [] kvm_arch_vm_ioctl+0x160/0x9df [kvm] [32882.360980] [] kvm_vm_ioctl+0x36a/0x39c [kvm] [32882.360984] [] ? lock_release_non_nested+0x8b/0x241 [32882.360987] [] vfs_ioctl+0x24/0x2f [32882.360990] [] do_vfs_ioctl+0x412/0x455 [32882.360993] [] ? fget_light+0x120/0x39b [32882.360996] [] sys_ioctl+0x56/0x7b [32882.360999] [] system_call_fastpath+0x1a/0x1f -ss