From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753273Ab2HMVVw (ORCPT ); Mon, 13 Aug 2012 17:21:52 -0400 Received: from mail-we0-f202.google.com ([74.125.82.202]:59487 "EHLO mail-we0-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752099Ab2HMVV3 (ORCPT ); Mon, 13 Aug 2012 17:21:29 -0400 From: Greg Thelen To: Glauber Costa Cc: , , , , Michal Hocko , Johannes Weiner , Andrew Morton , , Christoph Lameter , David Rientjes , Pekka Enberg , Pekka Enberg Subject: Re: [PATCH v2 06/11] memcg: kmem controller infrastructure References: <1344517279-30646-1-git-send-email-glommer@parallels.com> <1344517279-30646-7-git-send-email-glommer@parallels.com> <5028D016.1030902@parallels.com> Date: Mon, 13 Aug 2012 14:21:19 -0700 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 13 2012, Glauber Costa wrote: >> >> Here's the dmesg splat. >> > > Do you always get this report in the same way? > I managed to get a softirq inconsistency like yours, but the complaint > goes for a different lock. Yes, I repeatedly get the same dmesg splat below. Once I your 'execute the whole memcg freeing in rcu callback' patch, then the warnings are not printed. I'll take a closer look at the patch soon. >> [ 335.550398] ================================= >> [ 335.554739] [ INFO: inconsistent lock state ] >> [ 335.559091] 3.5.0-dbg-DEV #3 Tainted: G W >> [ 335.563946] --------------------------------- >> [ 335.568290] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. >> [ 335.574286] swapper/10/0 [HC0[0]:SC1[1]:HE1:SE0] takes: >> [ 335.579508] (&(&rtpz->lock)->rlock){+.?...}, at: [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.588525] {SOFTIRQ-ON-W} state was registered at: >> [ 335.593389] [] __lock_acquire+0x623/0x1a50 >> [ 335.599200] [] lock_acquire+0x95/0x150 >> [ 335.604670] [] _raw_spin_lock+0x41/0x50 >> [ 335.610232] [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.616135] [] mem_cgroup_put+0x45/0x50 >> [ 335.621696] [] mem_cgroup_destroy+0x22/0x30 >> [ 335.627592] [] cgroup_diput+0xbf/0x160 >> [ 335.633062] [] d_delete+0x12f/0x1a0 >> [ 335.638276] [] vfs_rmdir+0x11e/0x140 >> [ 335.643565] [] do_rmdir+0x113/0x130 >> [ 335.648773] [] sys_rmdir+0x16/0x20 >> [ 335.653900] [] cstar_dispatch+0x7/0x1f >> [ 335.659370] irq event stamp: 399732 >> [ 335.662846] hardirqs last enabled at (399732): [] res_counter_uncharge_until+0x68/0xa0 >> [ 335.672383] hardirqs last disabled at (399731): [] res_counter_uncharge_until+0x28/0xa0 >> [ 335.681916] softirqs last enabled at (399710): [] _local_bh_enable+0x13/0x20 >> [ 335.690590] softirqs last disabled at (399711): [] call_softirq+0x1c/0x30 >> [ 335.698914] >> [ 335.698914] other info that might help us debug this: >> [ 335.705415] Possible unsafe locking scenario: >> [ 335.705415] >> [ 335.711317] CPU0 >> [ 335.713757] ---- >> [ 335.716198] lock(&(&rtpz->lock)->rlock); >> [ 335.720282] >> [ 335.722896] lock(&(&rtpz->lock)->rlock); >> [ 335.727153] >> [ 335.727153] *** DEADLOCK *** >> [ 335.727153] >> [ 335.733055] no locks held by swapper/10/0. >> [ 335.737141] >> [ 335.737141] stack backtrace: >> [ 335.741483] Pid: 0, comm: swapper/10 Tainted: G W 3.5.0-dbg-DEV #3 >> [ 335.748510] Call Trace: >> [ 335.750952] [] print_usage_bug+0x1fc/0x20d >> [ 335.757286] [] ? save_stack_trace+0x2f/0x50 >> [ 335.763098] [] mark_lock+0x29d/0x300 >> [ 335.768309] [] ? print_irq_inversion_bug.part.36+0x1f0/0x1f0 >> [ 335.775599] [] __lock_acquire+0x5ac/0x1a50 >> [ 335.781323] [] ? __lock_acquire+0x2e4/0x1a50 >> [ 335.787224] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.793212] [] lock_acquire+0x95/0x150 >> [ 335.798594] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.804581] [] ? res_counter_uncharge_until+0x3d/0xa0 >> [ 335.811263] [] _raw_spin_lock+0x41/0x50 >> [ 335.816731] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.822724] [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.828538] [] mem_cgroup_put+0x45/0x50 >> [ 335.834002] [] __memcg_kmem_free_page+0xa6/0x110 >> [ 335.840256] [] free_accounted_pages+0x99/0xa0 >> [ 335.846243] [] free_task+0x3f/0x70 >> [ 335.851278] [] __put_task_struct+0xbc/0x130 >> [ 335.857094] [] delayed_put_task_struct+0x54/0xd0 >> [ 335.863338] [] __rcu_process_callbacks+0x1e4/0x490 >> [ 335.869757] [] rcu_process_callbacks+0x2f/0x80 >> [ 335.875835] [] __do_softirq+0xc5/0x270 >> [ 335.881218] [] ? clockevents_program_event+0x74/0x100 >> [ 335.887895] [] ? tick_program_event+0x24/0x30 >> [ 335.893882] [] call_softirq+0x1c/0x30 >> [ 335.899179] [] do_softirq+0x8d/0xc0 >> [ 335.904301] [] irq_exit+0xae/0xe0 >> [ 335.909251] [] smp_apic_timer_interrupt+0x6e/0x99 >> [ 335.915591] [] apic_timer_interrupt+0x6c/0x80 >> [ 335.921583] [] ? default_idle+0x67/0x270 >> [ 335.927741] [] ? default_idle+0x65/0x270 >> From mboxrd@z Thu Jan 1 00:00:00 1970 From: Greg Thelen Subject: Re: [PATCH v2 06/11] memcg: kmem controller infrastructure Date: Mon, 13 Aug 2012 14:21:19 -0700 Message-ID: References: <1344517279-30646-1-git-send-email-glommer@parallels.com> <1344517279-30646-7-git-send-email-glommer@parallels.com> <5028D016.1030902@parallels.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:cc:subject:references:date:message-id:user-agent :mime-version:content-type; bh=icN9rpeS69CYj/ld4045I3dnsSuJlq27MAer5TuHErc=; b=ohKmeEE/aqL/d5asyvWRl6innxhSrwc/6RL2O8mGcdugtBSd9HfkgQTYAMtAbLWCNC tdr/rMwy96NDWPyvwahVkL1EO3e88vsdZTbsbm4DQMPTmoF/aoSVhTBsNCUR3Ej5dcJs cDyQmxOOMObtzu7LwnCvqdtGw0K0Mg+DjIuv0o0mQaOeLzGVULjbfbgGIdOtky0I0Y0t V0Lek/b+K7N+x9oTI/xEtdmEYA+B525ICFj10z2QFX1MnC1m1Oh/0zS5YgEEQBmAwlaI sWfb/wJPZpCLVeVAw3kL39+SwCzwYwk3l1xz+V5Cpt1TWeN7bZKAW/2RbYKsFFYiHcoa zsdQ== Sender: owner-linux-mm@kvack.org List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Glauber Costa Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, devel@openvz.org, Michal Hocko , Johannes Weiner , Andrew Morton , kamezawa.hiroyu@jp.fujitsu.com, Christoph Lameter , David Rientjes , Pekka Enberg , Pekka Enberg On Mon, Aug 13 2012, Glauber Costa wrote: >> >> Here's the dmesg splat. >> > > Do you always get this report in the same way? > I managed to get a softirq inconsistency like yours, but the complaint > goes for a different lock. Yes, I repeatedly get the same dmesg splat below. Once I your 'execute the whole memcg freeing in rcu callback' patch, then the warnings are not printed. I'll take a closer look at the patch soon. >> [ 335.550398] ================================= >> [ 335.554739] [ INFO: inconsistent lock state ] >> [ 335.559091] 3.5.0-dbg-DEV #3 Tainted: G W >> [ 335.563946] --------------------------------- >> [ 335.568290] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. >> [ 335.574286] swapper/10/0 [HC0[0]:SC1[1]:HE1:SE0] takes: >> [ 335.579508] (&(&rtpz->lock)->rlock){+.?...}, at: [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.588525] {SOFTIRQ-ON-W} state was registered at: >> [ 335.593389] [] __lock_acquire+0x623/0x1a50 >> [ 335.599200] [] lock_acquire+0x95/0x150 >> [ 335.604670] [] _raw_spin_lock+0x41/0x50 >> [ 335.610232] [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.616135] [] mem_cgroup_put+0x45/0x50 >> [ 335.621696] [] mem_cgroup_destroy+0x22/0x30 >> [ 335.627592] [] cgroup_diput+0xbf/0x160 >> [ 335.633062] [] d_delete+0x12f/0x1a0 >> [ 335.638276] [] vfs_rmdir+0x11e/0x140 >> [ 335.643565] [] do_rmdir+0x113/0x130 >> [ 335.648773] [] sys_rmdir+0x16/0x20 >> [ 335.653900] [] cstar_dispatch+0x7/0x1f >> [ 335.659370] irq event stamp: 399732 >> [ 335.662846] hardirqs last enabled at (399732): [] res_counter_uncharge_until+0x68/0xa0 >> [ 335.672383] hardirqs last disabled at (399731): [] res_counter_uncharge_until+0x28/0xa0 >> [ 335.681916] softirqs last enabled at (399710): [] _local_bh_enable+0x13/0x20 >> [ 335.690590] softirqs last disabled at (399711): [] call_softirq+0x1c/0x30 >> [ 335.698914] >> [ 335.698914] other info that might help us debug this: >> [ 335.705415] Possible unsafe locking scenario: >> [ 335.705415] >> [ 335.711317] CPU0 >> [ 335.713757] ---- >> [ 335.716198] lock(&(&rtpz->lock)->rlock); >> [ 335.720282] >> [ 335.722896] lock(&(&rtpz->lock)->rlock); >> [ 335.727153] >> [ 335.727153] *** DEADLOCK *** >> [ 335.727153] >> [ 335.733055] no locks held by swapper/10/0. >> [ 335.737141] >> [ 335.737141] stack backtrace: >> [ 335.741483] Pid: 0, comm: swapper/10 Tainted: G W 3.5.0-dbg-DEV #3 >> [ 335.748510] Call Trace: >> [ 335.750952] [] print_usage_bug+0x1fc/0x20d >> [ 335.757286] [] ? save_stack_trace+0x2f/0x50 >> [ 335.763098] [] mark_lock+0x29d/0x300 >> [ 335.768309] [] ? print_irq_inversion_bug.part.36+0x1f0/0x1f0 >> [ 335.775599] [] __lock_acquire+0x5ac/0x1a50 >> [ 335.781323] [] ? __lock_acquire+0x2e4/0x1a50 >> [ 335.787224] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.793212] [] lock_acquire+0x95/0x150 >> [ 335.798594] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.804581] [] ? res_counter_uncharge_until+0x3d/0xa0 >> [ 335.811263] [] _raw_spin_lock+0x41/0x50 >> [ 335.816731] [] ? __mem_cgroup_free+0x8d/0x1b0 >> [ 335.822724] [] __mem_cgroup_free+0x8d/0x1b0 >> [ 335.828538] [] mem_cgroup_put+0x45/0x50 >> [ 335.834002] [] __memcg_kmem_free_page+0xa6/0x110 >> [ 335.840256] [] free_accounted_pages+0x99/0xa0 >> [ 335.846243] [] free_task+0x3f/0x70 >> [ 335.851278] [] __put_task_struct+0xbc/0x130 >> [ 335.857094] [] delayed_put_task_struct+0x54/0xd0 >> [ 335.863338] [] __rcu_process_callbacks+0x1e4/0x490 >> [ 335.869757] [] rcu_process_callbacks+0x2f/0x80 >> [ 335.875835] [] __do_softirq+0xc5/0x270 >> [ 335.881218] [] ? clockevents_program_event+0x74/0x100 >> [ 335.887895] [] ? tick_program_event+0x24/0x30 >> [ 335.893882] [] call_softirq+0x1c/0x30 >> [ 335.899179] [] do_softirq+0x8d/0xc0 >> [ 335.904301] [] irq_exit+0xae/0xe0 >> [ 335.909251] [] smp_apic_timer_interrupt+0x6e/0x99 >> [ 335.915591] [] apic_timer_interrupt+0x6c/0x80 >> [ 335.921583] [] ? default_idle+0x67/0x270 >> [ 335.927741] [] ? default_idle+0x65/0x270 >> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org