From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752480AbdB1ODU (ORCPT ); Tue, 28 Feb 2017 09:03:20 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:52920 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751441AbdB1ODT (ORCPT ); Tue, 28 Feb 2017 09:03:19 -0500 Date: Tue, 28 Feb 2017 12:57:29 +0100 From: Heiko Carstens To: Michal Hocko , Vladimir Davydov Cc: Sebastian Ott , Dan Williams , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , "Rafael J. Wysocki" Subject: Re: [PATCH] mm, add_memory_resource: hold device_hotplug lock over mem_hotplug_{begin, done} References: <20170227162031.GA27937@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170227162031.GA27937@dhcp22.suse.cz> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17022811-0036-0000-0000-000003755DFC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17022811-0037-0000-0000-00001543CE24 Message-Id: <20170228115729.GB13872@osiris> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-02-28_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=14 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1702280111 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Feb 27, 2017 at 05:20:31PM +0100, Michal Hocko wrote: > [CC Rafael] > > I've got lost in the acpi indirection (again). I can see > acpi_device_hotplug calling lock_device_hotplug() but i cannot find a > path down to add_memory() which might call add_memory_resource. But the > patch below sounds suspicious to me. Is it possible that this could lead > to a deadlock. I would suspect that it is the s390 code which needs to > do the locking. But I would have to double check - it is really easy to > get lost there. To me it rather looks like bfc8c90139eb ("mem-hotplug: implement get/put_online_mems") introduced quite subtle and probably wrong locking rules. The patch introduced mem_hotplug_begin() in order to have something like cpu_hotplug_begin() for memory. Note that for cpu hotplug all cpu_hotplug_begin() calls are serialized by cpu_maps_update_begin(). Especially this makes sure that active_writer can only be changed by one process. (See also Dan's commit which introduced the lock_device_hotplug() calls: https://marc.info/?l=linux-kernel&m=148693912419972&w=2 ) If you look at the above commit bfc8c90139eb: there is nothing like cpu_maps_update_begin() for memory. And therefore it's possible to have concurrent writers to active_writer. It looks like now lock_device_hotplug() is supposed to be the new cpu_maps_update_begin() for memory. But.. this looks like a mess, unless I read the code completely wrong ;) > On Sun 26-02-17 12:42:44, Sebastian Ott wrote: > > With 4.10.0-10265-gc4f3f22 the following warning is triggered on s390: > > > > WARNING: CPU: 6 PID: 1 at drivers/base/core.c:643 assert_held_device_hotplug+0x4a/0x58 > > [ 5.731214] Call Trace: > > [ 5.731219] ([<000000000067b8b0>] assert_held_device_hotplug+0x40/0x58) > > [ 5.731225] [<0000000000337914>] mem_hotplug_begin+0x34/0xc8 > > [ 5.731231] [<00000000008b897e>] add_memory_resource+0x7e/0x1f8 > > [ 5.731236] [<00000000008b8bd2>] add_memory+0xda/0x130 > > [ 5.731243] [<0000000000d7f0dc>] add_memory_merged+0x15c/0x178 > > [ 5.731247] [<0000000000d7f3a6>] sclp_detect_standby_memory+0x2ae/0x2f8 > > [ 5.731252] [<00000000001002ba>] do_one_initcall+0xa2/0x150 > > [ 5.731258] [<0000000000d3adc0>] kernel_init_freeable+0x228/0x2d8 > > [ 5.731263] [<00000000008b6572>] kernel_init+0x2a/0x140 > > [ 5.731267] [<00000000008c3972>] kernel_thread_starter+0x6/0xc > > [ 5.731272] [<00000000008c396c>] kernel_thread_starter+0x0/0xc > > [ 5.731276] no locks held by swapper/0/1. > > [ 5.731280] Last Breaking-Event-Address: > > [ 5.731285] [<000000000067b8b6>] assert_held_device_hotplug+0x46/0x58 > > [ 5.731292] ---[ end trace 46480df21194c96a ]--- > > such an informtion belongs to the changelog > > > ----->8 > > mm, add_memory_resource: hold device_hotplug lock over mem_hotplug_{begin, done} > > > > With commit 3fc219241 ("mm: validate device_hotplug is held for memory hotplug") > > a lock assertion was added to mem_hotplug_begin() which led to a warning > > when add_memory() is called. Fix this by acquiring device_hotplug_lock in > > add_memory_resource(). > > > > Signed-off-by: Sebastian Ott > > --- > > mm/memory_hotplug.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > > index 1d3ed58..c633bbc 100644 > > --- a/mm/memory_hotplug.c > > +++ b/mm/memory_hotplug.c > > @@ -1361,6 +1361,7 @@ int __ref add_memory_resource(int nid, struct resource *res, bool online) > > new_pgdat = !p; > > } > > > > + lock_device_hotplug(); > > mem_hotplug_begin(); > > > > /* > > @@ -1416,6 +1417,7 @@ int __ref add_memory_resource(int nid, struct resource *res, bool online) > > > > out: > > mem_hotplug_done(); > > + unlock_device_hotplug(); > > return ret; > > } > > EXPORT_SYMBOL_GPL(add_memory_resource); > > -- > > 2.3.0 > > > > -- > > To unsubscribe, send a message with 'unsubscribe linux-mm' in > > the body to majordomo@kvack.org. For more info on Linux MM, > > see: http://www.linux-mm.org/ . > > Don't email: email@kvack.org > > -- > Michal Hocko > SUSE Labs > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 7DF3D6B039D for ; Tue, 28 Feb 2017 06:57:41 -0500 (EST) Received: by mail-pf0-f199.google.com with SMTP id x66so12127381pfb.2 for ; Tue, 28 Feb 2017 03:57:41 -0800 (PST) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id r19si1600805pfe.12.2017.02.28.03.57.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Feb 2017 03:57:40 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v1SBs3QB038249 for ; Tue, 28 Feb 2017 06:57:39 -0500 Received: from e06smtp09.uk.ibm.com (e06smtp09.uk.ibm.com [195.75.94.105]) by mx0b-001b2d01.pphosted.com with ESMTP id 28vqj1mqrh-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 28 Feb 2017 06:57:39 -0500 Received: from localhost by e06smtp09.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 28 Feb 2017 11:57:36 -0000 Date: Tue, 28 Feb 2017 12:57:29 +0100 From: Heiko Carstens Subject: Re: [PATCH] mm, add_memory_resource: hold device_hotplug lock over mem_hotplug_{begin, done} References: <20170227162031.GA27937@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170227162031.GA27937@dhcp22.suse.cz> Message-Id: <20170228115729.GB13872@osiris> Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko , Vladimir Davydov Cc: Sebastian Ott , Dan Williams , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , "Rafael J. Wysocki" On Mon, Feb 27, 2017 at 05:20:31PM +0100, Michal Hocko wrote: > [CC Rafael] > > I've got lost in the acpi indirection (again). I can see > acpi_device_hotplug calling lock_device_hotplug() but i cannot find a > path down to add_memory() which might call add_memory_resource. But the > patch below sounds suspicious to me. Is it possible that this could lead > to a deadlock. I would suspect that it is the s390 code which needs to > do the locking. But I would have to double check - it is really easy to > get lost there. To me it rather looks like bfc8c90139eb ("mem-hotplug: implement get/put_online_mems") introduced quite subtle and probably wrong locking rules. The patch introduced mem_hotplug_begin() in order to have something like cpu_hotplug_begin() for memory. Note that for cpu hotplug all cpu_hotplug_begin() calls are serialized by cpu_maps_update_begin(). Especially this makes sure that active_writer can only be changed by one process. (See also Dan's commit which introduced the lock_device_hotplug() calls: https://marc.info/?l=linux-kernel&m=148693912419972&w=2 ) If you look at the above commit bfc8c90139eb: there is nothing like cpu_maps_update_begin() for memory. And therefore it's possible to have concurrent writers to active_writer. It looks like now lock_device_hotplug() is supposed to be the new cpu_maps_update_begin() for memory. But.. this looks like a mess, unless I read the code completely wrong ;) > On Sun 26-02-17 12:42:44, Sebastian Ott wrote: > > With 4.10.0-10265-gc4f3f22 the following warning is triggered on s390: > > > > WARNING: CPU: 6 PID: 1 at drivers/base/core.c:643 assert_held_device_hotplug+0x4a/0x58 > > [ 5.731214] Call Trace: > > [ 5.731219] ([<000000000067b8b0>] assert_held_device_hotplug+0x40/0x58) > > [ 5.731225] [<0000000000337914>] mem_hotplug_begin+0x34/0xc8 > > [ 5.731231] [<00000000008b897e>] add_memory_resource+0x7e/0x1f8 > > [ 5.731236] [<00000000008b8bd2>] add_memory+0xda/0x130 > > [ 5.731243] [<0000000000d7f0dc>] add_memory_merged+0x15c/0x178 > > [ 5.731247] [<0000000000d7f3a6>] sclp_detect_standby_memory+0x2ae/0x2f8 > > [ 5.731252] [<00000000001002ba>] do_one_initcall+0xa2/0x150 > > [ 5.731258] [<0000000000d3adc0>] kernel_init_freeable+0x228/0x2d8 > > [ 5.731263] [<00000000008b6572>] kernel_init+0x2a/0x140 > > [ 5.731267] [<00000000008c3972>] kernel_thread_starter+0x6/0xc > > [ 5.731272] [<00000000008c396c>] kernel_thread_starter+0x0/0xc > > [ 5.731276] no locks held by swapper/0/1. > > [ 5.731280] Last Breaking-Event-Address: > > [ 5.731285] [<000000000067b8b6>] assert_held_device_hotplug+0x46/0x58 > > [ 5.731292] ---[ end trace 46480df21194c96a ]--- > > such an informtion belongs to the changelog > > > ----->8 > > mm, add_memory_resource: hold device_hotplug lock over mem_hotplug_{begin, done} > > > > With commit 3fc219241 ("mm: validate device_hotplug is held for memory hotplug") > > a lock assertion was added to mem_hotplug_begin() which led to a warning > > when add_memory() is called. Fix this by acquiring device_hotplug_lock in > > add_memory_resource(). > > > > Signed-off-by: Sebastian Ott > > --- > > mm/memory_hotplug.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > > index 1d3ed58..c633bbc 100644 > > --- a/mm/memory_hotplug.c > > +++ b/mm/memory_hotplug.c > > @@ -1361,6 +1361,7 @@ int __ref add_memory_resource(int nid, struct resource *res, bool online) > > new_pgdat = !p; > > } > > > > + lock_device_hotplug(); > > mem_hotplug_begin(); > > > > /* > > @@ -1416,6 +1417,7 @@ int __ref add_memory_resource(int nid, struct resource *res, bool online) > > > > out: > > mem_hotplug_done(); > > + unlock_device_hotplug(); > > return ret; > > } > > EXPORT_SYMBOL_GPL(add_memory_resource); > > -- > > 2.3.0 > > > > -- > > To unsubscribe, send a message with 'unsubscribe linux-mm' in > > the body to majordomo@kvack.org. For more info on Linux MM, > > see: http://www.linux-mm.org/ . > > Don't email: email@kvack.org > > -- > Michal Hocko > SUSE Labs > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org