From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3BA7C433DB for ; Wed, 13 Jan 2021 13:16:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6781823434 for ; Wed, 13 Jan 2021 13:16:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6781823434 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 731A88D0051; Wed, 13 Jan 2021 08:16:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BB538D002E; Wed, 13 Jan 2021 08:16:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5836E8D0051; Wed, 13 Jan 2021 08:16:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 43F908D002E for ; Wed, 13 Jan 2021 08:16:47 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id EDFD33632 for ; Wed, 13 Jan 2021 13:16:46 +0000 (UTC) X-FDA: 77700801612.02.patch47_28168e32751e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id C186410097AA1 for ; Wed, 13 Jan 2021 13:16:46 +0000 (UTC) X-HE-Tag: patch47_28168e32751e X-Filterd-Recvd-Size: 4233 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 13:16:46 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id DD262B803; Wed, 13 Jan 2021 13:16:44 +0000 (UTC) From: Vlastimil Babka To: Andrew Morton , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vladimir Davydov , Qian Cai , David Hildenbrand , Michal Hocko , Vlastimil Babka Subject: [PATCH 3/3] mm, slab, slub: stop taking cpu hotplug lock Date: Wed, 13 Jan 2021 14:16:34 +0100 Message-Id: <20210113131634.3671-4-vbabka@suse.cz> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210113131634.3671-1-vbabka@suse.cz> References: <20210113131634.3671-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: SLAB has been using get/put_online_cpus() around creating, destroying and shrinking kmem caches since 95402b382901 ("cpu-hotplug: replace per-subsy= stem mutexes with get_online_cpus()") in 2008, which is supposed to be replaci= ng a private mutex (cache_chain_mutex, called slab_mutex today) with system-= wide mechanism, but in case of SLAB it's in fact used in addition to the exist= ing mutex, without explanation why. SLUB appears to have avoided the cpu hotplug lock initially, but gained i= t due to common code unification, such as 20cea9683ecc ("mm, sl[aou]b: Move kmem_cache_create mutex handling to common code"). Regardless of the history, checking if the hotplug lock is actually neede= d today suggests that it's not, and therefore it's better to avoid this system-wide lock and the ordering this imposes wrt other locks (such as slab_mutex). Specifically, in SLAB we have for_each_online_cpu() in do_tune_cpucache() protected by slab_mutex, and cpu hotplug callbacks that also take the slab_mutex, which is also taken by the common slab function that currentl= y also take the hotplug lock. Thus the slab_mutex protection should be sufficien= t. Also per-cpu array caches are allocated for each possible cpu, so not aff= ected by their online/offline state. In SLUB we have for_each_online_cpu() in functions that show statistics a= nd are already unprotected today, as racing with hotplug is not harmful. Otherwi= se SLUB relies on percpu allocator. The slub_cpu_dead() hotplug callback tak= es the slab_mutex. To sum up, this patch removes get/put_online_cpus() calls from slab as it should be safe without further adjustments. Signed-off-by: Vlastimil Babka --- mm/slab_common.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index e040b3820a75..0ca99ebefbf2 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -310,8 +310,6 @@ kmem_cache_create_usercopy(const char *name, const char *cache_name; int err; =20 - get_online_cpus(); - mutex_lock(&slab_mutex); =20 err =3D kmem_cache_sanity_check(name, size); @@ -360,8 +358,6 @@ kmem_cache_create_usercopy(const char *name, out_unlock: mutex_unlock(&slab_mutex); =20 - put_online_cpus(); - if (err) { if (flags & SLAB_PANIC) panic("kmem_cache_create: Failed to create slab '%s'. Error %d\n", @@ -487,8 +483,6 @@ void kmem_cache_destroy(struct kmem_cache *s) if (unlikely(!s)) return; =20 - get_online_cpus(); - mutex_lock(&slab_mutex); =20 s->refcount--; @@ -503,8 +497,6 @@ void kmem_cache_destroy(struct kmem_cache *s) } out_unlock: mutex_unlock(&slab_mutex); - - put_online_cpus(); } EXPORT_SYMBOL(kmem_cache_destroy); =20 @@ -521,12 +513,10 @@ int kmem_cache_shrink(struct kmem_cache *cachep) { int ret; =20 - get_online_cpus(); =20 kasan_cache_shrink(cachep); ret =3D __kmem_cache_shrink(cachep); =20 - put_online_cpus(); return ret; } EXPORT_SYMBOL(kmem_cache_shrink); --=20 2.29.2