From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F067C432C3 for ; Mon, 25 Nov 2019 19:20:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 59AE720678 for ; Mon, 25 Nov 2019 19:20:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GGHZW0/q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59AE720678 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EA1E66B0278; Mon, 25 Nov 2019 14:20:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E51986B0279; Mon, 25 Nov 2019 14:20:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D69206B027A; Mon, 25 Nov 2019 14:20:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0087.hostedemail.com [216.40.44.87]) by kanga.kvack.org (Postfix) with ESMTP id C26AD6B0278 for ; Mon, 25 Nov 2019 14:20:49 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 4BDC64DB0 for ; Mon, 25 Nov 2019 19:20:49 +0000 (UTC) X-FDA: 76195767018.11.shirt64_5e6043a0be85f X-HE-Tag: shirt64_5e6043a0be85f X-Filterd-Recvd-Size: 7365 Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com [209.85.167.195]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Mon, 25 Nov 2019 19:20:48 +0000 (UTC) Received: by mail-oi1-f195.google.com with SMTP id l202so14241693oig.1 for ; Mon, 25 Nov 2019 11:20:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kHLgwwWlJiLxr2ARt5GzYa5qJrBUsXgP8lqORrP4CBI=; b=GGHZW0/qHglQhTKyr9FT7LArU+/8B2MJwt0ERTENNILqCOR0zSFG5JQ6h85doOjbHk LNTaVuVoX5VDHXd8l/o9UYnIYvh/r+ZYOrUr5VcbqzY5DTyQcIBl8NEIu14TmaCYYAgZ ZnSf0kNazLgPtrS24ssoKF1aJsxWI9Zq6ekt/og/UomfWjhdflUN4vEMEdysBkgO69Y4 EVt2jwQQPfU3MiCLeXDPHG+YExP3K/V9K2RFxGeSFVhn+jx8YiIme2BQGVOb39YkP0sx 2lZYa0h6a6oKnzbtw/sWu3CYSjSwSo00Kj79t7uQCCl0rrg0l2PegD6+hLz2pQMm/P3h nQUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kHLgwwWlJiLxr2ARt5GzYa5qJrBUsXgP8lqORrP4CBI=; b=sBFkjAK6ZMOzKwQa7C63DPNFX0oGhBQFm+OHGHQoWBPIzFu5V684IkjiDhKJLbqB+s o22SW4mEQNLqmNLeMJSe5e784o4yl8UKv+0Izd3tmQkXKwo5/vfJKxBUbXU/aF1ucQ7+ 8AxMrDyvOtuqK1kPyBWBfN+QHJiOvS41mq6eSrQzL7gVHDXl2wL3nfR2jH1K0bNDxKjU +Kyc2TJQJjvuwKjspBlwhNCq5Zq2sf9OmnHRj3CiPedT4s0vmDr2vzLM1Z4KizvVL17v hxaUVfhG4QSETR/f2sDzBj39Yt2ycfDT0zhq/QpW9HnU0Y8+M/k32SFDX7Mb9EygZLkF pDBQ== X-Gm-Message-State: APjAAAUNP+ViN0tDPoSyBceWzTx26r/FW4cY8kfrdKzrFW5E5/pEYinj iaa32gj3mM8KZ9lgdRlPQr2ieFS0KSLkv5znOkheQQ== X-Google-Smtp-Source: APXvYqyD4ah9bhyn/it46CSx9F7IM1K0Q0W5H77mtH0vZkSgt3LuPykHUgPh/okw2tgZ2UWF30HMQ8MfxqKfhIhCQB0= X-Received: by 2002:aca:4f58:: with SMTP id d85mr403850oib.142.1574709647312; Mon, 25 Nov 2019 11:20:47 -0800 (PST) MIME-Version: 1.0 References: <20191125185453.278468-1-guro@fb.com> In-Reply-To: <20191125185453.278468-1-guro@fb.com> From: Shakeel Butt Date: Mon, 25 Nov 2019 11:20:35 -0800 Message-ID: Subject: Re: [PATCH] mm: memcg/slab: wait for !root kmem_cache refcnt killing on root kmem_cache destruction To: Roman Gushchin Cc: Linux MM , Andrew Morton , Michal Hocko , Johannes Weiner , LKML , Kernel Team , Christian Borntraeger , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 25, 2019 at 10:55 AM Roman Gushchin wrote: > > Christian reported a warning like the following obtained during running some > KVM-related tests on s390: > > WARNING: CPU: 8 PID: 208 at lib/percpu-refcount.c:108 percpu_ref_exit+0x50/0x58 > Modules linked in: kvm(-) xt_CHECKSUM xt_MASQUERADE bonding xt_tcpudp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ip6table_na> > CPU: 8 PID: 208 Comm: kworker/8:1 Not tainted 5.2.0+ #66 > Hardware name: IBM 2964 NC9 712 (LPAR) > Workqueue: events sysfs_slab_remove_workfn > Krnl PSW : 0704e00180000000 0000001529746850 (percpu_ref_exit+0x50/0x58) > R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 > Krnl GPRS: 00000000ffff8808 0000001529746740 000003f4e30e8e18 0036008100000000 > 0000001f00000000 0035008100000000 0000001fb3573ab8 0000000000000000 > 0000001fbdb6de00 0000000000000000 0000001529f01328 0000001fb3573b00 > 0000001fbb27e000 0000001fbdb69300 000003e009263d00 000003e009263cd0 > Krnl Code: 0000001529746842: f0a0000407fe srp 4(11,%r0),2046,0 > 0000001529746848: 47000700 bc 0,1792 > #000000152974684c: a7f40001 brc 15,152974684e > >0000001529746850: a7f4fff2 brc 15,1529746834 > 0000001529746854: 0707 bcr 0,%r7 > 0000001529746856: 0707 bcr 0,%r7 > 0000001529746858: eb8ff0580024 stmg %r8,%r15,88(%r15) > 000000152974685e: a738ffff lhi %r3,-1 > Call Trace: > ([<000003e009263d00>] 0x3e009263d00) > [<00000015293252ea>] slab_kmem_cache_release+0x3a/0x70 > [<0000001529b04882>] kobject_put+0xaa/0xe8 > [<000000152918cf28>] process_one_work+0x1e8/0x428 > [<000000152918d1b0>] worker_thread+0x48/0x460 > [<00000015291942c6>] kthread+0x126/0x160 > [<0000001529b22344>] ret_from_fork+0x28/0x30 > [<0000001529b2234c>] kernel_thread_starter+0x0/0x10 > Last Breaking-Event-Address: > [<000000152974684c>] percpu_ref_exit+0x4c/0x58 > ---[ end trace b035e7da5788eb09 ]--- > > The problem occurs because kmem_cache_destroy() is called immediately > after deleting of a memcg, so it races with the memcg kmem_cache > deactivation. > > flush_memcg_workqueue() at the beginning of kmem_cache_destroy() > is supposed to guarantee that all deactivation processes are finished, > but failed to do so. It waits for an rcu grace period, after which all > children kmem_caches should be deactivated. During the deactivation > percpu_ref_kill() is called for non root kmem_cache refcounters, > but it requires yet another rcu grace period to finish the transition > to the atomic (dead) state. > > So in a rare case when not all children kmem_caches are destroyed > at the moment when the root kmem_cache is about to be gone, we need > to wait another rcu grace period before destroying the root > kmem_cache. > > Reported-by: Christian Borntraeger > Tested-by: Christian Borntraeger > Fixes: f0a3a24b532d ("mm: memcg/slab: rework non-root kmem_cache lifecycle management") > Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt > Cc: stable@vger.kernel.org > --- > mm/slab_common.c | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/mm/slab_common.c b/mm/slab_common.c > index 8afa188f6e20..f0ab6d4ceb4c 100644 > --- a/mm/slab_common.c > +++ b/mm/slab_common.c > @@ -904,6 +904,18 @@ static void flush_memcg_workqueue(struct kmem_cache *s) > * previous workitems on workqueue are processed. > */ > flush_workqueue(memcg_kmem_cache_wq); > + > + /* > + * If we're racing with children kmem_cache deactivation, it might > + * take another rcu grace period to complete their destruction. > + * At this moment the corresponding percpu_ref_kill() call should be > + * done, but it might take another rcu grace period to complete > + * switching to the atomic mode. > + * Please, note that we check without grabbing the slab_mutex. It's safe > + * because at this moment the children list can't grow. > + */ > + if (!list_empty(&s->memcg_params.children)) > + rcu_barrier(); > } > #else > static inline int shutdown_memcg_caches(struct kmem_cache *s) > -- > 2.23.0 >