From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84CF1C10DCE for ; Tue, 10 Mar 2020 22:10:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2ECBC222C4 for ; Tue, 10 Mar 2020 22:10:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2ECBC222C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B96146B0003; Tue, 10 Mar 2020 18:10:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B46F76B0006; Tue, 10 Mar 2020 18:10:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5D726B0007; Tue, 10 Mar 2020 18:10:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 8AC2E6B0003 for ; Tue, 10 Mar 2020 18:10:23 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5D31052C6 for ; Tue, 10 Mar 2020 22:10:23 +0000 (UTC) X-FDA: 76580847126.19.game33_2076dc31b4206 X-HE-Tag: game33_2076dc31b4206 X-Filterd-Recvd-Size: 4632 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Mar 2020 22:10:22 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id v11so18306wrm.9 for ; Tue, 10 Mar 2020 15:10:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=fzoPcCcArgslJEjFGamnoESKQ6n8NlkNe0nFrWqr2Us=; b=CM0FhU81duXwnK02ctbMOXiQ+vIUHjGiex+w7A90Y+u1obnLHV2It8bSeq4Tea5xSC 1Aa/D1L0Rm+WdOPm5o+Woo9+lAGAx8jjFvmBaESizHrpwZcjQuR7MnyOlThfqTGnFOn8 Q2BWDSnf2e+QQMhHJrJyln4Z8KrgeFX/2KgbzNKbYqiHeUiD0YVYNPaqckNYZfRQBG3Y qNLrJITg1BD8yx3nC/Ktab82LROItzikmX+ioLRkAgteXUUnJeYJ6NRjB96m9ihd89F7 tMkabnm87xml3xtQE5HPAJrCZQ7g0k73wkkBfg6gtV02MQO+M3LT0Rn8EU0IVZl0U5S/ mOuA== X-Gm-Message-State: ANhLgQ3brcUvQnEeWQtru47ANcX9q0zXYc9QX6OOIXrkpss3QPddj4/J GX/Xv/zh3+dNoUt2Hre/Pzs= X-Google-Smtp-Source: ADFU+vuz7Kl5/dFfUzD4Yt/LB7B4aNfvbc25VgZibMXjn6QBDhlwhZtu3Pi6nHtgw2nFnnWEkb0Jcw== X-Received: by 2002:adf:aa04:: with SMTP id p4mr13601wrd.238.1583878221819; Tue, 10 Mar 2020 15:10:21 -0700 (PDT) Received: from localhost (ip-37-188-253-35.eurotel.cz. [37.188.253.35]) by smtp.gmail.com with ESMTPSA id q5sm21114106wrc.68.2020.03.10.15.10.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 15:10:21 -0700 (PDT) Date: Tue, 10 Mar 2020 23:10:19 +0100 From: Michal Hocko To: David Rientjes Cc: Andrew Morton , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [patch] mm, oom: prevent soft lockup on memcg oom for UP systems Message-ID: <20200310221019.GE8447@dhcp22.suse.cz> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 10-03-20 14:39:48, David Rientjes wrote: > When a process is oom killed as a result of memcg limits and the victim > is waiting to exit, nothing ends up actually yielding the processor back > to the victim on UP systems with preemption disabled. Instead, the > charging process simply loops in memcg reclaim and eventually soft > lockups. > > Memory cgroup out of memory: Killed process 808 (repro) total-vm:41944kB, anon-rss:35344kB, file-rss:504kB, shmem-rss:0kB, UID:0 pgtables:108kB oom_score_adj:0 > watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [repro:806] > CPU: 0 PID: 806 Comm: repro Not tainted 5.6.0-rc5+ #136 > RIP: 0010:shrink_lruvec+0x4e9/0xa40 > ... > Call Trace: > shrink_node+0x40d/0x7d0 > do_try_to_free_pages+0x13f/0x470 > try_to_free_mem_cgroup_pages+0x16d/0x230 > try_charge+0x247/0xac0 > mem_cgroup_try_charge+0x10a/0x220 > mem_cgroup_try_charge_delay+0x1e/0x40 > handle_mm_fault+0xdf2/0x15f0 > do_user_addr_fault+0x21f/0x420 > page_fault+0x2f/0x40 > > Make sure that something ends up actually yielding the processor back to > the victim to allow for memory freeing. Most appropriate place appears to > be shrink_node_memcgs() where the iteration of all decendant memcgs could > be particularly lengthy. There is a cond_resched in shrink_lruvec and another one in shrink_page_list. Why doesn't any of them hit? Is it because there are no pages on the LRU list? Because rss data suggests there should be enough pages to go that path. Or maybe it is shrink_slab path that takes too long? The patch itself makes sense to me but I would like to see more explanation on how that happens. Thanks. > Cc: Vlastimil Babka > Cc: Michal Hocko > Cc: stable@vger.kernel.org > Signed-off-by: David Rientjes > --- > mm/vmscan.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -2637,6 +2637,8 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) > unsigned long reclaimed; > unsigned long scanned; > > + cond_resched(); > + > switch (mem_cgroup_protected(target_memcg, memcg)) { > case MEMCG_PROT_MIN: > /* -- Michal Hocko SUSE Labs