From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABCAFC433DB for ; Fri, 26 Mar 2021 08:16:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0511661A4C for ; Fri, 26 Mar 2021 08:16:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0511661A4C Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3EC666B0036; Fri, 26 Mar 2021 04:16:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3765D6B006E; Fri, 26 Mar 2021 04:16:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22A956B0070; Fri, 26 Mar 2021 04:16:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 01CA76B0036 for ; Fri, 26 Mar 2021 04:16:17 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AF03B181BA3DE for ; Fri, 26 Mar 2021 08:16:17 +0000 (UTC) X-FDA: 77961317994.15.768D80F Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf07.hostedemail.com (Postfix) with ESMTP id CD1F4A0000FC for ; Fri, 26 Mar 2021 08:16:16 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1616746576; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3wWiCP2GdmHrn2OrTkc2vGMcI3kl/aA9xBCE5d/QgWs=; b=pKRnzBkIj9hAOyLkdwfof7uhc6UajPRLCJeaiP3d8e6Jc3hLh4cT7Z0COqjq+Z0Uaz2o95 zQUSvDSwpKzExw42dnH7xFq+SQi7jZ7Ym0RNJS8ow2ea8vAgX1WZlw3jauiWibb8iSFfIJ FaQcL6N/ktz2N+ofK+Yd7esWrrGkpxE= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id EE2CEADDD; Fri, 26 Mar 2021 08:16:15 +0000 (UTC) Date: Fri, 26 Mar 2021 09:16:12 +0100 From: Michal Hocko To: Aaron Tomlin Cc: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: Re: [PATCH] mm/page_alloc: try oom if reclaim is unable to make forward progress Message-ID: References: <20210315165837.789593-1-atomlin@redhat.com> <20210319172901.cror2u53b7caws3a@ava.usersys.com> <20210325210159.r565fvfitoqeuykp@ava.usersys.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210325210159.r565fvfitoqeuykp@ava.usersys.com> X-Stat-Signature: zwxkqbqj98kdb8mm5mjety98yuu357f9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: CD1F4A0000FC Received-SPF: none (suse.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616746576-62607 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: [Cc Vlastimil] On Thu 25-03-21 21:01:59, Aaron Tomlin wrote: > On Mon 2021-03-22 11:47 +0100, Michal Hocko wrote: > > Costly orders already do have heuristics for the retry in place. Could > > you be more specific what kind of problem you see with those? > > If I understand correctly, when the gfp_mask consists of > GFP_KERNEL | __GFP_RETRY_MAYFAIL in particular, an allocation request will > fail, if and only if reclaim is unable to make progress. > > The costly order allocation retry logic is handled primarily in > should_reclaim_retry(). Looking at should_reclaim_retry() we see that the > no progress counter value is always incremented in the costly order case > even when "some" progress has been made which is represented by the value > stored in did_some_progress. > > if (costly_order && !(gfp_mask & __GFP_RETRY_MAYFAIL)) > goto nopage; > > if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags, > did_some_progress > 0, &no_progress_loops)) > goto retry; > > I think after we have tried MAX_RECLAIM_RETRIES in a row without success > and the last known compaction attempt was "skipped", perhaps it would be > better to try and use the OOM killer or fail the allocation attempt? The oom killer is never triggered for costly allocation request. Both reclaim and compaction maintain their own retries counters as they are targeting a different operation. Although the compaction really depends on the reclaim to do some progress. > I encountered a situation when the value of no_progress_loops was found to > be 31,611,688 i.e. significantly over MAX_RECLAIM_RETRIES; the allocation > order was 5. The gfp_mask contained the following: OK, this sound unexpected as it indicates that the reclaim is able to make a forward progress but compaction doesn't want to give up and keeps retrying. Are you able to reproduce this or could you find out which specific condition keeps compaction retrying? I would expect that it is one of the 3 conditions before the max_retries is checked. > #define ___GFP_HIGHMEM 0x02 > #define ___GFP_IO 0x40 > #define ___GFP_FS 0x80 > #define ___GFP_NOWARN 0x200 > #define ___GFP_RETRY_MAYFAIL 0x400 > #define ___GFP_COMP 0x4000 > #define ___GFP_HARDWALL 0x20000 > #define ___GFP_DIRECT_RECLAIM 0x200000 > #define ___GFP_KSWAPD_RECLAIM 0x400000 > > > > -- > Aaron Tomlin > -- Michal Hocko SUSE Labs