From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D5D8C433E0 for ; Thu, 9 Jul 2020 06:26:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1808120739 for ; Thu, 9 Jul 2020 06:26:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1808120739 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A33016B000D; Thu, 9 Jul 2020 02:26:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E4A46B000E; Thu, 9 Jul 2020 02:26:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F9916B0010; Thu, 9 Jul 2020 02:26:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 796446B000D for ; Thu, 9 Jul 2020 02:26:48 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3976B180AD815 for ; Thu, 9 Jul 2020 06:26:48 +0000 (UTC) X-FDA: 77017554096.30.touch76_4a14a5826ec3 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id F2E4A180B3C8E for ; Thu, 9 Jul 2020 06:26:47 +0000 (UTC) X-HE-Tag: touch76_4a14a5826ec3 X-Filterd-Recvd-Size: 7182 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Thu, 9 Jul 2020 06:26:47 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id f139so610141wmf.5 for ; Wed, 08 Jul 2020 23:26:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=EqoSOpI0Yr9mddV/DCsZncHxvUzGgRX2DR5bu1RCU3w=; b=XXiGVOozl2m0DIKUCOXKgHj7vmqor2a0WTC88fwQ4OAQrzzo0QFcvExbsL5jlZcOp8 mbVF47sbW/lb7o32CIL3O/zuMd2BpLd0zMGsh408N7YS0KKawpEBGuw4sN1pyid8avQJ UzF7n9Bd+aJMhNkXlSu42i3T3XIyHxp0cFVLCMtbeJh250913AnXYD2Z4kkrwm2zMaOG Rpe+P9rbeTxGU0M/8j8PzkwoTh6CddeUsW+OD1QNup9tiy6P6VkFIvvTSBkqNE8w5OOF 60VDzcj6n7FeU1e633ofGsB8HObXw2NoCFQaDRo3JTSdF3iMJ/GT0rj4ofPqbuN06E6p vg0A== X-Gm-Message-State: AOAM533XrHTPcBGGwbDBDYp6Z14ccKyzLeWk8iJM8nuZnTb0Ct0gqG4g eAohbay/1rn/2BV6g/GUAs4= X-Google-Smtp-Source: ABdhPJxo1qZKmBnVZb8dPWEKI/NQewyCMm/ojgwdXh4ZMmvVuJ6zedV4vNc8e3II4pdn2Qyda2nWPg== X-Received: by 2002:a1c:1d46:: with SMTP id d67mr13565694wmd.152.1594276006397; Wed, 08 Jul 2020 23:26:46 -0700 (PDT) Received: from localhost (ip-37-188-179-51.eurotel.cz. [37.188.179.51]) by smtp.gmail.com with ESMTPSA id 138sm17083567wmb.1.2020.07.08.23.26.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 23:26:45 -0700 (PDT) Date: Thu, 9 Jul 2020 08:26:44 +0200 From: Michal Hocko To: Yafang Shao Cc: David Rientjes , Andrew Morton , Linux MM Subject: Re: [PATCH] mm, oom: make the calculation of oom badness more accurate Message-ID: <20200709062644.GA12704@dhcp22.suse.cz> References: <1594214649-9837-1-git-send-email-laoar.shao@gmail.com> <20200708142806.GJ7271@dhcp22.suse.cz> <20200708143211.GK7271@dhcp22.suse.cz> <20200708190225.GM7271@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: F2E4A180B3C8E X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 09-07-20 10:14:14, Yafang Shao wrote: > On Thu, Jul 9, 2020 at 3:02 AM Michal Hocko wrote: > > > > On Wed 08-07-20 10:57:27, David Rientjes wrote: > > > On Wed, 8 Jul 2020, Michal Hocko wrote: > > > > > > > I have only now realized that David is not on Cc. Add him here. T= he > > > > patch is http://lkml.kernel.org/r/1594214649-9837-1-git-send-emai= l-laoar.shao@gmail.com. > > > > > > > > I believe the main problem is that we are normalizing to oom_scor= e_adj > > > > units rather than usage/total. I have a very vague recollection t= his has > > > > been done in the past but I didn't get to dig into details yet. > > > > > > > > > > The memcg max is 4194304 pages, and an oom_score_adj of -998 would = yield a > > > page adjustment of: > > > > > > adj =3D -998 * 4194304 / 1000 =3D =E2=88=924185915 pages > > > > > > The largest pid 58406 (data_sim) has rss 3967322 pages, > > > pgtables 37101568 / 4096 =3D 9058 pages, and swapents 0. So it's u= nadjusted > > > badness is > > > > > > 3967322 + 9058 pages =3D 3976380 pages > > > > > > Factoring in oom_score_adj, all of these processes will have a badn= ess of > > > 1 because oom_badness() doesn't underflow, which I think is the poi= nt of > > > Yafang's proposal. > > > > > > I think the patch can work but, as you mention, also needs an updat= e to > > > proc_oom_score(). proc_oom_score() is using the global amount of m= emory > > > so Yafang is likely not seeing it go negative for that reason but i= t could > > > happen. > > > > Yes, memcg just makes it more obvious but the same might happen for t= he > > global case. I am not sure how we can both alow underflow and present > > the value that would fit the existing model. The exported value shoul= d > > really reflect what the oom killer is using for the calculation or we > > are going to see discrepancies between the real oom decision and > > presented values. So I believe we really have to change the calculati= on > > rather than just make it tolerant to underflows. > > >=20 > Hi Michal, >=20 > - Before my patch, > The result of oom_badness() is [1, 2 * totalpages), > and the result of proc_oom_score() is [0, 2000). >=20 > While the badness score in the Documentation/filesystems/proc.rst is: [= 0, 1000] > "The badness heuristic assigns a value to each candidate task ranging f= rom 0 > (never kill) to 1000 (always kill) to determine which process is target= ed" >=20 > That means, we need to update the documentation anyway unless my > calculation is wrong. No, your calculation is correct. The documentation is correct albeit slightly misleading. The net score calculation is indeed in range of [0, = 1000]. It is the oom_score_adj added on top which skews it. This is documented as "The value of /proc//oom_score_adj is added to the badness score bef= ore it is used to determine which task to kill." This is the exported value but paragraph "3.2 /proc//oom_score" only= says "This file can be used to check the current score used by the oom-killer = is for any given ." which is not really explicit about the exported range. Maybe clarifying that would be helpful. I will post a patch. There are few other things to sync up with the current state. > So the point will be how to change it ? >=20 > - After my patch > oom_badness(): (-totalpages, 2 * totalpages) > proc_oom_score(): (-1000, 2000) >=20 > If we allow underflow, we can change the documentation as "from -1000 > (never kill) to 2000(always kill)". > While if we don't allow underflow, we can make bellow simple change, >=20 > diff --git a/fs/proc/base.c b/fs/proc/base.c > index 774784587..0da8efa41 100644 > --- a/fs/proc/base.c > +++ b/fs/proc/base.c > @@ -528,7 +528,7 @@ static int proc_oom_score(struct seq_file *m, > struct pid_namespace *ns, > unsigned long totalpages =3D totalram_pages + total_swap_pages; > unsigned long points =3D 0; >=20 > - points =3D oom_badness(task, NULL, NULL, totalpages) * > + points =3D 1000 + oom_badness(task, NULL, NULL, totalpages) * > 1000 / totalpages; > seq_printf(m, "%lu\n", points); >=20 > And then update the documentation as "from 0 (never kill) to 3000 > (always kill)" This is still not quite there yet, I am afraid. OOM_SCORE_ADJ_MIN tasks h= ave always reported 0 and I can imagine somebody might depend on this fact. So you need to special case LONG_MIN at least. It would be also better to stick with [0, 2000] range. --=20 Michal Hocko SUSE Labs