From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 263A7C433E0 for ; Thu, 9 Jul 2020 02:14:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D172F206F6 for ; Thu, 9 Jul 2020 02:14:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Rkmxfytp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D172F206F6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7AD4C6B0024; Wed, 8 Jul 2020 22:14:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 75D956B0025; Wed, 8 Jul 2020 22:14:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69ABE6B0026; Wed, 8 Jul 2020 22:14:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 551BE6B0024 for ; Wed, 8 Jul 2020 22:14:52 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 091AC82499B9 for ; Thu, 9 Jul 2020 02:14:52 +0000 (UTC) X-FDA: 77016919224.18.dime35_3d14d1d26ec2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id E1600100EDBFE for ; Thu, 9 Jul 2020 02:14:51 +0000 (UTC) X-HE-Tag: dime35_3d14d1d26ec2 X-Filterd-Recvd-Size: 6323 Received: from mail-il1-f194.google.com (mail-il1-f194.google.com [209.85.166.194]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Thu, 9 Jul 2020 02:14:51 +0000 (UTC) Received: by mail-il1-f194.google.com with SMTP id h16so703139ilj.11 for ; Wed, 08 Jul 2020 19:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=EchH+g0Ks2yJg3E4H9l141s5TJNKRGkIY2Sb//EJ7nk=; b=RkmxfytppI5Iv9vh7IyWLMPzFhBm6KbEMXLQg7J7x8h3nK4jGaMwY0F7B3DO3Fzu/V Vrj3EBUnuXJ+k8njRD/9PSSpv2splZCF17dJZyEw28fzzs2sLRu6V7UreiZyFbaMSnVu LEnfHF0ocshYWoGXm71XshNC6DKvZcoX/xWi+K+tTNK0CX02x6Y4MF9cgEZUg1Fc90KY fnFznVUbpUeEzMAbCt2cuLBNiAlmTkmtcSBuLD6zudPcyRDJtBPQrKwiGvcRi8Y5W+do DDxIIQXHpzGPdR/8JNScCvHUq12pJABI0DsOPNk9g6r78ZQmDcJs9XcyFC2wmcIwTi0/ Xxhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=EchH+g0Ks2yJg3E4H9l141s5TJNKRGkIY2Sb//EJ7nk=; b=a5BWAoMXtj2PwV93yLYxmtVLewQZILkCJW+ioTgGXKJd2AUp3Ly0cOUkukPzsfmaMj e9Q3IKAqQbWrLJ9Gy1H5EqPOO9jnpqMqTXAYvRuP5YCqDLVaN/5GT4nQYRQnPUP/wnbz 6MIMqaR5kxoerRdPcHcXltX3wbaK2c17QFkFUY3R8mYVNtBrQMeB1sjVcLhom0sZV5oj 7dey0PcHnpDeeLFJJCRb+0t2DmvES03ZPQgw0FaZVDoeimFxk3jHXB5KALTAwHm+2dn3 hNViIPva2ZuaWCpk6FakEecUHR44WdkoKouUyxNLaxOmQZ/QVElxltlvqlt0xB06Aj8n TJ/Q== X-Gm-Message-State: AOAM5304YuCaaFnj0oVyA49wvudSoVyZSAnJNkgKgyrX55/Zw98DFTco 4BH9bxc6m0YjhEjzH6kD0YaZnoq1ftYaw0JMwwc= X-Google-Smtp-Source: ABdhPJzC0AQFpSLQid+4OQKQxvO5eaW7II1jjcd5Zq0bPao034EqVwBYziuA8h62rwviPrNxWB9rn5aPmqXOLEYOjIc= X-Received: by 2002:a92:da4c:: with SMTP id p12mr15954463ilq.142.1594260890836; Wed, 08 Jul 2020 19:14:50 -0700 (PDT) MIME-Version: 1.0 References: <1594214649-9837-1-git-send-email-laoar.shao@gmail.com> <20200708142806.GJ7271@dhcp22.suse.cz> <20200708143211.GK7271@dhcp22.suse.cz> <20200708190225.GM7271@dhcp22.suse.cz> In-Reply-To: <20200708190225.GM7271@dhcp22.suse.cz> From: Yafang Shao Date: Thu, 9 Jul 2020 10:14:14 +0800 Message-ID: Subject: Re: [PATCH] mm, oom: make the calculation of oom badness more accurate To: Michal Hocko Cc: David Rientjes , Andrew Morton , Linux MM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E1600100EDBFE X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jul 9, 2020 at 3:02 AM Michal Hocko wrote: > > On Wed 08-07-20 10:57:27, David Rientjes wrote: > > On Wed, 8 Jul 2020, Michal Hocko wrote: > > > > > I have only now realized that David is not on Cc. Add him here. The > > > patch is http://lkml.kernel.org/r/1594214649-9837-1-git-send-email-la= oar.shao@gmail.com. > > > > > > I believe the main problem is that we are normalizing to oom_score_ad= j > > > units rather than usage/total. I have a very vague recollection this = has > > > been done in the past but I didn't get to dig into details yet. > > > > > > > The memcg max is 4194304 pages, and an oom_score_adj of -998 would yiel= d a > > page adjustment of: > > > > adj =3D -998 * 4194304 / 1000 =3D =E2=88=924185915 pages > > > > The largest pid 58406 (data_sim) has rss 3967322 pages, > > pgtables 37101568 / 4096 =3D 9058 pages, and swapents 0. So it's unadj= usted > > badness is > > > > 3967322 + 9058 pages =3D 3976380 pages > > > > Factoring in oom_score_adj, all of these processes will have a badness = of > > 1 because oom_badness() doesn't underflow, which I think is the point o= f > > Yafang's proposal. > > > > I think the patch can work but, as you mention, also needs an update to > > proc_oom_score(). proc_oom_score() is using the global amount of memor= y > > so Yafang is likely not seeing it go negative for that reason but it co= uld > > happen. > > Yes, memcg just makes it more obvious but the same might happen for the > global case. I am not sure how we can both alow underflow and present > the value that would fit the existing model. The exported value should > really reflect what the oom killer is using for the calculation or we > are going to see discrepancies between the real oom decision and > presented values. So I believe we really have to change the calculation > rather than just make it tolerant to underflows. > Hi Michal, - Before my patch, The result of oom_badness() is [1, 2 * totalpages), and the result of proc_oom_score() is [0, 2000). While the badness score in the Documentation/filesystems/proc.rst is: [0, 1= 000] "The badness heuristic assigns a value to each candidate task ranging from = 0 (never kill) to 1000 (always kill) to determine which process is targeted" That means, we need to update the documentation anyway unless my calculation is wrong. So the point will be how to change it ? - After my patch oom_badness(): (-totalpages, 2 * totalpages) proc_oom_score(): (-1000, 2000) If we allow underflow, we can change the documentation as "from -1000 (never kill) to 2000(always kill)". While if we don't allow underflow, we can make bellow simple change, diff --git a/fs/proc/base.c b/fs/proc/base.c index 774784587..0da8efa41 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -528,7 +528,7 @@ static int proc_oom_score(struct seq_file *m, struct pid_namespace *ns, unsigned long totalpages =3D totalram_pages + total_swap_pages; unsigned long points =3D 0; - points =3D oom_badness(task, NULL, NULL, totalpages) * + points =3D 1000 + oom_badness(task, NULL, NULL, totalpages) * 1000 / totalpages; seq_printf(m, "%lu\n", points); And then update the documentation as "from 0 (never kill) to 3000 (always kill)" --=20 Thanks Yafang