From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67624C433F5 for ; Fri, 25 Feb 2022 22:19:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2DCA8D0002; Fri, 25 Feb 2022 17:19:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DDC6E8D0001; Fri, 25 Feb 2022 17:19:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA42E8D0002; Fri, 25 Feb 2022 17:19:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id BBB578D0001 for ; Fri, 25 Feb 2022 17:19:40 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8479B9D271 for ; Fri, 25 Feb 2022 22:19:40 +0000 (UTC) X-FDA: 79182720120.17.BF58C60 Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) by imf17.hostedemail.com (Postfix) with ESMTP id EDC2241B66 for ; Fri, 25 Feb 2022 22:19:07 +0000 (UTC) Received: by mail-ot1-f44.google.com with SMTP id l20-20020a0568302b1400b005af8c95bbe4so4623433otv.1 for ; Fri, 25 Feb 2022 14:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=g/zvsshx2/OeJp8fgGHHkDWog+GcU0vpTGsYvVwa7rE=; b=nBPL0l0X78lLuNAZSEsXi6l8sUddsgWeOxOt7ngTp0aGlf5ekcx1Xkf+ujapcLH2si vjmqsyDJkr/V/w5TdmbzPvxOnVkiaCbw/5ZWCgJqlh/xH6vXhRKYOdIBl6L54qa5mnsR csl7gsv37VCaP0rGky9G0LVtpntXMZRFHe4XU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=g/zvsshx2/OeJp8fgGHHkDWog+GcU0vpTGsYvVwa7rE=; b=hSvZEQo9gjDmsRwJupt9Ie5JxJ+qgyl/0bIZNaSluLZypLuYPqOc6zfDwprw0TDwAZ HD7JUEX9RmvItBvvEyOIdlSFFgC3WwqFAO7gIW7V34SmFNkRCvEASavpMAPsASa7LBRH eTBCw8P4QqPt5xaK5yACaInw813QsAlcdZJ1NK2n4Or22HaSJ+lsnlKn3Jc25/HArVlC v76SLfbpAAGt/61JqxGTc/jW8z32AOl/DBNtQUT4vPmk2QFe1h9GiHoN3fg59LrM3+6r Y/nBgGvgncvm1Wi8gRNPc5a32uZJ8mO8FBbT+QDoCb5H4sr5H48qRuLPJ+Gi0dmX/dhK 930w== X-Gm-Message-State: AOAM531ziiQjiQ/VfwAAYHkgmaaSwwz3+dD23Tg+fOTtCvrIXvCa/bEW Y8O4wXcMqlhY2KVXE+s+j7miakXvch+yc9627fGWBdaemg1+tw== X-Google-Smtp-Source: ABdhPJx6H8lBnptse0wF8EoQ1rykSkWzJiq7EYDTDK9YvUgAEnJil4cxAujgelbswGwG3enbKekjTXQjTjX5GTpU/ig= X-Received: by 2002:a05:6902:1282:b0:61a:7abd:6785 with SMTP id i2-20020a056902128200b0061a7abd6785mr8216389ybu.558.1645808941196; Fri, 25 Feb 2022 09:09:01 -0800 (PST) MIME-Version: 1.0 References: <20220224165838.oir5clpkkqpstpx3@google.com> <20220224185236.qgzm3jpoz2orjfcw@google.com> In-Reply-To: From: Ivan Babrou Date: Fri, 25 Feb 2022 09:08:50 -0800 Message-ID: Subject: Re: Regression in workingset_refault latency on 5.15 To: Daniel Dao Cc: Shakeel Butt , kernel-team , Linux MM , Johannes Weiner , Roman Gushchin , Feng Tang , Michal Hocko , Hillf Danton , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Andrew Morton , Linus Torvalds Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: EDC2241B66 X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=cloudflare.com header.s=google header.b=nBPL0l0X; dmarc=pass (policy=reject) header.from=cloudflare.com; spf=none (imf17.hostedemail.com: domain of ivan@cloudflare.com has no SPF policy when checking 209.85.210.44) smtp.mailfrom=ivan@cloudflare.com X-Stat-Signature: easkdu8nfxx94tg9gkicb3twq6gd3tef X-HE-Tag: 1645827547-723303 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 25, 2022 at 2:23 AM Daniel Dao wrote: > I think this looks good so far. I compared a flamegraph before to a flamegraph after (10s @ 99Hz on 96-core CPU evenly loaded to ~75% in both cases). Before: 1.4% spent in workingset_refault. After: 0.5% spent in flush_memcg_stats_dwork. The latter is all in kworkers (as expected), while the former is spread across IO active tasks. This seems like a great first step that should be merged on its own. It would be good to also do something to improve the CPU time spent in delayed work, if possible, as 0.5% of on-CPU time is not a negligible amount.