From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7B4BC433F5 for ; Mon, 21 Feb 2022 06:32:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50DA86B0072; Mon, 21 Feb 2022 01:32:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BD7D8D0002; Mon, 21 Feb 2022 01:32:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3849B8D0001; Mon, 21 Feb 2022 01:32:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 2A8786B0072 for ; Mon, 21 Feb 2022 01:32:44 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E9FC220D0D for ; Mon, 21 Feb 2022 06:32:43 +0000 (UTC) X-FDA: 79165818606.04.4EDB42D Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf26.hostedemail.com (Postfix) with ESMTP id C83BA140002 for ; Mon, 21 Feb 2022 06:32:43 +0000 (UTC) Received: by mail-qv1-f45.google.com with SMTP id e22so29592756qvf.9 for ; Sun, 20 Feb 2022 22:32:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:in-reply-to:message-id:references :mime-version; bh=wmHTVh2NfcEW45izXbwh4sjY94it2WsAI7ES7pYD8FU=; b=a3g8jiw/4of17us/bKcnaheUrxEsyWgmSsheKhiWNGsJa/GPADHQibwMw3kTEl5jYP vCO6jPKhBEjwYh+Otg3D/SvCgD6vEcMI7FbuZh+YTusQB1TTYASCw5bx1YAPIQga7iNM Uv6WkkRBatCkaEJ9iooe+TpLljCxIGkRNJlhAqkTEhV9QpzB9LwHd/5IpN69y652f9vY 6B/cMZRznwHd2NsBG8SdjvOS7GTquxWzWHgZv3ou+x1xMXUVQQvZelaL1Zlt9pu9Oe3b J1rhKle0GVyX4KlN9/FAOMf2WxE4jd7luDwvhr16K4B2aa7zTshKeUA2nEeJyIbrvWwr 81Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:mime-version; bh=wmHTVh2NfcEW45izXbwh4sjY94it2WsAI7ES7pYD8FU=; b=iBmdAj4BfLHdmkW3Cq+f+nK5wyXxBzeN6uovFEAG1N3aYf/lWQ6piAiGsXGZbxyjwJ RiQBmUftaKwEIQ29jljrSsbqYxtHx62WnJqtwpxaEIH2NOrsIeTRtn8emhevSQGJ0PWi 1x+39eZlJWQaQ1+NotDbDggpb1SEZ+Mb1pkpLq32h6Po7iLv2EK87O5mswSB17Jv2mBQ /NKEwJJhEFqU8YwEYWCt/Y5CkAmhU4k19Z0bIfc4bm4KIsARPUgzP2wH7tDE+9ohD4Uf fias3KhSI90i4JJs0e2HRloXLsK9L5Wpl8PfMJw+9dkKQgfxfW0CE6ht/mdHoDWEZi9/ T3ug== X-Gm-Message-State: AOAM530gzXAjmxuKURC3GkHTrbtsuZ+1baBvP1wiOK8Ivz3kKq60q9Z8 njE/i0NNxN6XpG0VZlhvqZZSdg== X-Google-Smtp-Source: ABdhPJzuN5WHnPaCleK+xyaSD2sjr9IzNkUaaB49xdktpcIA/iCrSR2CJy44WgmH3PO5Ogn9s8hHAw== X-Received: by 2002:ac8:5f4c:0:b0:2d5:e2d7:de22 with SMTP id y12-20020ac85f4c000000b002d5e2d7de22mr16794816qta.472.1645425162416; Sun, 20 Feb 2022 22:32:42 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id p70sm25476041qka.62.2022.02.20.22.32.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 22:32:41 -0800 (PST) Date: Sun, 20 Feb 2022 22:32:28 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.anvils To: kernel test robot cc: Hugh Dickins , 0day robot , Vlastimil Babka , LKML , lkp@lists.01.org, ying.huang@intel.com, feng.tang@intel.com, zhengjun.xing@linux.intel.com, fengwei.yin@intel.com, Andrew Morton , Michal Hocko , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Alistair Popple , Johannes Weiner , Rik van Riel , Suren Baghdasaryan , Yu Zhao , Greg Thelen , Shakeel Butt , Hillf Danton , linux-mm@kvack.org Subject: Re: [mm/munlock] 237b445401: stress-ng.remap.ops_per_sec -62.6% regression In-Reply-To: <7d35c86-77f6-8bf-84a-5c23ff610f4@google.com> Message-ID: References: <20220218063536.GA4377@xsang-OptiPlex-9020> <7d35c86-77f6-8bf-84a-5c23ff610f4@google.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="0-2040830854-1645425161=:2653" X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="a3g8jiw/"; spf=pass (imf26.hostedemail.com: domain of hughd@google.com designates 209.85.219.45 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C83BA140002 X-Stat-Signature: mugegzakg3asacytx9wue4w5ymp3u9r9 X-HE-Tag: 1645425163-75552 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-2040830854-1645425161=:2653 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE On Fri, 18 Feb 2022, Hugh Dickins wrote: > On Fri, 18 Feb 2022, kernel test robot wrote: > >=20 > > Greeting, > >=20 > > FYI, we noticed a -62.6% regression of stress-ng.remap.ops_per_sec due = to commit: > >=20 > >=20 > > commit: 237b4454014d3759acc6459eb329c5e3d55113ed ("[PATCH v2 07/13] mm/= munlock: mlock_pte_range() when mlocking or munlocking") > > url: https://github.com/0day-ci/linux/commits/Hugh-Dickins/mm-munlock-r= ework-of-mlock-munlock-page-handling/20220215-104421 > > base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git ee28855a= 54493ce83bc2a3fbe30210be61b57bc7 > > patch link: https://lore.kernel.org/lkml/d39f6e4d-aa4f-731a-68ee-e77cdb= f1d7bb@google.com > >=20 > > in testcase: stress-ng > > on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Platinum 8358 C= PU @ 2.60GHz with 128G memory > > with following parameters: > >=20 > > =09nr_threads: 100% > > =09testtime: 60s > > =09class: memory > > =09test: remap > > =09cpufreq_governor: performance > > =09ucode: 0xd000280 > >=20 > >=20 > >=20 > >=20 > > If you fix the issue, kindly add following tag > > Reported-by: kernel test robot > >=20 > >=20 > > Details are as below: > > -----------------------------------------------------------------------= ---------------------------> > >=20 > >=20 > > To reproduce: > >=20 > > git clone https://github.com/intel/lkp-tests.git > > cd lkp-tests > > sudo bin/lkp install job.yaml # job file is attached = in this email > > bin/lkp split-job --compatible job.yaml # generate the yaml fil= e for lkp run > > sudo bin/lkp run generated-yaml-file > >=20 > > # if come across any failure that blocks the test, > > # please remove ~/.lkp and /lkp dir to run from a clean state. > >=20 > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/te= st/testcase/testtime/ucode: > > memory/gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-2020= 0603.cgz/lkp-icl-2sp6/remap/stress-ng/60s/0xd000280 > >=20 > > commit:=20 > > c479426e09 ("mm/munlock: maintain page->mlock_count while unevictable= ") > > 237b445401 ("mm/munlock: mlock_pte_range() when mlocking or munlockin= g") > >=20 > > c479426e09c8088d 237b4454014d3759acc6459eb32=20 > > ---------------- ---------------------------=20 > > %stddev %change %stddev > > \ | \ =20 > > 109459 -62.5% 41003 =B1 2% stress-ng.remap.ops > > 1823 -62.6% 682.54 =B1 2% stress-ng.remap.ops_per= _sec > > 2.242e+08 -62.5% 83989157 =B1 2% stress-ng.time.minor_pa= ge_faults > > 30.00 =B1 2% -61.2% 11.65 =B1 4% stress-ng.time.user_t= ime >=20 > Thanks a lot for trying it out, I did hope that you would find something. >=20 > However, IIUC, this by itself is not very interesting: > the comparison is between c479426e09 (06/13) as base and 237b445401? > 237b445401 is 07/13, "Fill in missing pieces", where the series gets > to be correct again, after dropping the old implementation and piecing > together the rest of the new implementation. It's not a surprise that > those tests which need what's added back in 07/13 will get much slower > at this stage. And later 10/13 brings in a pagevec to speed it up. I probably did not understand correctly: I expect you did try the whole series at first, found a regression, and then bisected to that commit. >=20 > What would be much more interesting is to treat the series of 13 as one, > and compare the baseline before any of it against the end of the series: > is that something that the 0day robot can easily do? That would still be more interesting to me - though probably not actionable (see below), so not worth you going to any effort. But I hope the bad result on this test did not curtail further testing. >=20 > But I'll look more closely at the numbers you've provided later today, > numbers that I've snipped off here: there may still be useful things to > learn from them; and maybe I'll try following the instructions you've > supplied, though I probably won't do a good job of following them. I did look more closely, didn't try lkp itself, but did observe stress-ng --timeout 60 --times --verify --metrics-brief --remap 128 in the reproduce file, and followed that up (but with 8 not 128). In my config, the series managed about half the ops as the baseline. Comparison of unevictable_pgs in /proc/vmstat was instructive. Baseline 5.17-rc: With mm/munlock series applied: unevictable_pgs_culled 17 53097984 unevictable_pgs_rescued 17 53097984 unevictable_pgs_mlocked 97251331 53097984 unevictable_pgs_munlocked 97251331 53097984 I was very surprised by those low culled/rescued baseline numbers, but a look at stress-remap-file-pages.c shows that each thread is repetitively doing mlock of one page, remap_file_pages on that address (with implicit munlock of old page and mlock of new), munlock of new. Whereas usually, we don't munlock a range immediately after mlocking it. The baseline's "if (!isolate_lru_page(page)) putback_lru_page(page);" technique works very much in its favour on this test: the munlocking isolates fail because mlock puts the page back on a pagevec, nothing reaches the unevictable list; whereas the mm/munlock series under test fastidiously moves each page to unevictable before bringing it back. This certainly modifies my view of the role of the pagevec, and I think it indicates that we *may* find a regression in some realistic workload which demands a smarter approach. I have experimented with munlock_page() "cancelling" an mlock found earlier on its pagevec; but I very much doubt that the experimental code is correct yet, it worked well but not as well as hoped (there are various lru_add_drain()s which are limiting it), and it's just not a complication I'd like to get into: unless pushed to do so by a realistic workload. stress-ng --remap is not that realistic workload (and does not pretend to be). I'm glad that it has highlighted the issue, thanks to you, but I don't intend to propose any "fix" for this yet. Hugh --0-2040830854-1645425161=:2653--