From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ADA5EB64D7 for ; Wed, 21 Jun 2023 18:16:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E5428D0003; Wed, 21 Jun 2023 14:16:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 495478D0002; Wed, 21 Jun 2023 14:16:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35C408D0003; Wed, 21 Jun 2023 14:16:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2282C8D0002 for ; Wed, 21 Jun 2023 14:16:59 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DFECB1407E5 for ; Wed, 21 Jun 2023 18:16:58 +0000 (UTC) X-FDA: 80927561316.01.A64AA1B Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf16.hostedemail.com (Postfix) with ESMTP id 1ABEF18001A for ; Wed, 21 Jun 2023 18:16:56 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=gH6qZbnm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3lz6TZAcKCIwEAq3sxAw44w1u.s421y3AD-220Bqs0.47w@flex--yuanchu.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3lz6TZAcKCIwEAq3sxAw44w1u.s421y3AD-220Bqs0.47w@flex--yuanchu.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687371417; a=rsa-sha256; cv=none; b=7bPlRc3YdcRCnwFshGy+6EOgf9FHyeIkoXV3enZIno6hpdBhERQuXEJQwCFb1DyzmzgR+R HXak+UxYytjm+tUCzwlbUO4w0SAHXZ3XnaX+Wa80jiL30vIYRokfFcmBYHrutm/5QxSQ2W X6ghZy24Wb2+Oq8n/CReEb5hntdN53o= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=gH6qZbnm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3lz6TZAcKCIwEAq3sxAw44w1u.s421y3AD-220Bqs0.47w@flex--yuanchu.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3lz6TZAcKCIwEAq3sxAw44w1u.s421y3AD-220Bqs0.47w@flex--yuanchu.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687371417; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=SErWj7qxRQBNUC6Smh628AB0v7WlXXsygGd167Tdi3o=; b=gKznuQhmAjdt7ZBvRq43UtDD71gRgUxwAdLLKel2thhTXUUciT1Xseenp6qQokn5J2zwNn HADnOjD6Ed36/bMmRE4DMeYyImBwxpHnYtfH/+faksqDw6iXlK4y3KKaujPvvnvI4vWWWY HSDXUIDdjijRrwziE3r0E8rFGrhYJAA= Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-66700a28586so2160060b3a.2 for ; Wed, 21 Jun 2023 11:16:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687371415; x=1689963415; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=SErWj7qxRQBNUC6Smh628AB0v7WlXXsygGd167Tdi3o=; b=gH6qZbnmbvzhnYXoZHM0cADGdDaQIJB+Zb3K2J/nt0RJuILN8TTUQfNKhL1NoOpRoz 4eVB/B6O+ns16KrHXkmaeikTQ1b4ePaYZisr/NBGvmSQq0hmkMR56XmPQ9zVAPgyYpmv NWCVADLarekZTb/YF8tUjVsARrtG4eYeUd7AA+vBvQZiMqWNtFY3+HGIkyHzwljwaele W05jZa+gTCAsFM8uZ+8pwELeyZ6jb1Kp2qCG2b0T90gXpNA9TB2I4C8u3P2k6nyj8VqU jSnRcjmfT6h2C4RkMJHux+0cYzROGVEdGqr7gLxMMg1pIBqe2KL8660Qf2/lKBB+AAlN vztQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687371415; x=1689963415; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=SErWj7qxRQBNUC6Smh628AB0v7WlXXsygGd167Tdi3o=; b=bhW2KAZ64dt957a0t3q7B7SM6MQzzvyzL3xJ2S15VMNng+sbAn1Z2VDeK3nEy2qSvb fU21zHPMOB1wyl5e0kAcp10cAFNBiYLFWfDo0LVJcFVUm2pbQr+bnAEvDI0eFA87Kwtv bR/vhVgaEazz93ija5JlEcnHcscvaiebKnxMkjFhzpIsExCvu1VVJU717EZUBqopTr8I QFEG3bn23IGY8ckLwrUbEHn5/21k0DQ0SO7kVYjBvv8ZPL8rIUfYKYPVK4QLAJp447IX igDWkc1Qoy/1A6AXfJdoCh8lC5yDPuxT9zCBM+8JobiVGaxMBoQJoJ7n6h0UmC2KL/Ss no6Q== X-Gm-Message-State: AC+VfDy20kw+xYmPb+bURYqfBQiFz+/j3vL+7sE7zxYE86hHG1Xovi8G MXGir0tceAvs1HEsG19OMLrf9VkhPDFV X-Google-Smtp-Source: ACHHUZ7nxJJH9K2lG/iQd5bdjKxZhjoz3HwnSi034CqmZTOfysPDXpRBX0XDcr7KKuoAxK436AtEeNkdKwqq X-Received: from yuanchu.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:4024]) (user=yuanchu job=sendgmr) by 2002:a05:6a00:22c7:b0:666:e954:8ab6 with SMTP id f7-20020a056a0022c700b00666e9548ab6mr3973259pfj.3.1687371415509; Wed, 21 Jun 2023 11:16:55 -0700 (PDT) Date: Wed, 21 Jun 2023 18:04:48 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621180454.973862-1-yuanchu@google.com> Subject: [RFC PATCH v2 0/6] mm: working set reporting From: Yuanchu Xie To: Greg Kroah-Hartman , "Rafael J . Wysocki" , "Michael S . Tsirkin" , David Hildenbrand , Jason Wang , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Yu Zhao , Kefeng Wang , Kairui Song , Yosry Ahmed , Yuanchu Xie , "T . J . Alumbaugh" Cc: Wei Xu , SeongJae Park , Sudarshan Rajagopalan , kai.huang@intel.com, hch@lst.de, jon@nutanix.com, Aneesh Kumar K V , Matthew Wilcox , Vasily Averin , linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 1ABEF18001A X-Stat-Signature: 5pjbsc5aef1ihpmxm3wyr6hfyy9839dd X-HE-Tag: 1687371416-47068 X-HE-Meta: U2FsdGVkX1+JmUEEttJLNhdFvT4FhK/FgiYo6+x0voaj/5JsRTdPz+GAOE8aGOY+HtQKhEYtkjE1EofM2VLIlFF2swuS25o5T508VMZi/F9zgp0gKdPfuaoM59E6QN5TzhBWk5XIIPyRAB/taoyS045o+VhuL90BrSU5uRDnQOlmnyr+pdAKU3/1HnyurH5CNUp5cVQI9wEyOrhAr+eho/QI+ebmMnwlrsEICYtGXKmH/bs4Bm/6F8+tXDNF79VcSO6AnvgvbQPf5Gd9AMEnFRX3Z5SkdNh1QkHmCJJfV9QE/slSUXYYRGzVvI7edDsd8F2MhwX8iG0yT+xA76cGt/mXBsCxdKSDSSZvEI/pRPWYs91plloJ9XiUvElg3zBNiltgLVutNeke4ZucXCkw4BBNlT3F5N4s8bRJbvPQp1X/BI1gjwtSLt8mO5ngAOTkrTZa9+ZrTR9byGk15ZdFuKYsOOhiaT9fqH7oBfcUItb+HPunysx/RAw8s6sA/lAxhixcX+oBAOfXz1LTrrwoGsLbW4FTXKCOuQAvsFWRO2Xog62BC0UXouUhh8sUQitaF0bFodMLotgTYDfkfI6Uc6xldm/usD8gqrGI3zChha3kDyXEwaBY5sM5dbs6g562bhrR7RuiL9qVxmiNe5tUIbZp9l/qCUbpNtHFd2D06mHVDCriM6S2SnmpyDoJLu0RNUrwQt39tgq/X0WPfOGUxvoUxiW+2yOKkE6MuXZHf8z9of8R9OIOQbCbAr0VBCyoBRoW4CMaKCyktn1zRXrPhVDL1MJF0vfDCZRHcFr6QP0GtzpUE0gS02ary7HT5yRPSQaFd5Ipp8oVn9iFW0lwH8gsHA/l90j6+HY7iSeJkubcAGr5QDW7kmbCdsbZy7bEPAYTjL+VUpOqgt23M1OADj4B4jEOePDoB4lAuH5wa4zl7dcYS3WgbdyIn3d/5QrpaimZOERz79JgoV7WNk6 v8idpu+q vj4tll0KSIT8aEAntkKxaqkCy2JqcoqLWS7iFp5FAlOEaNyfbA+e0PGuz0PkzfDD69mk4NFVFEmH1zXHI7+1qlCsCb6gfiocI/x5+u6KvUP7XJLNIGgnuTk8hxYdmCHkY1PIMr9dswh/YTeF3GQjfda6OSSaONK9E8EomKwfh90aRdJNVW97AD8ISzv9B/QcS67WPAqPdysGxq48oIZCaF/+UYEzoOb8mFf4pHojOZwc1jWGvK3CSjcEB1pGiFn9B5w+G/WZ1+FevE7XHi7z4+F4Xj7UxTEAHPnTpRNNhMNgw52Qd34ktqffrCUr8YVz1opLWzOawUQJ5b73U+4jrhFgsqhx3rOg57aRNfspmfZNB17Z+CiwkCEmWSaN4ee5TL4VF1KhJ3B/BVLVpE2L6FVH/1X+hj3VfMyw/kgBVM67fJ1nYjMBzokkTl1NUgxlzwu0NSMSIPirfLb36+lEK/lQ3lgEGgJIQ/u45ywHfXzOi3ttvLye5XNsNH2+fcVJGBSYXm15+2KwxomAb+PQgV/C4I5wrOdbffkLOAtOqxLQWp5NKNgbfEaUTnOYoULUq2cgLHXZG7ChUYEYXgT9K2V3hHh+l6y5/qh/JKEjpq2t6PZbKQIiZ8icaMcC/PE5drzX8sUSRK8o4fx/fEqYwRkZbN79g0JiKTIHICvyano/7aaNIe7VKDXU6cKL55wTPd/LwN+oO9hLI0Okp43jUcB8Zztgp6YyVQcycURrZ1o3gX7HTHQtPvgmrbwZIP580qW6P0FYtgkJqn4QDqOoYsDHidQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: RFC v1: https://lore.kernel.org/linux-mm/20230509185419.1088297-1-yuanchu@google.com/ For background and interfaces, see the RFC v1 posting. Changes from v1 -> v2: - Refactored the patchs into smaller pieces - Renamed interfaces and functions from wss to wsr (Working Set Reporting) - Fixed build errors when CONFIG_WSR is not set - Changed working_set_num_bins to u8 for virtio-balloon - Added support for per-NUMA node reporting for virtio-balloon The RFC adds CONFIG_WSR and requires MGLRU to function. T.J. and I aim to support the active/inactive LRU and working set estimation from the userspace as well. This series should be build with the following configs: CONFIG_LRU_GEN=y CONFIG_LRU_GEN_ENABLED=y CONFIG_VIRTIO_BALLOON=y CONFIG_WSR=y TODO list: - There's a hack in mm/vmscan.c that calls into the virtio-balloon driver, which doesn't work if CONFIG_VIRTIO_BALLOON=m. T.J. Alumbaugh (talumbau@google.com) and I plan on solving this problem with a working set notification mechanism that would allow multiple consumers to subscribe for working set changes. - memory.reaccess.histogram does not consider swapped out pages to be reaccessed. I plan to implement this with the shadow entry computed from mm/workingset.c. QEMU device implementation: https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg06617.html virtio-dev spec proposal v1 (v2 to be posted by T.J.): https://lore.kernel.org/virtio-dev/CABmGT5Hv6Jd_F9EoQqVMDo4w5=7wJYmS4wwYDqXK3wov44Tf=w@mail.gmail.com/ LSF/MM discussion slides: https://lore.kernel.org/linux-mm/CABmGT5HK9xHz=E4q4sECCD8XodP9DUcH0dMeQ8kznUQB5HTQhQ@mail.gmail.com/ T.J. Alumbaugh (1): virtio-balloon: Add Working Set reporting Yuanchu Xie (5): mm: aggregate working set information into histograms mm: add working set refresh threshold to rate-limit aggregation mm: report working set when under memory pressure mm: extend working set reporting to memcgs mm: add per-memcg reaccess histogram drivers/base/node.c | 3 + drivers/virtio/virtio_balloon.c | 288 +++++++++++++++++ include/linux/balloon_compaction.h | 3 + include/linux/memcontrol.h | 6 + include/linux/mmzone.h | 5 + include/linux/wsr.h | 114 +++++++ include/uapi/linux/virtio_balloon.h | 33 ++ mm/Kconfig | 7 + mm/Makefile | 1 + mm/internal.h | 12 + mm/memcontrol.c | 351 ++++++++++++++++++++- mm/mmzone.c | 3 + mm/vmscan.c | 194 +++++++++++- mm/wsr.c | 464 ++++++++++++++++++++++++++++ 14 files changed, 1480 insertions(+), 4 deletions(-) create mode 100644 include/linux/wsr.h create mode 100644 mm/wsr.c -- 2.41.0.162.gfafddb0af9-goog