From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C08ECAAD1 for ; Wed, 31 Aug 2022 21:47:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F39106B0072; Wed, 31 Aug 2022 17:47:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC2E36B0073; Wed, 31 Aug 2022 17:47:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3B938D0001; Wed, 31 Aug 2022 17:47:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BF0AB6B0072 for ; Wed, 31 Aug 2022 17:47:29 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9D95EA143E for ; Wed, 31 Aug 2022 21:47:29 +0000 (UTC) X-FDA: 79861224618.23.0FCFA61 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf18.hostedemail.com (Postfix) with ESMTP id 50DD51C003F for ; Wed, 31 Aug 2022 21:47:29 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id q15so10334700pfn.11 for ; Wed, 31 Aug 2022 14:47:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=ELixXInJVYw7QNOqvSmbAXONuMRK6cy3zM9Z/tGjGTI=; b=C/jS+be9Q+YMOi1N3uYIuS8tm9uYTNrB0ZSLSLBTTKZlPagUMDisDD9dQJjcXN8l/4 XAIAQwooCGhp+NkoYNU0ikqCXj8zOsL5cVRO6XN/bYuBLX3W1ePzLS+Tcs9BswIgG+gF eAcDQuP8VkVlvyLv/VMfePE0SlDdLAsFwWmtR4X0XDFAPCypLdD2N7aMoaTkzFDxoZ27 CZaTtcHuhLe/NkTFlzywJHzMrLORetyQI3DnqD7cqMFOZhZV9ieQOUnom4I9Z9c1AQ8s 5nUe4nltIy7/M8kOZFamKTR0lLP0XR5FKNBBDMTQTyO/n/wn4gFfkxRkK7x64PR8t67J cZBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=ELixXInJVYw7QNOqvSmbAXONuMRK6cy3zM9Z/tGjGTI=; b=6njU87MCXEHNDUAg2NXH54gXOEYlW6NEHdPwV8biIkPcqcYO2DQz8x6kUwOwEm+sTQ Su4wYYzWsof/Lv+JrCd7kBqcJNFFic8z5i0v/JniT2KM4aH+o3c5mJTq34rJRWEC1gAU QY1Mw9iToc/MvtfKlbUbQZZJgKYUmgEitcxkofOyUkH6NdqED+hbn60mgvWGxCaNQK3j hEnuqIgPUlDOY11oO4pUeHl7+NA4rgZ99YN/W6TCaP+1tj1z322MpKXfCM3DdfLeraEW zuurcjmu4cyu9pdm/632UrjpRmUug64CWW92smZ9tFb32Y3JWr6EWX8v6N90ZlJ11qQO WWaQ== X-Gm-Message-State: ACgBeo2h67ZSf3nMbKFlmTKbNKIGH9QPPhtG0enrZtjvC1lYrYy8lEtn pcSjK1ZedLImZjy40KyV0b2G61+XiIR5wzzlMz4= X-Google-Smtp-Source: AA6agR5ra5USkJUdskHOKQoaMlkJskiIbdCsZIHLzh3CgQDh9+ZhY5dGpUkzghnIqepLPWl3f0poG6t4QKfzt+J3KCQ= X-Received: by 2002:a05:6a00:4393:b0:52f:3603:e62f with SMTP id bt19-20020a056a00439300b0052f3603e62fmr27986300pfb.23.1661982448387; Wed, 31 Aug 2022 14:47:28 -0700 (PDT) MIME-Version: 1.0 References: <20220826220329.1495407-1-zokeefe@google.com> In-Reply-To: <20220826220329.1495407-1-zokeefe@google.com> From: Yang Shi Date: Wed, 31 Aug 2022 14:47:16 -0700 Message-ID: Subject: Re: To: "Zach O'Keefe" Cc: linux-mm@kvack.org, Andrew Morton , linux-api@vger.kernel.org, Axel Rasmussen , James Houghton , Hugh Dickins , Miaohe Lin , David Hildenbrand , David Rientjes , Matthew Wilcox , Pasha Tatashin , Peter Xu , Rongwei Wang , SeongJae Park , Song Liu , Vlastimil Babka , Chris Kennelly , "Kirill A. Shutemov" , Minchan Kim , Patrick Xia Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661982449; a=rsa-sha256; cv=none; b=NSDqpwVzvAYhli8LOzXc7jEkwwJPkfbJG/uhG7urPORcF/vQjxfTNEHNom7cj+U4UiKqBJ h+2CKYF7dPb7sQZ9bvAeBJJ4YZ0jnEzfZV++z+8k4RTIoKD9urU5Hw671KbbZPlCaxlW00 sT8no74rPA3nGnrhWCBqjD3OZvG1UXU= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="C/jS+be9"; spf=pass (imf18.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661982449; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ELixXInJVYw7QNOqvSmbAXONuMRK6cy3zM9Z/tGjGTI=; b=k78mHE/Y/sfxtqK3Mv4q0kep1KK9FolUtOi3Divb6hyc7Rl3GxtChHgxKohradGI6Q61pk bEXqgFqTHx4fixcjPIB/VI0NP9mTmokHFgr8H/F7YnoLJzn5KanFoe8nhMIQfTIQglDFhr VsK30BQX7TEMleHyYd/dL94FRJ4UozY= X-Stat-Signature: j65mukr61d6gztga6g6c8b88b36hfwsw X-Rspamd-Queue-Id: 50DD51C003F Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="C/jS+be9"; spf=pass (imf18.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1661982449-747226 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Zach, I did a quick look at the series, basically no show stopper to me. But I didn't find time to review them thoroughly yet, quite busy on something else. Just a heads up, I didn't mean to ignore you. I will review them when I find some time. Thanks, Yang On Fri, Aug 26, 2022 at 3:03 PM Zach O'Keefe wrote: > > Subject: [PATCH mm-unstable v2 0/9] mm: add file/shmem support to MADV_COLLAPSE > > v2 Forward > > Mostly a RESEND: rebase on latest mm-unstable + minor bug fixes from > kernel test robot. > -------------------------------- > > This series builds on top of the previous "mm: userspace hugepage collapse" > series which introduced the MADV_COLLAPSE madvise mode and added support > for private, anonymous mappings[1], by adding support for file and shmem > backed memory to CONFIG_READ_ONLY_THP_FOR_FS=y kernels. > > File and shmem support have been added with effort to align with existing > MADV_COLLAPSE semantics and policy decisions[2]. Collapse of shmem-backed > memory ignores kernel-guiding directives and heuristics including all > sysfs settings (transparent_hugepage/shmem_enabled), and tmpfs huge= mount > options (shmem always supports large folios). Like anonymous mappings, on > successful return of MADV_COLLAPSE on file/shmem memory, the contents of > memory mapped by the addresses provided will be synchronously pmd-mapped > THPs. > > This functionality unlocks two important uses: > > (1) Immediately back executable text by THPs. Current support provided > by CONFIG_READ_ONLY_THP_FOR_FS may take a long time on a large > system which might impair services from serving at their full rated > load after (re)starting. Tricks like mremap(2)'ing text onto > anonymous memory to immediately realize iTLB performance prevents > page sharing and demand paging, both of which increase steady state > memory footprint. Now, we can have the best of both worlds: Peak > upfront performance and lower RAM footprints. > > (2) userfaultfd-based live migration of virtual machines satisfy UFFD > faults by fetching native-sized pages over the network (to avoid > latency of transferring an entire hugepage). However, after guest > memory has been fully copied to the new host, MADV_COLLAPSE can > be used to immediately increase guest performance. > > khugepaged has received a small improvement by association and can now > detect and collapse pte-mapped THPs. However, there is still work to be > done along the file collapse path. Compound pages of arbitrary order still > needs to be supported and THP collapse needs to be converted to using > folios in general. Eventually, we'd like to move away from the read-only > and executable-mapped constraints currently imposed on eligible files and > support any inode claiming huge folio support. That said, I think the > series as-is covers enough to claim that MADV_COLLAPSE supports file/shmem > memory. > > Patches 1-3 Implement the guts of the series. > Patch 4 Is a tracepoint for debugging. > Patches 5-8 Refactor existing khugepaged selftests to work with new > memory types. > Patch 9 Adds a userfaultfd selftest mode to mimic a functional test > of UFFDIO_REGISTER_MODE_MINOR+MADV_COLLAPSE live migration. > > Applies against mm-unstable. > > [1] https://lore.kernel.org/linux-mm/20220706235936.2197195-1-zokeefe@google.com/ > [2] https://lore.kernel.org/linux-mm/YtBmhaiPHUTkJml8@google.com/ > > v1 -> v2: > - Add missing definition for khugepaged_add_pte_mapped_thp() in > !CONFIG_SHEM builds, in "mm/khugepaged: attempt to map > file/shmem-backed pte-mapped THPs by pmds" > - Minor bugfixes in "mm/madvise: add file and shmem support to > MADV_COLLAPSE" for !CONFIG_SHMEM, !CONFIG_TRANSPARENT_HUGEPAGE and some > compiler settings. > - Rebased on latest mm-unstable > > Zach O'Keefe (9): > mm/shmem: add flag to enforce shmem THP in hugepage_vma_check() > mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by > pmds > mm/madvise: add file and shmem support to MADV_COLLAPSE > mm/khugepaged: add tracepoint to hpage_collapse_scan_file() > selftests/vm: dedup THP helpers > selftests/vm: modularize thp collapse memory operations > selftests/vm: add thp collapse file and tmpfs testing > selftests/vm: add thp collapse shmem testing > selftests/vm: add selftest for MADV_COLLAPSE of uffd-minor memory > > include/linux/khugepaged.h | 13 +- > include/linux/shmem_fs.h | 10 +- > include/trace/events/huge_memory.h | 36 + > kernel/events/uprobes.c | 2 +- > mm/huge_memory.c | 2 +- > mm/khugepaged.c | 289 ++++-- > mm/shmem.c | 18 +- > tools/testing/selftests/vm/Makefile | 2 + > tools/testing/selftests/vm/khugepaged.c | 828 ++++++++++++------ > tools/testing/selftests/vm/soft-dirty.c | 2 +- > .../selftests/vm/split_huge_page_test.c | 12 +- > tools/testing/selftests/vm/userfaultfd.c | 171 +++- > tools/testing/selftests/vm/vm_util.c | 36 +- > tools/testing/selftests/vm/vm_util.h | 5 +- > 14 files changed, 1040 insertions(+), 386 deletions(-) > > -- > 2.37.2.672.g94769d06f0-goog >