From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEB41C433E0 for ; Tue, 2 Mar 2021 07:44:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AC6964DE5 for ; Tue, 2 Mar 2021 07:44:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381045AbhCBETu (ORCPT ); Mon, 1 Mar 2021 23:19:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344283AbhCBACS (ORCPT ); Mon, 1 Mar 2021 19:02:18 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11F3AC061756 for ; Mon, 1 Mar 2021 16:01:38 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id v62so20589172ybb.15 for ; Mon, 01 Mar 2021 16:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=9heSrccM4QtbO8zJbZROi2fke/pBPWI/6xjb8kl8EnA=; b=v6/QDeREsxB4ARFCeFWrSwJ2dSOaKDO4I9YTPff/uItNGTtUIPsya5PcsPffq9USJV GyLgJlAiJtpUdoBfsiCVqJ8PnpkRa85psOWf6na+KGzRfr2Lrk9fNewpTU5x6pczbof1 xnh7jFSPMCqUCA3rIJEHHdO9iQiZKpT/iXDIYNtVbOYVpbORcp+OEKp/seXIsX3DKu5z vz3jixxH+biRLNNZR5UwrHfpYr2uLeMroJEM1ECn5+/UPmTS5QfRMdLU9LU4k2ewYTsN 4RmHX1IZkJBxo51233WQAq/+zg+fU7F1yG06Tv4YSY9xxqIsE9C/ypSk8QzdibOfDWRq HqBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=9heSrccM4QtbO8zJbZROi2fke/pBPWI/6xjb8kl8EnA=; b=W1Kdjc4WRoDP+on2nMjzm5aMnvW+rwjo5mI9ijEC8IoCkVc8zBJCcpl7D4ohkwIhwO MNAF+g2Iam+pT7azqrYSvlW36zdvWwQiuVwqbxHew1znsgjWjm0rLhxH7/xtix/TNMRj BYlpiJ9FpAflFfNYUSFxU+RFiGTA8M/HMo+D/z5/lap6igv2LuuzIKd9HSlFIn4hDe5J rHxLGFleIsBziZIw/6SIj8sdVs7MtDsZJDg4QO06SX6urpy1CO9cai4SR2rErCbUYskG yUG2vofZYc3PEQFVtk5C0+UWW67oz5YvmnSrtRTkyMbj2KTmCycv92s3ryzKE72/kZvu 0gyQ== X-Gm-Message-State: AOAM533TIroRbBSTLwsJJa6XWyiEmKO7/Sp2U51V/H1/UfYusQdDy8iN PDifuaSkf0rVMbvlN191WB1V258LHKOuJWG4hRMr X-Google-Smtp-Source: ABdhPJwsjvTDQKynUq7mWB/rP6Sd9oEm0+6px27QpmseIPIUTtqRcvUs45V4lejD7oru/ynawP80FvtnrstPwSCt+XlZ Sender: "axelrasmussen via sendgmr" X-Received: from ajr0.svl.corp.google.com ([2620:15c:2cd:203:1998:8165:ca50:ab8d]) (user=axelrasmussen job=sendgmr) by 2002:a25:c0ca:: with SMTP id c193mr27567242ybf.74.1614643297301; Mon, 01 Mar 2021 16:01:37 -0800 (PST) Date: Mon, 1 Mar 2021 16:01:28 -0800 Message-Id: <20210302000133.272579-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.30.1.766.gb4fecdf3b7-goog Subject: [PATCH v2 0/5] userfaultfd: support minor fault handling for shmem From: Axel Rasmussen To: Alexander Viro , Andrea Arcangeli , Andrew Morton , Hugh Dickins , Jerome Glisse , Joe Perches , Lokesh Gidra , Mike Rapoport , Peter Xu , Shaohua Li , Shuah Khan , Wang Qing Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen , Brian Geffon , Cannon Matthews , "Dr . David Alan Gilbert" , David Rientjes , Michel Lespinasse , Mina Almasry , Oliver Upton Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Base ==== This series is based on top of my series which adds minor fault handling for hugetlbfs [1]. (And, therefore, it is based on 5.12-rc1 and Peter Xu's series for disabling huge pmd sharing as well.) [1] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen@google.com/T/#t Changelog ========= v1->v2: - For UFFDIO_CONTINUE, don't mess with page flags. Just use find_lock_page to get a locked page from the page cache, instead of doing __SetPageLocked. This fixes a VM_BUG_ON v1 hit when handling minor faults for THP-backed shmem (a tmpfs mounted with huge=always). Overview ======== See my original series linked above for a detailed overview of minor fault handling in general. The feature in this series works exactly like the hugetblfs version (from userspace's perspective). I'm sending this as a separate series because: - The original minor fault handling series has a full set of R-Bs, and seems close to being merged. So, it seems reasonable to start looking at this next step, which extends the basic functionality. - shmem is different enough that this series may require some additional work before it's ready, and I don't want to delay the original series unnecessarily by bundling them together. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra ) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." Axel Rasmussen (5): userfaultfd: support minor fault handling for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support fs/userfaultfd.c | 6 +- include/linux/shmem_fs.h | 26 +- include/uapi/linux/userfaultfd.h | 4 +- mm/memory.c | 8 +- mm/shmem.c | 92 +++---- mm/userfaultfd.c | 27 +- tools/testing/selftests/vm/userfaultfd.c | 322 +++++++++++++++-------- 7 files changed, 295 insertions(+), 190 deletions(-) -- 2.30.1.766.gb4fecdf3b7-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FB53C433DB for ; Tue, 2 Mar 2021 00:01:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8F226023B for ; Tue, 2 Mar 2021 00:01:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8F226023B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 587DC8D00B9; Mon, 1 Mar 2021 19:01:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 511A98D0063; Mon, 1 Mar 2021 19:01:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D8B58D00B9; Mon, 1 Mar 2021 19:01:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 209568D0063 for ; Mon, 1 Mar 2021 19:01:43 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C901F4853 for ; Tue, 2 Mar 2021 00:01:42 +0000 (UTC) X-FDA: 77872980444.28.9C5B2A4 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf28.hostedemail.com (Postfix) with ESMTP id 44D1F200A757 for ; Tue, 2 Mar 2021 00:01:38 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id s187so20730989ybs.22 for ; Mon, 01 Mar 2021 16:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=9heSrccM4QtbO8zJbZROi2fke/pBPWI/6xjb8kl8EnA=; b=v6/QDeREsxB4ARFCeFWrSwJ2dSOaKDO4I9YTPff/uItNGTtUIPsya5PcsPffq9USJV GyLgJlAiJtpUdoBfsiCVqJ8PnpkRa85psOWf6na+KGzRfr2Lrk9fNewpTU5x6pczbof1 xnh7jFSPMCqUCA3rIJEHHdO9iQiZKpT/iXDIYNtVbOYVpbORcp+OEKp/seXIsX3DKu5z vz3jixxH+biRLNNZR5UwrHfpYr2uLeMroJEM1ECn5+/UPmTS5QfRMdLU9LU4k2ewYTsN 4RmHX1IZkJBxo51233WQAq/+zg+fU7F1yG06Tv4YSY9xxqIsE9C/ypSk8QzdibOfDWRq HqBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=9heSrccM4QtbO8zJbZROi2fke/pBPWI/6xjb8kl8EnA=; b=qCR8TsiVfrXyzj3OjSAkVa31khGcLHh1gqxznUcHN5Tm8MwoylbVs1cHxy24w56Bkf oRt03LCJP8Tr/rA7bO+pmbM//pskcJL8G4wV96dV8++R9jnzO4fvJY5pUs4TNMZfdQJe Erl6NOA/zUVE1UkpuWoXZvL/LKx9CBr3S9jkyX5t5eTtvlPrG1BvGBali93IU/TaG9HA E28eNgHk9/0FQiyttvbsQ8DcrZ/QMVaXt2itUqNhT1Nq7FedShCngzSRLlZxId5VI5m0 b6fBWiLueLN2Sgabr9tjQArJxHzr7iTWZvLCIHniDSWKYSA+RbIUgNccKV/TapORTJtO DpvQ== X-Gm-Message-State: AOAM533wVfyNb0UDHVl+hvfmi6A50mF53QQcG/vg8zJuP1KMBsIqC3Ig aXvU06tDZGRm11ghlHjX9dLFz2Xmg/wrx7kU+vK2 X-Google-Smtp-Source: ABdhPJwsjvTDQKynUq7mWB/rP6Sd9oEm0+6px27QpmseIPIUTtqRcvUs45V4lejD7oru/ynawP80FvtnrstPwSCt+XlZ X-Received: from ajr0.svl.corp.google.com ([2620:15c:2cd:203:1998:8165:ca50:ab8d]) (user=axelrasmussen job=sendgmr) by 2002:a25:c0ca:: with SMTP id c193mr27567242ybf.74.1614643297301; Mon, 01 Mar 2021 16:01:37 -0800 (PST) Date: Mon, 1 Mar 2021 16:01:28 -0800 Message-Id: <20210302000133.272579-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.30.1.766.gb4fecdf3b7-goog Subject: [PATCH v2 0/5] userfaultfd: support minor fault handling for shmem From: Axel Rasmussen To: Alexander Viro , Andrea Arcangeli , Andrew Morton , Hugh Dickins , Jerome Glisse , Joe Perches , Lokesh Gidra , Mike Rapoport , Peter Xu , Shaohua Li , Shuah Khan , Wang Qing Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen , Brian Geffon , Cannon Matthews , "Dr . David Alan Gilbert" , David Rientjes , Michel Lespinasse , Mina Almasry , Oliver Upton Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 44D1F200A757 X-Stat-Signature: jd6n4b64kt4qghmetppiw7zdqq1uot8g Received-SPF: none (flex--axelrasmussen.bounces.google.com>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from="<3YYA9YA0KCGA8VCJP8QKSQQCLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--axelrasmussen.bounces.google.com>"; helo=mail-yb1-f202.google.com; client-ip=209.85.219.202 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614643298-66738 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Base ==== This series is based on top of my series which adds minor fault handling for hugetlbfs [1]. (And, therefore, it is based on 5.12-rc1 and Peter Xu's series for disabling huge pmd sharing as well.) [1] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen@google.com/T/#t Changelog ========= v1->v2: - For UFFDIO_CONTINUE, don't mess with page flags. Just use find_lock_page to get a locked page from the page cache, instead of doing __SetPageLocked. This fixes a VM_BUG_ON v1 hit when handling minor faults for THP-backed shmem (a tmpfs mounted with huge=always). Overview ======== See my original series linked above for a detailed overview of minor fault handling in general. The feature in this series works exactly like the hugetblfs version (from userspace's perspective). I'm sending this as a separate series because: - The original minor fault handling series has a full set of R-Bs, and seems close to being merged. So, it seems reasonable to start looking at this next step, which extends the basic functionality. - shmem is different enough that this series may require some additional work before it's ready, and I don't want to delay the original series unnecessarily by bundling them together. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra ) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." Axel Rasmussen (5): userfaultfd: support minor fault handling for shmem userfaultfd/selftests: use memfd_create for shmem test type userfaultfd/selftests: create alias mappings in the shmem test userfaultfd/selftests: reinitialize test context in each test userfaultfd/selftests: exercise minor fault handling shmem support fs/userfaultfd.c | 6 +- include/linux/shmem_fs.h | 26 +- include/uapi/linux/userfaultfd.h | 4 +- mm/memory.c | 8 +- mm/shmem.c | 92 +++---- mm/userfaultfd.c | 27 +- tools/testing/selftests/vm/userfaultfd.c | 322 +++++++++++++++-------- 7 files changed, 295 insertions(+), 190 deletions(-) -- 2.30.1.766.gb4fecdf3b7-goog