From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 339FCC433DB for ; Fri, 19 Mar 2021 01:21:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA9D164E04 for ; Fri, 19 Mar 2021 01:21:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232253AbhCSBVY (ORCPT ); Thu, 18 Mar 2021 21:21:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232294AbhCSBVM (ORCPT ); Thu, 18 Mar 2021 21:21:12 -0400 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C14DC06174A for ; Thu, 18 Mar 2021 18:21:12 -0700 (PDT) Received: by mail-qk1-x735.google.com with SMTP id g20so1346720qkk.1 for ; Thu, 18 Mar 2021 18:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=suObc3/+Vf1EytucrxjvYtf/CBrpHkGZ5JxHoK4z38E=; b=bhJcjepUfXuLEoJOGLGkpyQu0P6brTjnnbhvfmYXw7eHdG3fIGA3Hbh/OsxZBdzFmu gFVLug0cMvQ10NA++5eqgG1dgoYJKY0i/krruCcE9nM7PVn488/q6l5TwekudWhDsQr8 qaImdyjuzs7hb2Qzru8Wp3MP1ufLaCJIzFkWA0XlMXhzidZgAbJ7n6MNkQ5VbgPyiUcx efNpSwQArhHoDqxp5S2D0nTNxPqbBpRx2gmtZZzO8QCxAbqpOK0XGEteJZBBp5yOCp/e 5QsmD7NLKR1zC+4g2CFnDL/mPwurAUQHgvGmAeQmzQd7I6o2faJZC5UvLuaB+rlVYPQp dEjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=suObc3/+Vf1EytucrxjvYtf/CBrpHkGZ5JxHoK4z38E=; b=L21GtGOySi042d0XOcDPHeMIdqEFmkNAZwq6tBooUF9cEUQIf518RT05w7VlEmBVOi 3E5Kk8mZrwd4S0FyQCI6u4jezLMag00EGqd9cigFDecSfuvioxLUkKCv81wkp9f2zDNs bqQ36FB58SHOSRNdepCyTWffep6YAZJV4pYbUBGKCIvVglIo3B+9tWp62cOrhcjOGe8U eWl1i62BT+7thkktFZemX9zcDgdA8hK4E0Mf+O5nwfdnjXMF7/zFYgOwMnvBzKnD2jLk HHzG+0IBFoAlEOj5m5tLeAypTaw0kbSTtsK6ZuU48YQIhkN9FtSEjC4pjjGt9MHZD6aJ xwfg== X-Gm-Message-State: AOAM531QfZhL/eWUBImZo4zF+LCBhvGSmXv+uFUxieOmoa9QHBIFuDEw jK7J+TGLvr/53uO3DIv1W7qkLQ== X-Google-Smtp-Source: ABdhPJzxaTzfnuMMfHVnGVJvkXMVIR6OJ9/8nydBOwNcHky3r1Qs+nKNrDbEYoVcwP2Pp6h8IPKEYw== X-Received: by 2002:a05:620a:122c:: with SMTP id v12mr7154881qkj.344.1616116871126; Thu, 18 Mar 2021 18:21:11 -0700 (PDT) Received: from eggly.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id v35sm2807770qtd.56.2021.03.18.18.21.09 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Thu, 18 Mar 2021 18:21:10 -0700 (PDT) Date: Thu, 18 Mar 2021 18:20:56 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Brian Geffon cc: Andrew Morton , Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" , Dmitry Safonov Subject: Re: [PATCH v3 1/2] mm: Allow non-VM_DONTEXPAND and VM_PFNMAP mappings with MREMAP_DONTUNMAP In-Reply-To: <20210317214147.682687-1-bgeffon@google.com> Message-ID: References: <20210303175235.3308220-1-bgeffon@google.com> <20210317214147.682687-1-bgeffon@google.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If Andrew is happy with such a long patch name, okay; but personally I'd prefer brevity to all that detail: mm: Extend MREMAP_DONTUNMAP to non-anonymous mappings On Wed, 17 Mar 2021, Brian Geffon wrote: > Currently MREMAP_DONTUNMAP only accepts private anonymous mappings. This > change will widen the support to include any mappings which are not > VM_DONTEXPAND or VM_PFNMAP. The primary use case is to support > MREMAP_DONTUNMAP on mappings which may have been created from a memfd. > > This change will result in mremap(MREMAP_DONTUNMAP) returning -EINVAL > if VM_DONTEXPAND or VM_PFNMAP mappings are specified. > > Lokesh Gidra who works on the Android JVM, provided an explanation of how > such a feature will improve Android JVM garbage collection: > "Android is developing a new garbage collector (GC), based on userfaultfd. > The garbage collector will use userfaultfd (uffd) on the java heap during > compaction. On accessing any uncompacted page, the application threads will > find it missing, at which point the thread will create the compacted page > and then use UFFDIO_COPY ioctl to get it mapped and then resume execution. > Before starting this compaction, in a stop-the-world pause the heap will be > mremap(MREMAP_DONTUNMAP) so that the java heap is ready to receive > UFFD_EVENT_PAGEFAULT events after resuming execution. > > To speedup mremap operations, pagetable movement was optimized by moving > PUD entries instead of PTE entries [1]. It was necessary as mremap of even > modest sized memory ranges also took several milliseconds, and stopping the > application for that long isn't acceptable in response-time sensitive > cases. > > With UFFDIO_CONTINUE feature [2], it will be even more efficient to > implement this GC, particularly the 'non-moveable' portions of the heap. > It will also help in reducing the need to copy (UFFDIO_COPY) the pages. > However, for this to work, the java heap has to be on a 'shared' vma. > Currently MREMAP_DONTUNMAP only supports private anonymous mappings, this > patch will enable using UFFDIO_CONTINUE for the new userfaultfd-based heap > compaction." > > [1] https://lore.kernel.org/linux-mm/20201215030730.NC3CU98e4%25akpm@linux-foundation.org/ > [2] https://lore.kernel.org/linux-mm/20210302000133.272579-1-axelrasmussen@google.com/ > > Signed-off-by: Brian Geffon Acked-by: Hugh Dickins Thanks Brian, just what I wanted :) You wondered in another mail about this returning -EINVAL whereas the VM_DONTEXPAND size error returns -EFAULT: I've pondered, and I've read the manpage, and I'm sure it would be wrong to change the old -EFAULT to -EINVAL now; and I don't see good reason to change your -EINVAL to -EFAULT either. Let them differ, that's okay (and it's only in special corner cases that either of these fail anyway). > --- > mm/mremap.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/mm/mremap.c b/mm/mremap.c > index ec8f840399ed..db5b8b28c2dd 100644 > --- a/mm/mremap.c > +++ b/mm/mremap.c > @@ -653,8 +653,8 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, > return ERR_PTR(-EINVAL); > } > > - if (flags & MREMAP_DONTUNMAP && (!vma_is_anonymous(vma) || > - vma->vm_flags & VM_SHARED)) > + if ((flags & MREMAP_DONTUNMAP) && > + (vma->vm_flags & (VM_DONTEXPAND | VM_PFNMAP))) > return ERR_PTR(-EINVAL); > > if (is_vm_hugetlb_page(vma)) > -- > 2.31.0.rc2.261.g7f71774620-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41C16C433E0 for ; Fri, 19 Mar 2021 01:21:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AAD1E64E01 for ; Fri, 19 Mar 2021 01:21:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAD1E64E01 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ECA2C6B006E; Thu, 18 Mar 2021 21:21:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E7A2F6B0071; Thu, 18 Mar 2021 21:21:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFF836B0072; Thu, 18 Mar 2021 21:21:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id B0EC56B006E for ; Thu, 18 Mar 2021 21:21:20 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 6A9CE1813F55A for ; Fri, 19 Mar 2021 01:21:20 +0000 (UTC) X-FDA: 77934870720.28.642EAFB Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by imf18.hostedemail.com (Postfix) with ESMTP id 3FE06200038C for ; Fri, 19 Mar 2021 01:21:12 +0000 (UTC) Received: by mail-qk1-f170.google.com with SMTP id x14so1347702qki.10 for ; Thu, 18 Mar 2021 18:21:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=suObc3/+Vf1EytucrxjvYtf/CBrpHkGZ5JxHoK4z38E=; b=bhJcjepUfXuLEoJOGLGkpyQu0P6brTjnnbhvfmYXw7eHdG3fIGA3Hbh/OsxZBdzFmu gFVLug0cMvQ10NA++5eqgG1dgoYJKY0i/krruCcE9nM7PVn488/q6l5TwekudWhDsQr8 qaImdyjuzs7hb2Qzru8Wp3MP1ufLaCJIzFkWA0XlMXhzidZgAbJ7n6MNkQ5VbgPyiUcx efNpSwQArhHoDqxp5S2D0nTNxPqbBpRx2gmtZZzO8QCxAbqpOK0XGEteJZBBp5yOCp/e 5QsmD7NLKR1zC+4g2CFnDL/mPwurAUQHgvGmAeQmzQd7I6o2faJZC5UvLuaB+rlVYPQp dEjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=suObc3/+Vf1EytucrxjvYtf/CBrpHkGZ5JxHoK4z38E=; b=YT8iuEtoDmrobmDDVAQQE2Z7akMcKHwI/KYMaYLiyBgdb5F5pBUWN7AqUEWAruHTH7 mWQD9ftJGE9nyMo1By6jy5jFRQAkYiIr+8SBmCRMH1HkgRvcZWBkaOoJWwV+lCZQRNqw ikJDVmtuy9BqtOgDx6ECh/YyyJfQ0oQayO0oIWGgYYKGANBvhR+PiJScE+LPGMa06arL ybbWrS6H3duEH1wE9RYRb9S+0B+c6AU4t6fCoZqLwlKuAHVHPnVdPsYWfUxplm2VE6Ok ujfonWdyaaI+IDWK9FDbJ4FksnLCNIZ7SE5oJHMVUw4beoHlt/Vb274wCbfjSttcMJ/c QiPg== X-Gm-Message-State: AOAM533ceVxArHD8XKPbqn0Bj0wKXYLkXPZl2F4dLVeUEEMHYRLQlTE7 hbqicBmqgdVSdEugDL64NLX06g== X-Google-Smtp-Source: ABdhPJzxaTzfnuMMfHVnGVJvkXMVIR6OJ9/8nydBOwNcHky3r1Qs+nKNrDbEYoVcwP2Pp6h8IPKEYw== X-Received: by 2002:a05:620a:122c:: with SMTP id v12mr7154881qkj.344.1616116871126; Thu, 18 Mar 2021 18:21:11 -0700 (PDT) Received: from eggly.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id v35sm2807770qtd.56.2021.03.18.18.21.09 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Thu, 18 Mar 2021 18:21:10 -0700 (PDT) Date: Thu, 18 Mar 2021 18:20:56 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Brian Geffon cc: Andrew Morton , Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" , Dmitry Safonov Subject: Re: [PATCH v3 1/2] mm: Allow non-VM_DONTEXPAND and VM_PFNMAP mappings with MREMAP_DONTUNMAP In-Reply-To: <20210317214147.682687-1-bgeffon@google.com> Message-ID: References: <20210303175235.3308220-1-bgeffon@google.com> <20210317214147.682687-1-bgeffon@google.com> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Stat-Signature: u8aqurehi4z9b5wcdd8xijyw95mnbzq9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3FE06200038C Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-qk1-f170.google.com; client-ip=209.85.222.170 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616116872-430460 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If Andrew is happy with such a long patch name, okay; but personally I'd prefer brevity to all that detail: mm: Extend MREMAP_DONTUNMAP to non-anonymous mappings On Wed, 17 Mar 2021, Brian Geffon wrote: > Currently MREMAP_DONTUNMAP only accepts private anonymous mappings. This > change will widen the support to include any mappings which are not > VM_DONTEXPAND or VM_PFNMAP. The primary use case is to support > MREMAP_DONTUNMAP on mappings which may have been created from a memfd. > > This change will result in mremap(MREMAP_DONTUNMAP) returning -EINVAL > if VM_DONTEXPAND or VM_PFNMAP mappings are specified. > > Lokesh Gidra who works on the Android JVM, provided an explanation of how > such a feature will improve Android JVM garbage collection: > "Android is developing a new garbage collector (GC), based on userfaultfd. > The garbage collector will use userfaultfd (uffd) on the java heap during > compaction. On accessing any uncompacted page, the application threads will > find it missing, at which point the thread will create the compacted page > and then use UFFDIO_COPY ioctl to get it mapped and then resume execution. > Before starting this compaction, in a stop-the-world pause the heap will be > mremap(MREMAP_DONTUNMAP) so that the java heap is ready to receive > UFFD_EVENT_PAGEFAULT events after resuming execution. > > To speedup mremap operations, pagetable movement was optimized by moving > PUD entries instead of PTE entries [1]. It was necessary as mremap of even > modest sized memory ranges also took several milliseconds, and stopping the > application for that long isn't acceptable in response-time sensitive > cases. > > With UFFDIO_CONTINUE feature [2], it will be even more efficient to > implement this GC, particularly the 'non-moveable' portions of the heap. > It will also help in reducing the need to copy (UFFDIO_COPY) the pages. > However, for this to work, the java heap has to be on a 'shared' vma. > Currently MREMAP_DONTUNMAP only supports private anonymous mappings, this > patch will enable using UFFDIO_CONTINUE for the new userfaultfd-based heap > compaction." > > [1] https://lore.kernel.org/linux-mm/20201215030730.NC3CU98e4%25akpm@linux-foundation.org/ > [2] https://lore.kernel.org/linux-mm/20210302000133.272579-1-axelrasmussen@google.com/ > > Signed-off-by: Brian Geffon Acked-by: Hugh Dickins Thanks Brian, just what I wanted :) You wondered in another mail about this returning -EINVAL whereas the VM_DONTEXPAND size error returns -EFAULT: I've pondered, and I've read the manpage, and I'm sure it would be wrong to change the old -EFAULT to -EINVAL now; and I don't see good reason to change your -EINVAL to -EFAULT either. Let them differ, that's okay (and it's only in special corner cases that either of these fail anyway). > --- > mm/mremap.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/mm/mremap.c b/mm/mremap.c > index ec8f840399ed..db5b8b28c2dd 100644 > --- a/mm/mremap.c > +++ b/mm/mremap.c > @@ -653,8 +653,8 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, > return ERR_PTR(-EINVAL); > } > > - if (flags & MREMAP_DONTUNMAP && (!vma_is_anonymous(vma) || > - vma->vm_flags & VM_SHARED)) > + if ((flags & MREMAP_DONTUNMAP) && > + (vma->vm_flags & (VM_DONTEXPAND | VM_PFNMAP))) > return ERR_PTR(-EINVAL); > > if (is_vm_hugetlb_page(vma)) > -- > 2.31.0.rc2.261.g7f71774620-goog