From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6EDAC2D0A8 for ; Mon, 28 Sep 2020 18:39:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F18EB207D8 for ; Mon, 28 Sep 2020 18:39:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="GmtvDFVa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F18EB207D8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F40E96B009B; Mon, 28 Sep 2020 14:39:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC9876B009C; Mon, 28 Sep 2020 14:39:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D91D16B009D; Mon, 28 Sep 2020 14:39:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id BBAC76B009B for ; Mon, 28 Sep 2020 14:39:31 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 80F27349B for ; Mon, 28 Sep 2020 18:39:31 +0000 (UTC) X-FDA: 77313333342.30.plate09_1a16ee027183 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 57931180B3C85 for ; Mon, 28 Sep 2020 18:39:31 +0000 (UTC) X-HE-Tag: plate09_1a16ee027183 X-Filterd-Recvd-Size: 7503 Received: from mail-qv1-f66.google.com (mail-qv1-f66.google.com [209.85.219.66]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Mon, 28 Sep 2020 18:39:30 +0000 (UTC) Received: by mail-qv1-f66.google.com with SMTP id b13so989060qvl.2 for ; Mon, 28 Sep 2020 11:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=QojT9TgDU+CsWYco4TCvXduWy2QP7Gs2vHLY0goYgnU=; b=GmtvDFVavvKQMUl/ma0Pm6Bl9VfREFQPF8KAUqbRMAQZfbs1cpdr0Q6viw5sIKGOye hoyU0SclRWDMng9a3aot9KTr5CNxs1PPbUS9VWEL3cfmRxYKb8fYOqpbyxmblb5uVRnJ mR1AJ6VQY4B/XOq79sgaEMSBlZBMQapjtSp/S1mPwBQATeLrkp5KYN1f81z7CAxP4zyM j92WLhH61mqdrg3vPCoKdoxkpej0D/qP+GMg90U6W1UvKUD/Z08trPUYA0FDB/OCwEtV qeAu3hZz56UI7ImUyoX0LeBPLXhXZHo0sMswyorPIcR6rjlCWi4XooxSmEMsp07yWeV6 5F3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=QojT9TgDU+CsWYco4TCvXduWy2QP7Gs2vHLY0goYgnU=; b=hKOVH14iyCNLL4Wi8gRJcq4P+hjQUGSRvHaUthP1gzvCg/KmqTrttI2sgQVFPyZzE7 Gj77KA/emztEdE//F90kls/kAiSMpD5R2pCXMHJVXnYDBluckwdpBOAH91JjZN+kuNrb S1x1pXm5OElwMZRUnMkvqZJqyBNPOVlE07FpDbGj6uX/Jn75ccmHUQdzhyo4YUytdRMc 7qt5PkJVGwcWxfHZZtAd8/kQrr5LFoKvg4nwv0hrx09kP1I0R70oO/jgWOs+DW1pBZM6 KdMG4U77yMxUTB7TEyeG1wkp2aiQohf7Bttozl2ZyX1l9r9zeKxjP9VeNhQFI7U9tzhQ IQiA== X-Gm-Message-State: AOAM531OoAK/MgmE3N2hbEqLuUwEpszlb1u/ah7xD3VYkUbY0KGYcdXD S9VVVN28ErUAhm9IRxUlmkWrXA== X-Google-Smtp-Source: ABdhPJyKC947gKvBbGydXx49IehA8jWfsTSAsHh1FpOQ1V10dmVJaKuVvYI/Ld9MTP7P2ziJUbmSgQ== X-Received: by 2002:a05:6214:1784:: with SMTP id ct4mr871185qvb.10.1601318369933; Mon, 28 Sep 2020 11:39:29 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-48-30.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.48.30]) by smtp.gmail.com with ESMTPSA id 192sm1853863qkm.110.2020.09.28.11.39.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Sep 2020 11:39:29 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1kMy3c-002TBq-Ju; Mon, 28 Sep 2020 15:39:28 -0300 Date: Mon, 28 Sep 2020 15:39:28 -0300 From: Jason Gunthorpe To: Linus Torvalds Cc: Peter Xu , Leon Romanovsky , John Hubbard , Linux-MM , Linux Kernel Mailing List , Andrew Morton , Jan Kara , Michal Hocko , Kirill Tkhai , Kirill Shutemov , Hugh Dickins , Christoph Hellwig , Andrea Arcangeli , Oleg Nesterov , Jann Horn Subject: Re: [PATCH 1/5] mm: Introduce mm_struct.has_pinned Message-ID: <20200928183928.GR9916@ziepe.ca> References: <20200926004136.GJ9916@ziepe.ca> <20200927062337.GE2280698@unreal> <20200928124937.GN9916@ziepe.ca> <20200928172256.GB59869@xz-x1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 28, 2020 at 10:54:28AM -0700, Linus Torvalds wrote: > On Mon, Sep 28, 2020 at 10:23 AM Peter Xu wrote: > > > > Yes... Actually I am also thinking about the complete solution to cover > > read-only fast-gups too, but now I start to doubt this, at least for the fork() > > path. E.g. if we'd finally like to use pte_protnone() to replace the current > > pte_wrprotect(), we'll be able to also block the read gups, but we'll suffer > > the same degradation on normal fork()s, or even more. Seems unacceptable. > > So I think the real question about pinned read gups is what semantics > they should have. > > Because honestly, I think we have two options: > > - the current "it gets a shared copy from the page tables" > > - the "this is an exclusive pin, and it _will_ follow the source VM > changes, and never break" The requirement depends on what the driver is doing. Consider a simple ring-to-device scheme: ring = mmap() pin_user_pages(FOLL_LONGTERM) ring[0] = [..] trigger_kernel() Sort of like iouring. Here the kernel will pin the zero page and will never see any user modifications to the buffer. We must do #2. While something like read(O_DIRECT) which only needs the buffer to be stable during a system call would be fine with #1 (and data incoherence in general) > In other words, a read pin isn't really any different from a read GUP. > You get a reference to a page that is valid at the time of the page > lookup, and absolutely nothing more. Yes, so far all this new pin stuff has really focused on the write side - the motivating issue was the set_page_dirty() oops > But honestly, that is largely going to _be_ the same as a write pin, > because it absolutely needs to do a page COW at the time of the > pinning to get that initial exclusive guarantee in the first place. > Without that initial exclusivity, you cannot avoid future COW events > breaking the wrong way. Right, I see places using FOLL_WRITE when they only need read. It is a bit ugly, IMHO. > The downside of a write pin is that it not only makes that page > exclusive, it also (a) marks it dirty and (b) requires write access. RDMA adds FOLL_FORCE because people complained they couldn't do read-only transfers from .rodata - uglyier still :\ > So the copy_page_range() code would do a write count around the copy: > > write_seqcount_begin(&mm->seq); > .. do the copy .. > write_seqcount_end(&mm->seq); All of gup_fast and copy_mm could be wrappered in a seq count so that gup_fast always goes to the slow path if fork is concurrent. That doesn't sound too expensive and avoids all the problems you pointed with the WP scheme. As you say fork & pin & threads is already questionable, so an unconditional slow path on race should be OK. > If we want to and really need to. I would like to see some reasonable definition for the read-side. Having drivers do FOLL_FORCE | FOLL_WRITE for read is just confusing and weird for a driver facing API. It may also help focus the remaining discussion for solving set_page_dirty() if pin_user_pages() had a solid definition. I prefer the version where read pin and write pin are symmetric. The PTE in the MM should not change once pinned. This is useful and if something only needs the original GUP semantics then GUP is still there. Jason