From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AABC3C46466 for ; Sun, 4 Oct 2020 11:20:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD5FB206A1 for ; Sun, 4 Oct 2020 11:20:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="f5dd3Mrk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD5FB206A1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CF3616B005D; Sun, 4 Oct 2020 07:20:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA5328E0001; Sun, 4 Oct 2020 07:20:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B93406B0068; Sun, 4 Oct 2020 07:20:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 8939C6B005D for ; Sun, 4 Oct 2020 07:20:44 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 11795180AD804 for ; Sun, 4 Oct 2020 11:20:44 +0000 (UTC) X-FDA: 77334000408.04.boat21_3f0daae271b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id E6327800A31E for ; Sun, 4 Oct 2020 11:20:43 +0000 (UTC) X-HE-Tag: boat21_3f0daae271b5 X-Filterd-Recvd-Size: 6446 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Sun, 4 Oct 2020 11:20:43 +0000 (UTC) Received: by mail-ot1-f49.google.com with SMTP id a2so5843204otr.11 for ; Sun, 04 Oct 2020 04:20:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=F1P39aVq/hZE1a/ZNFpkwqSichMZqIMoJHAVD0hctIo=; b=f5dd3MrkYcy8/aD7b+pp4n3Qd4ummEslPZLTRt6CmyxxeMAlfleGd5Dfp874p+qUpY j6kAiBeOvNPrJhgZA8ZwL8CfE9QHGJf7l9mEUa3q3twT+o7VwA7yWSbyIEkatqa7DjKZ 2o7kSRHm9MLEBW4gDBATa2dJOssUGc4hiGwFE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=F1P39aVq/hZE1a/ZNFpkwqSichMZqIMoJHAVD0hctIo=; b=pCWeUMuQPPx5v0M8KMOEHEPS+JlAb+YDyup/fl90+Hhl6EsNdNg1cWUhtYhFAiKI2A 9SYRq7H2VnxHo1Y6l+S2apVAdIXQ8v9+5gB5UOzWVrA3bgSEoTPup/zqwEJkTdgYcKxT /fwOqjR/rBRDwQPPM6zvkDjeU0SNhvDojQxGkF2BgCqOCQBcJbbSBKA3JQK7hmu44x5T jwFrvnc5H0AXBdGYD/0KHmekT3YJlMZZJDGoIfrdY80a+w525+7wb2lf+d/r6AWAUT1a +wb9ylOXm8gI7YoX4i5k0YCoTpaP82zRuWlp98fJFpie6Rkh0EWSBSce9g38OyKwqGVa XzLg== X-Gm-Message-State: AOAM53018BCudmBGqiFYlcW+ewUn4dG9eNe9x0qw+OTcteDXPZJe32UZ dEaiF7JGc2/nF7d9d0r4mGqRomyt7XI6d08k/npl8Q== X-Google-Smtp-Source: ABdhPJzk4utKZUHKkD2Pntaa4TtV+H5VfSo7BOy9dlnzPXQ7sPmOEnfTjNGZnPb4bUQ2llmVT9XDHSXvI1R3pQKQgMQ= X-Received: by 2002:a05:6830:1647:: with SMTP id h7mr777182otr.281.1601810442219; Sun, 04 Oct 2020 04:20:42 -0700 (PDT) MIME-Version: 1.0 References: <20201002175303.390363-1-daniel.vetter@ffwll.ch> <20201002175303.390363-2-daniel.vetter@ffwll.ch> <20201003232441.GO9916@ziepe.ca> In-Reply-To: <20201003232441.GO9916@ziepe.ca> From: Daniel Vetter Date: Sun, 4 Oct 2020 13:20:31 +0200 Message-ID: Subject: Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM To: Jason Gunthorpe Cc: John Hubbard , DRI Development , LKML , Daniel Vetter , Andrew Morton , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jan Kara , Dan Williams , Linux MM , Linux ARM , linux-samsung-soc , "open list:DMA BUFFER SHARING FRAMEWORK" Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Oct 4, 2020 at 1:24 AM Jason Gunthorpe wrote: > On Sat, Oct 03, 2020 at 03:52:32PM -0700, John Hubbard wrote: > > On 10/3/20 2:45 AM, Daniel Vetter wrote: > > > On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: > > > > > > > > On 10/2/20 10:53 AM, Daniel Vetter wrote: > > > > > For $reasons I've stumbled over this code and I'm not sure the change > > > > > to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: > > > > > convert get_user_pages() --> pin_user_pages()") was entirely correct. > > > > > > > > > > This here is used for long term buffers (not just quick I/O) like > > > > > RDMA, and John notes this in his patch. But I thought the rule for > > > > > these is that they need to add FOLL_LONGTERM, which John's patch > > > > > didn't do. > > > > > > > > Yep. The earlier gup --> pup conversion patches were intended to not > > > > have any noticeable behavior changes, and FOLL_LONGTERM, with it's > > > > special cases and such, added some risk that I wasn't ready to take > > > > on yet. Also, FOLL_LONGTERM rules are only *recently* getting firmed > > > > up. So there was some doubt at least in my mind, about which sites > > > > should have it. > > > > > > > > But now that we're here, I think it's really good that you've brought > > > > this up. It's definitely time to add FOLL_LONGTERM wherever it's missing. > > > > > > So should I keep this patch, or will it collide with a series you're working on? > > > > It doesn't collide with anything on my end yet, because I've been slow to > > pick up on the need for changing callsites to add FOLL_LONGTERM. :) > > > > And it looks like that's actually a problem, because: > > > > > > > > Also with the firmed up rules, correct that I can also drop the > > > vma_is_fsdax check when the FOLL_LONGTERM flag is set? > > > > That's the right direction to go *in general*, but I see that the > > pin_user_pages code is still a bit stuck in the past. And this patch > > won't actually work, with or without that vma_is_fsdax() check. > > Because: > > > > get_vaddr_frames(FOLL_LONGTERM) > > pin_user_pages_locked() > > if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) > > return -EINVAL; > > There is no particular reason this code needs to have the mm sem at > that point. > > It should call pin_user_pages_fast() and only if that fails get the mmap > lock and extract the VMA to do broken hackery. Yeah I think that works. I tried understanding gup.c code a bit more, and it looks like FOLL_LONGTERM only works for the pup_fast variant right now? All others seem to have this comment that it's somehow incompatible with FAULT_FLAG_ALLOW_RETRY and daxfs. But grepping around for that didn't show up anything, at least not nearby dax code. For my understanding of all this, what's the hiccup there? For plans, I only started this for a bit of my own learning, but I think I'll respin with the following changes: - convert exynos and habanalabs to pin_user_pages_fast directly, instead of going through this frame-vector detour - move the locking and convert get_vaddr_frames to pup_fast as Jason suggested - hack up some truly gross rfc to plug the follow_pfn hole Cheers, Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch