From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 097B1C8300A for ; Wed, 29 Apr 2020 00:29:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DAC9F20737 for ; Wed, 29 Apr 2020 00:29:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="gBF0juzP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726457AbgD2A3I (ORCPT ); Tue, 28 Apr 2020 20:29:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726348AbgD2A3H (ORCPT ); Tue, 28 Apr 2020 20:29:07 -0400 Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6973C03C1AD for ; Tue, 28 Apr 2020 17:29:05 -0700 (PDT) Received: by mail-qv1-xf41.google.com with SMTP id 59so305246qva.13 for ; Tue, 28 Apr 2020 17:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=sum6zvnI9uuuHONOiogQ9HloeE5QbhDG16IxplXpyKw=; b=gBF0juzP6lNNN9CsauhPBhO02OANPJ2mLHyQLraLJuxmTXzYX3a/IrHNLRunWWLgiG YQVRsmq/CQUxnSlv750R4HNOe+f4kPwaWyk0MtrKXzmwYnqUFpOJ5CXJFPShUc+SQm/P 4qBERO3b+TESmrHgaop2a4RXN2SQvRu7y9GSisP3W+GLS3SS3+L73GmaaALGOymT7K0K nIIR4A0TZGwRH9dThz9J/IhinJXc+OLdMJaRrWSTTSghWuweWS/w5EJSyzxLOzHuLQl0 5dL0b3ddwCB3mIe+/kUSVtBffiLucojHhM3/PGLUAMEKHbSw9EZAapDnMT57NkF8ZjD5 LVHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=sum6zvnI9uuuHONOiogQ9HloeE5QbhDG16IxplXpyKw=; b=m+UDHoxWycXZjYBKqVJXESQGY8tN1hPqEvjvcYdePjXrYcjb/yJhw8DNXFKZmNTkJt QfHj2pH8QLwGEahV45Jlp2UrXghTZqnhtTHdgLYfkJq/QhQLzJyqsHcaTcI2lAfUjAms cCEFPrQ1sZ2VuyS07i5MkezWojpNk1DdU+tYAw7n7/6Do/RqPLUCWaZinp6OcOWfQsi7 5CEbOKkgdBYbveGZJJPzTuFBFEAvTHXg1I0k0Vgj6p3uutnfYMvZ+fRrV2P1ZyJ14I/O uYjXmA3N6I4E2XSPzb/gswakNJXtpXS9oCRWOaT4QIy1KeT9thuwpi2iHQJRIH4JMGQk iobQ== X-Gm-Message-State: AGi0PuYW9JKPIRnE96BrEahKKS3xXe2/2WNjvdJ17VzSFmPD35q4qac2 XBY0kOrdrC7LpIwCFxIdlYd1bNbprhFd4g== X-Google-Smtp-Source: APiQypKpF7REJXdifsNrbjdxjECMvWqO7VD/BFBVFuZ0JXzB0iNDmPtTStSILhzusynqtgGAwoi6Fw== X-Received: by 2002:a0c:b651:: with SMTP id q17mr2047236qvf.135.1588120144906; Tue, 28 Apr 2020 17:29:04 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id 134sm14193711qki.16.2020.04.28.17.29.03 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 28 Apr 2020 17:29:04 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jTab1-0001tN-Gx; Tue, 28 Apr 2020 21:29:03 -0300 Date: Tue, 28 Apr 2020 21:29:03 -0300 From: Jason Gunthorpe To: Alex Williamson Cc: linux-doc@vger.kernel.org, John Hubbard , LKML , Andrew Morton , Al Viro , Christoph Hellwig , Dan Williams , Dave Chinner , Ira Weiny , Jan Kara , Jonathan Corbet , =?utf-8?B?SsOpcsO0bWU=?= Glisse , "Kirill A . Shutemov" , Michal Hocko , Mike Kravetz , Shuah Khan , Vlastimil Babka , Matthew Wilcox , linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, "Kirill A . Shutemov" Subject: Re: [regression?] Re: [PATCH v6 06/12] mm/gup: track FOLL_PIN pages Message-ID: <20200429002903.GZ26002@ziepe.ca> References: <20200211001536.1027652-7-jhubbard@nvidia.com> <20200424121846.5ee2685f@w520.home> <5b901542-d949-8d7e-89c7-f8d5ee20f6e9@nvidia.com> <20200424141548.5afdd2bb@w520.home> <665ffb48-d498-90f4-f945-997a922fc370@nvidia.com> <20200428105455.30343fb4@w520.home> <20200428174957.GV26002@ziepe.ca> <20200428130752.75c153bd@w520.home> <20200428192251.GW26002@ziepe.ca> <20200428141223.5b1653db@w520.home> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200428141223.5b1653db@w520.home> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-doc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On Tue, Apr 28, 2020 at 02:12:23PM -0600, Alex Williamson wrote: > > > Maybe I was just getting lucky before this commit. For a > > > VM_PFNMAP, vaddr_get_pfn() only needs pin_user_pages_remote() to return > > > error and the vma information that we setup in vfio_pci_mmap(). > > > > I've written on this before, vfio should not be passing pages to the > > iommu that it cannot pin eg it should not touch VM_PFNMAP vma's in the > > first place. > > > > It is a use-after-free security issue the way it is.. > > Where is the user after free? Here I'm trying to map device mmio space > through the iommu, which we need to enable p2p when the user owns > multiple devices. Yes, I gathered what the intent was.. > The device is owned by the user, bound to vfio-pci, and can't be > unbound while the user has it open. The iommu mappings are torn > down on release. I guess I don't understand the problem. For PFNMAP VMAs the lifecycle rule is basically that the PFN inside the VMA can only be used inside the mmap_sem that read it. Ie you cannot take a PFN outside the mmap_sem and continue to use it. This is because the owner of the VMA owns the lifetime of that PFN, and under the write side of the mmap_sem it can zap the PFN, or close the VMA. Afterwards the VMA owner knows that there are no active reference to the PFN in the system and can reclaim the PFN ie the PFNMAP has no per-page pin counter. All lifetime revolves around the mmap_sem and the vma. What vfio does is take the PFN out of the mmap_sem and program it into the iommu. So when the VMA owner decides the PFN has no references, it actually doesn't: vfio continues to access it beyond its permitted lifetime. HW like mlx5 and GPUs have BAR pages which have security properties. Once the PFN is returned to the driver the security context of the PFN can be reset and re-assigned to another process. Using VFIO a hostile user space can retain access to the BAR page and upon its reassignment access a security context they were not permitted to access. This is why GUP does not return PFNMAP pages and vfio should not carry a reference outside the mmap_sem. It breaks all the lifetime rules. Jason