From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA673C433B4 for ; Tue, 18 May 2021 18:01:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC7A9611BF for ; Tue, 18 May 2021 18:01:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351415AbhERSDB (ORCPT ); Tue, 18 May 2021 14:03:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52484 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344188AbhERSC7 (ORCPT ); Tue, 18 May 2021 14:02:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621360900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=IDwGlXLr1PcmK2OfwCyfdYEBBpzgJOsiKuEmyVxaqjfIfHY2MkQlUvjwaDu7QQazRbmaaK zvTp6oimEI/VPZhXN2pvQfl9DhDLT8XWfw9P7igvy8xreA4vPZgGt/RjNos13fd/bq1dI5 YUzLGRrQuMX4koV2pUkeWcQ0t9Uw3sc= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-533-9dtl-DaMPjiw5p7i7wfXGA-1; Tue, 18 May 2021 14:01:39 -0400 X-MC-Unique: 9dtl-DaMPjiw5p7i7wfXGA-1 Received: by mail-qk1-f197.google.com with SMTP id v1-20020a05620a1221b02902ea88445e01so7755359qkj.9 for ; Tue, 18 May 2021 11:01:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=ZpMuWrMQyLFkOEL8Wh0HRKcLmcD8/3Z3E786G8clVqzqyqlvDiNgwUbVFljpuAK9x3 5HPxbRLwlzs79qKP2NUkMbM0Bb9RRoNRLuUnPXaYTbrnUAv0PM+0xR0GaraeLaZsDObJ kOXSPApVJTQ9yMeiG4RKr2z8VzlvTcumX6vPB1Of/8yrWa1FZHKkxEjhtgDLsyaY8W6U h4JQogo8OfQ/V2pTwolYSwaxWLnPQhYX1E5SH20hokWm885OXuXLOaw/4tSLm+SmXEOo 6IR5HlmFVrQRzUXkPDAO9HUGdILkznleNXRzYSge7m6SwPEEw/UybxOtm5tWho2q8Xg5 vgVA== X-Gm-Message-State: AOAM531emcWLQTqpAZhh2Fx0jdfI6PsIrIjbpMUmVySdCaTcQS7dMthn uA1JAFHggh/aNZVonX0IOGzfeqCDbVEME4MCSU460RlC+Z52BXuS3smU3WrZprZ2JPC4JNcDhx5 FFc4r3fqd3PSKqlVPQU5jCCNN X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060393qti.157.1621360898457; Tue, 18 May 2021 11:01:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxTICgj8/egwsE0Ivgqvw8kGv1T8HxyWhbScHPLgGtw0FYHBEe8vymzLu/hxY0OnGRaALaW5w== X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060354qti.157.1621360898147; Tue, 18 May 2021 11:01:38 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id b23sm1488671qtp.7.2021.05.18.11.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 11:01:37 -0700 (PDT) Date: Tue, 18 May 2021 14:01:36 -0400 From: Peter Xu To: Jason Gunthorpe Cc: Alistair Popple , linux-mm@kvack.org, nouveau@lists.freedesktop.org, bskeggs@redhat.com, akpm@linux-foundation.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, jhubbard@nvidia.com, rcampbell@nvidia.com, jglisse@redhat.com, hch@infradead.org, daniel@ffwll.ch, willy@infradead.org, bsingharora@gmail.com, Christoph Hellwig Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> <47694715.suB6H4Uo8R@nvdebian> <20210518173334.GE1002214@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20210518173334.GE1002214@nvidia.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 18, 2021 at 02:33:34PM -0300, Jason Gunthorpe wrote: > On Tue, May 18, 2021 at 01:27:42PM -0400, Peter Xu wrote: > > > I also have a pure and high level question regarding a process fork() when > > there're device exclusive ptes: would the two processes then own the device > > together? Is this a real usecase? > > If the pages are MAP_SHARED then yes. All VMAs should point at the > same device_exclusive page and all VMA should migrate back to CPU > pages together. Makes sense. If we keep the anonymous-only in this series (I think it's good to separate these), maybe we can drop the !COW case, plus some proper installed WARN_ON_ONCE()s. > > > Indeed it'll be odd for a COW page since for COW page then it means after > > parent/child writting to the page it'll clone into two, then it's a mistery on > > which one will be the one that "exclusived owned" by the device.. > > For COW pages it is like every other fork case.. We can't reliably > write-protect the device_exclusive page during fork so we must copy it > at fork time. > > Thus three reasonable choices: > - Copy to a new CPU page > - Migrate back to a CPU page and write protect it > - Copy to a new device exclusive page IMHO the ownership question would really help us to answer this one.. If the device ownership should be kept in parent IMHO option (1) is the best approach. To be explicit on page copy: we can do best-effort, even if the copy is during a device atomic operation, perhaps? If the ownership will be shared, seems option (3) will be easier as I don't see a strong reason to do immediate restorinng of ptes; as long as we're careful on the refcounting. Thanks, -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4B64C433ED for ; Tue, 18 May 2021 20:51:40 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 66F9F61028 for ; Tue, 18 May 2021 20:51:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 66F9F61028 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=nouveau-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3C6D96ECA2; Tue, 18 May 2021 20:51:36 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9ADD36EC5B for ; Tue, 18 May 2021 18:01:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621360902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=hdmPNN+69WNODzQFE+Gsjohg3krgpFRqEYWLUhhFFaIAV+RO7G2Zzr2SCfQnG7fKzOiV+R bBojKJ+7ghicw9Gc0vZ7cWJB4NjcSNhdDhyhPtuRGP9SPRkJMpVgzyd7WNY9uG5vl97xP1 +aYr86n2WmBCpsx84JESVDSvQCtnC08= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-330-RHbK_Z5cM0Cq9YffTP4I-Q-1; Tue, 18 May 2021 14:01:39 -0400 X-MC-Unique: RHbK_Z5cM0Cq9YffTP4I-Q-1 Received: by mail-qk1-f200.google.com with SMTP id s68-20020a372c470000b0290305f75a7cecso7764941qkh.8 for ; Tue, 18 May 2021 11:01:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=DB8wj2LTyDE8VsIT2C6Cotssuew8G7cxN0FmO8urk9JOu9Td6N6I95hTEAMm8Lkt7p JuwEFOqdFdsS5qjdnqVNMHF9/j3vPmmvauNnQSaYhKbhmqOCFIL5wYeUxOfKd1lMJKJR aZxJxGM8rp3u/dVxLKwJvV4wW1tHhA9U8JKnhF9ieK+5dKzSKjytNhQmiVRdPKgSCT3/ BuL6GB/alLv40/5Nba+OXiYzqN4jZElVMTDHGNbytG760p9m4Khz/Pl+xGZiB3eaTexv 5NADWaj7tkIF730rs+II/FIeMkJfDdFwg81UPT236uVNAjCi3XegotULOncGNVdGPt9x b0GQ== X-Gm-Message-State: AOAM5338kwxQYFjnp+f/Vv4Z/aefvX28ijMjP3aZ/ngPVJpf0W4IEEvq yF+VFFWoHCHSCDatMU+8v0quAuMA+67O2Ca++7khgllsrkOp+mtj0hWw0Vs4w168QpznJ4k4zxu 7HE554o1lVVa1L62matwJypMbyA== X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060390qti.157.1621360898457; Tue, 18 May 2021 11:01:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxTICgj8/egwsE0Ivgqvw8kGv1T8HxyWhbScHPLgGtw0FYHBEe8vymzLu/hxY0OnGRaALaW5w== X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060354qti.157.1621360898147; Tue, 18 May 2021 11:01:38 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id b23sm1488671qtp.7.2021.05.18.11.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 11:01:37 -0700 (PDT) Date: Tue, 18 May 2021 14:01:36 -0400 From: Peter Xu To: Jason Gunthorpe Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> <47694715.suB6H4Uo8R@nvdebian> <20210518173334.GE1002214@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20210518173334.GE1002214@nvidia.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Mailman-Approved-At: Tue, 18 May 2021 20:51:34 +0000 Subject: Re: [Nouveau] [PATCH v8 5/8] mm: Device exclusive memory access X-BeenThere: nouveau@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Nouveau development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rcampbell@nvidia.com, willy@infradead.org, linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org, bsingharora@gmail.com, Alistair Popple , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, hch@infradead.org, linux-mm@kvack.org, bskeggs@redhat.com, daniel@ffwll.ch, akpm@linux-foundation.org, Christoph Hellwig Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: nouveau-bounces@lists.freedesktop.org Sender: "Nouveau" On Tue, May 18, 2021 at 02:33:34PM -0300, Jason Gunthorpe wrote: > On Tue, May 18, 2021 at 01:27:42PM -0400, Peter Xu wrote: > > > I also have a pure and high level question regarding a process fork() when > > there're device exclusive ptes: would the two processes then own the device > > together? Is this a real usecase? > > If the pages are MAP_SHARED then yes. All VMAs should point at the > same device_exclusive page and all VMA should migrate back to CPU > pages together. Makes sense. If we keep the anonymous-only in this series (I think it's good to separate these), maybe we can drop the !COW case, plus some proper installed WARN_ON_ONCE()s. > > > Indeed it'll be odd for a COW page since for COW page then it means after > > parent/child writting to the page it'll clone into two, then it's a mistery on > > which one will be the one that "exclusived owned" by the device.. > > For COW pages it is like every other fork case.. We can't reliably > write-protect the device_exclusive page during fork so we must copy it > at fork time. > > Thus three reasonable choices: > - Copy to a new CPU page > - Migrate back to a CPU page and write protect it > - Copy to a new device exclusive page IMHO the ownership question would really help us to answer this one.. If the device ownership should be kept in parent IMHO option (1) is the best approach. To be explicit on page copy: we can do best-effort, even if the copy is during a device atomic operation, perhaps? If the ownership will be shared, seems option (3) will be easier as I don't see a strong reason to do immediate restorinng of ptes; as long as we're careful on the refcounting. Thanks, -- Peter Xu _______________________________________________ Nouveau mailing list Nouveau@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/nouveau From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8953C43461 for ; Tue, 18 May 2021 18:01:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 985D761073 for ; Tue, 18 May 2021 18:01:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 985D761073 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F26396EC5B; Tue, 18 May 2021 18:01:43 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6A7DA6EC5B for ; Tue, 18 May 2021 18:01:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621360901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=Ora0ec0MtMLAuvk+SbeTaOUni3CLwx59zVFX5AqLzUlof5EBxe0IA8CZNvO17Q+uDbisd0 FgtwJ9J3KXftksjHr0Rdrigywaka5q7oWMySvpVP28sE+X2eFqtF78oLGJQuZcJQ7DrQTk M8APNkPboCNwMbRfLfAbVDrK6NqsCJk= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-248-Aw3gbewvNp2lc26nYGSNQA-1; Tue, 18 May 2021 14:01:39 -0400 X-MC-Unique: Aw3gbewvNp2lc26nYGSNQA-1 Received: by mail-qv1-f72.google.com with SMTP id x2-20020a0cda020000b02901edb4c412fdso7987905qvj.11 for ; Tue, 18 May 2021 11:01:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Oue7U1wjftn7xVwPmUZCJccBeCJY9jE5A9d8yc2+hO4=; b=GbahVdl+3gSZD3fgR/4YbgnbIjv57Azm2Au43tMUXJRNkM9caOhiU3qUAzZXUzao/z 539Ri+mAnyN4qLGGoPmuK8c1mVCUWL9xEYY50AvmKAewwggtvvOlmfMQ7Ia5weKOwbv+ 7TilgOwJi0hsJPTv5RB8epiMPivYDAc6LlzKVTb7i61O1NUEbvPoQFU9TMukmVgalKmC FhFyI94AOvYaBAwDwuEs7GdCMNzmxVHNxjr3g79NsxXPxj9BVO1Zhq659MQyxcX19sdc CVqcyTnh+Qt7z/5/n0FVfZs7W3oDYBxAaoYzHBkPjv7SkN7/NXbCtfajH2NbBf93APxO g7KA== X-Gm-Message-State: AOAM531u0t+Qfz2NQtRad8nKN6RqdfEMnxNliln2G0TvZTZynJSanMnB a4YvC6gDz3JaZ+xfr8ldXltzQPf/gh0T75bgBF//q9pK/R3MYpdKpoWKIBmMkTGpLHJbxdWkD7i oM4YIi2Qg3S8MYT7FtwEWF7fCt3KF X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060397qti.157.1621360898458; Tue, 18 May 2021 11:01:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxTICgj8/egwsE0Ivgqvw8kGv1T8HxyWhbScHPLgGtw0FYHBEe8vymzLu/hxY0OnGRaALaW5w== X-Received: by 2002:ac8:a49:: with SMTP id f9mr6060354qti.157.1621360898147; Tue, 18 May 2021 11:01:38 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id b23sm1488671qtp.7.2021.05.18.11.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 11:01:37 -0700 (PDT) Date: Tue, 18 May 2021 14:01:36 -0400 From: Peter Xu To: Jason Gunthorpe Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> <47694715.suB6H4Uo8R@nvdebian> <20210518173334.GE1002214@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20210518173334.GE1002214@nvidia.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rcampbell@nvidia.com, willy@infradead.org, linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org, bsingharora@gmail.com, Alistair Popple , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, hch@infradead.org, linux-mm@kvack.org, jglisse@redhat.com, bskeggs@redhat.com, jhubbard@nvidia.com, akpm@linux-foundation.org, Christoph Hellwig Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Tue, May 18, 2021 at 02:33:34PM -0300, Jason Gunthorpe wrote: > On Tue, May 18, 2021 at 01:27:42PM -0400, Peter Xu wrote: > > > I also have a pure and high level question regarding a process fork() when > > there're device exclusive ptes: would the two processes then own the device > > together? Is this a real usecase? > > If the pages are MAP_SHARED then yes. All VMAs should point at the > same device_exclusive page and all VMA should migrate back to CPU > pages together. Makes sense. If we keep the anonymous-only in this series (I think it's good to separate these), maybe we can drop the !COW case, plus some proper installed WARN_ON_ONCE()s. > > > Indeed it'll be odd for a COW page since for COW page then it means after > > parent/child writting to the page it'll clone into two, then it's a mistery on > > which one will be the one that "exclusived owned" by the device.. > > For COW pages it is like every other fork case.. We can't reliably > write-protect the device_exclusive page during fork so we must copy it > at fork time. > > Thus three reasonable choices: > - Copy to a new CPU page > - Migrate back to a CPU page and write protect it > - Copy to a new device exclusive page IMHO the ownership question would really help us to answer this one.. If the device ownership should be kept in parent IMHO option (1) is the best approach. To be explicit on page copy: we can do best-effort, even if the copy is during a device atomic operation, perhaps? If the ownership will be shared, seems option (3) will be easier as I don't see a strong reason to do immediate restorinng of ptes; as long as we're careful on the refcounting. Thanks, -- Peter Xu