From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD310C433B4 for ; Tue, 18 May 2021 21:16:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 86EC761040 for ; Tue, 18 May 2021 21:16:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352430AbhERVSF (ORCPT ); Tue, 18 May 2021 17:18:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48174 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235250AbhERVSD (ORCPT ); Tue, 18 May 2021 17:18:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621372604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=VnmNKuXlbyVyxs8/KgMR3FJIb6fYWM2pKd6bjxL/tErwYwsLaTCuDyRMPc3OMZBJj0Gkwh EMtoCIfiPeDfNK6YM8GqGkrYlp+8OukqIInPmmDybzvuudVaIf+gtZDqLxRsB4WsU8CQPX iFD6UE3q+QTqjn2U02iM8uO+hsxXmoY= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-354-iQBpKZNfOxiUEgP-pYRMgg-1; Tue, 18 May 2021 17:16:42 -0400 X-MC-Unique: iQBpKZNfOxiUEgP-pYRMgg-1 Received: by mail-qt1-f198.google.com with SMTP id b17-20020ac854110000b02901f279c73d75so4460680qtq.2 for ; Tue, 18 May 2021 14:16:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=uZD7nI/zw+Jz2/z80xbu88iJhd16TyILz1Vt55toe2sdPWWU6o6Yh8iR7DrDrMzUR+ O3595b8aNRWn3zDcPvsmKIdcKLLNUcmbaax+QyBO0g6N4WHhlLpb40jIuGXcwDQ9uKgr fdgN2Zt099KCMqaN/RGgzGoDXUl4JGcsORvX48uBpV6WEp792XWIQN7ckFtQOgbXx3jE zoEicmDmyd67vrcst8UH79oc3mtfbf/FadghvgR10hQkK1sxYKYHwIEYeGNsDkhJZqWh +WC6sZjZpAapexxoOdgSLTSFYHvV4KTKSKWACYhbdnibBAkPZ6fYDSNuNms3B7iOjM40 B7TQ== X-Gm-Message-State: AOAM533Cn3Phh/r06ETtFib67VCPK+ZZ1Rq3JnEd0oX/IE89CdGYxgG3 mqO44hG8oUF82LNokVBl8A1ONsugiLCYkQ6MYn9MivRcHr/I+XVhSjOTwXWzMlWd/P7G0PqYKC4 oLGF++hT41qUETFLCyh+SRCX9 X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111156qta.172.1621372601481; Tue, 18 May 2021 14:16:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7CflObEdpN93nC2c9rCnQMv5qBohaNF3R0O5eiJLrnt/E6MHpUTkD6EfDzN6tiXMS/VwRag== X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111124qta.172.1621372601142; Tue, 18 May 2021 14:16:41 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id v66sm13507150qkd.113.2021.05.18.14.16.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:16:40 -0700 (PDT) Date: Tue, 18 May 2021 17:16:38 -0400 From: Peter Xu To: Alistair Popple Cc: linux-mm@kvack.org, nouveau@lists.freedesktop.org, bskeggs@redhat.com, akpm@linux-foundation.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, jhubbard@nvidia.com, rcampbell@nvidia.com, jglisse@redhat.com, jgg@nvidia.com, hch@infradead.org, daniel@ffwll.ch, willy@infradead.org, bsingharora@gmail.com, Christoph Hellwig Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20210407084238.20443-6-apopple@nvidia.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 07, 2021 at 06:42:35PM +1000, Alistair Popple wrote: [...] > +static bool try_to_protect(struct page *page, struct mm_struct *mm, > + unsigned long address, void *arg) > +{ > + struct ttp_args ttp = { > + .mm = mm, > + .address = address, > + .arg = arg, > + .valid = false, > + }; > + struct rmap_walk_control rwc = { > + .rmap_one = try_to_protect_one, > + .done = page_not_mapped, > + .anon_lock = page_lock_anon_vma_read, > + .arg = &ttp, > + }; > + > + /* > + * Restrict to anonymous pages for now to avoid potential writeback > + * issues. > + */ > + if (!PageAnon(page)) > + return false; > + > + /* > + * During exec, a temporary VMA is setup and later moved. > + * The VMA is moved under the anon_vma lock but not the > + * page tables leading to a race where migration cannot > + * find the migration ptes. Rather than increasing the > + * locking requirements of exec(), migration skips > + * temporary VMAs until after exec() completes. > + */ > + if (!PageKsm(page) && PageAnon(page)) > + rwc.invalid_vma = invalid_migration_vma; > + > + rmap_walk(page, &rwc); > + > + return ttp.valid && !page_mapcount(page); > +} I raised a question in the other thread regarding fork(): https://lore.kernel.org/lkml/YKQjmtMo+YQGx%2FwZ@t490s/ While I suddenly noticed that we may have similar issues even if we fork() before creating the ptes. In that case, we may see multiple read-only ptes pointing to the same page. We will convert all of them into device exclusive read ptes in rmap_walk() above, however how do we guarantee after all COW done in the parent and all the childs processes, the device owned page will be returned to the parent? E.g., if parent accesses the page earlier than the children processes (actually, as long as not the last one), do_wp_page() will do COW for parent on this page because refcount(page)>1, then the page seems to get lost to a random child too.. To resolve all these complexity, not sure whether try_to_protect() could enforce VM_DONTCOPY (needs madvise MADV_DONTFORK in the user app), meanwhile make sure mapcount(page)==1 before granting the page to the device, so that this will guarantee this mm owns this page forever, I think? It'll simplify fork() too as a side effect, since VM_DONTCOPY vma go away when fork. -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B033FC433ED for ; Wed, 19 May 2021 16:29:18 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 787E161007 for ; Wed, 19 May 2021 16:29:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 787E161007 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=nouveau-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A441C6EE1E; Wed, 19 May 2021 16:29:11 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id 969226ECA9 for ; Tue, 18 May 2021 21:16:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621372603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=S7oIJY1pBUY+y3xOY9sHqrU9CfblvFEvJZJ3pV8M0MfBEz4sV2n52FW0dsscQbRePEPone f/FXagl3vNCkFUasOmL3/j4Tg2cil5PFtD537sMB+9DKgfl46Sl1g7QTwY5gFnUr6rPC2c q9QEtwR6Y8kz+cFxqUOsynomcOSJi6k= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-456-JldQ032nP9i0CPDVA2_Dzg-1; Tue, 18 May 2021 17:16:42 -0400 X-MC-Unique: JldQ032nP9i0CPDVA2_Dzg-1 Received: by mail-qt1-f197.google.com with SMTP id 1-20020aed31010000b029019d1c685840so8365109qtg.3 for ; Tue, 18 May 2021 14:16:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=UjwdeVPCb20CrdAdK7lFgPRGDNpDPgWGJvJUXWs4WMTjqo2GF8pcXtcRVvAcSF4nme 3l8Nm0WAmDiZdhRPE2j5gsK+J4xcZCfYIyA9hKX0aDIQQrPM2TA76UU0Lyvv1Csi1009 gNHWOpA1ZveD8HZkg625JWZeTD1s1H/Y+JMSeXBYSaAs8byBrikdAgef6NzbDrKkfRqx bt6DSZnaEii2zuo6M6G5A/QoKwAua+WYdpVopAFuoblZVdvNgsrexpqsZDMgxkI6dUny aN7/UOmy4ANltv2DVqfIHNetoG+1zcr2ln1llrRaXSDQv6VLxV2+uiEQRjbQvpggjgyV R1RA== X-Gm-Message-State: AOAM530Cb4tgooAWTnKFp7tjVJpGCjNKrt8I5+eGapjERT3VSy0XSFeR FJAH66yUzVEPdQ6vuKmGJaNSEf5x+yvWQ2fD3sXc8njENYiB1sPvLIQpyVHwSNyX5quRfsY4mbu hdrRDyOznnQw2ix6flNQ7BU5x5A== X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111157qta.172.1621372601481; Tue, 18 May 2021 14:16:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7CflObEdpN93nC2c9rCnQMv5qBohaNF3R0O5eiJLrnt/E6MHpUTkD6EfDzN6tiXMS/VwRag== X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111124qta.172.1621372601142; Tue, 18 May 2021 14:16:41 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id v66sm13507150qkd.113.2021.05.18.14.16.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:16:40 -0700 (PDT) Date: Tue, 18 May 2021 17:16:38 -0400 From: Peter Xu To: Alistair Popple Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20210407084238.20443-6-apopple@nvidia.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Mailman-Approved-At: Wed, 19 May 2021 16:29:11 +0000 Subject: Re: [Nouveau] [PATCH v8 5/8] mm: Device exclusive memory access X-BeenThere: nouveau@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Nouveau development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rcampbell@nvidia.com, willy@infradead.org, daniel@ffwll.ch, linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org, bsingharora@gmail.com, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, hch@infradead.org, linux-mm@kvack.org, bskeggs@redhat.com, jgg@nvidia.com, akpm@linux-foundation.org, Christoph Hellwig Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: nouveau-bounces@lists.freedesktop.org Sender: "Nouveau" On Wed, Apr 07, 2021 at 06:42:35PM +1000, Alistair Popple wrote: [...] > +static bool try_to_protect(struct page *page, struct mm_struct *mm, > + unsigned long address, void *arg) > +{ > + struct ttp_args ttp = { > + .mm = mm, > + .address = address, > + .arg = arg, > + .valid = false, > + }; > + struct rmap_walk_control rwc = { > + .rmap_one = try_to_protect_one, > + .done = page_not_mapped, > + .anon_lock = page_lock_anon_vma_read, > + .arg = &ttp, > + }; > + > + /* > + * Restrict to anonymous pages for now to avoid potential writeback > + * issues. > + */ > + if (!PageAnon(page)) > + return false; > + > + /* > + * During exec, a temporary VMA is setup and later moved. > + * The VMA is moved under the anon_vma lock but not the > + * page tables leading to a race where migration cannot > + * find the migration ptes. Rather than increasing the > + * locking requirements of exec(), migration skips > + * temporary VMAs until after exec() completes. > + */ > + if (!PageKsm(page) && PageAnon(page)) > + rwc.invalid_vma = invalid_migration_vma; > + > + rmap_walk(page, &rwc); > + > + return ttp.valid && !page_mapcount(page); > +} I raised a question in the other thread regarding fork(): https://lore.kernel.org/lkml/YKQjmtMo+YQGx%2FwZ@t490s/ While I suddenly noticed that we may have similar issues even if we fork() before creating the ptes. In that case, we may see multiple read-only ptes pointing to the same page. We will convert all of them into device exclusive read ptes in rmap_walk() above, however how do we guarantee after all COW done in the parent and all the childs processes, the device owned page will be returned to the parent? E.g., if parent accesses the page earlier than the children processes (actually, as long as not the last one), do_wp_page() will do COW for parent on this page because refcount(page)>1, then the page seems to get lost to a random child too.. To resolve all these complexity, not sure whether try_to_protect() could enforce VM_DONTCOPY (needs madvise MADV_DONTFORK in the user app), meanwhile make sure mapcount(page)==1 before granting the page to the device, so that this will guarantee this mm owns this page forever, I think? It'll simplify fork() too as a side effect, since VM_DONTCOPY vma go away when fork. -- Peter Xu _______________________________________________ Nouveau mailing list Nouveau@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/nouveau From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74B82C43460 for ; Tue, 18 May 2021 21:16:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B0F861040 for ; Tue, 18 May 2021 21:16:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B0F861040 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 89A886ECAB; Tue, 18 May 2021 21:16:46 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by gabe.freedesktop.org (Postfix) with ESMTPS id CA4636ECAB for ; Tue, 18 May 2021 21:16:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621372603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=S7oIJY1pBUY+y3xOY9sHqrU9CfblvFEvJZJ3pV8M0MfBEz4sV2n52FW0dsscQbRePEPone f/FXagl3vNCkFUasOmL3/j4Tg2cil5PFtD537sMB+9DKgfl46Sl1g7QTwY5gFnUr6rPC2c q9QEtwR6Y8kz+cFxqUOsynomcOSJi6k= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-105--bMkAnWuOtGA46zeuGf5aA-1; Tue, 18 May 2021 17:16:42 -0400 X-MC-Unique: -bMkAnWuOtGA46zeuGf5aA-1 Received: by mail-qt1-f198.google.com with SMTP id b20-20020ac87fd40000b02901e1370c5e12so8297495qtk.17 for ; Tue, 18 May 2021 14:16:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=FHlAjMq4fd/2+3wzJ1Ve/DqUSuzKthBeN06aDjaf5eZCxAWBayC3H7IVHQVQgTaoxM ZVYt72B4kBZ4otihLZS6W4jIdfTDLRZkK/zTm5HDu1weGzZdt5epG11W2+j2bsCVy7qr wkG0B+Y3aluy40nnHjzo2CcpWhkNk0BuGmYnBQVdPP7kwecOtxRkZ6k8zsnRYJibh4pw E+udVHYDin+X0C9gJBQMuvZI5R7XYdaC5ISWWqAYkag3BT7mizzWpKfy0QFORSI4+p3X 2SRkckIPk8NKSRB1J31twwBypgbP4kh2qr8fA4Rho2543ENBsDi6Ihb5j+m1yQ2hhHD1 pfLw== X-Gm-Message-State: AOAM533jKYnLQe4cLuBapmOopekoGgBLxUdX8V5T2Pt/YMGPNCY1eCTv 9R9M76zXITo7sAShUrvHCFquvpKKN0z+JfeVPPPUVSnslN51iLnLt7mTwOVN8TZWCQ/mHiEOXFA 6jKVrxB/PM6hfyikYjeuaIGCiF5kx X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111159qta.172.1621372601481; Tue, 18 May 2021 14:16:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7CflObEdpN93nC2c9rCnQMv5qBohaNF3R0O5eiJLrnt/E6MHpUTkD6EfDzN6tiXMS/VwRag== X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111124qta.172.1621372601142; Tue, 18 May 2021 14:16:41 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id v66sm13507150qkd.113.2021.05.18.14.16.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:16:40 -0700 (PDT) Date: Tue, 18 May 2021 17:16:38 -0400 From: Peter Xu To: Alistair Popple Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20210407084238.20443-6-apopple@nvidia.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rcampbell@nvidia.com, willy@infradead.org, linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org, bsingharora@gmail.com, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, hch@infradead.org, linux-mm@kvack.org, jglisse@redhat.com, bskeggs@redhat.com, jgg@nvidia.com, jhubbard@nvidia.com, akpm@linux-foundation.org, Christoph Hellwig Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, Apr 07, 2021 at 06:42:35PM +1000, Alistair Popple wrote: [...] > +static bool try_to_protect(struct page *page, struct mm_struct *mm, > + unsigned long address, void *arg) > +{ > + struct ttp_args ttp = { > + .mm = mm, > + .address = address, > + .arg = arg, > + .valid = false, > + }; > + struct rmap_walk_control rwc = { > + .rmap_one = try_to_protect_one, > + .done = page_not_mapped, > + .anon_lock = page_lock_anon_vma_read, > + .arg = &ttp, > + }; > + > + /* > + * Restrict to anonymous pages for now to avoid potential writeback > + * issues. > + */ > + if (!PageAnon(page)) > + return false; > + > + /* > + * During exec, a temporary VMA is setup and later moved. > + * The VMA is moved under the anon_vma lock but not the > + * page tables leading to a race where migration cannot > + * find the migration ptes. Rather than increasing the > + * locking requirements of exec(), migration skips > + * temporary VMAs until after exec() completes. > + */ > + if (!PageKsm(page) && PageAnon(page)) > + rwc.invalid_vma = invalid_migration_vma; > + > + rmap_walk(page, &rwc); > + > + return ttp.valid && !page_mapcount(page); > +} I raised a question in the other thread regarding fork(): https://lore.kernel.org/lkml/YKQjmtMo+YQGx%2FwZ@t490s/ While I suddenly noticed that we may have similar issues even if we fork() before creating the ptes. In that case, we may see multiple read-only ptes pointing to the same page. We will convert all of them into device exclusive read ptes in rmap_walk() above, however how do we guarantee after all COW done in the parent and all the childs processes, the device owned page will be returned to the parent? E.g., if parent accesses the page earlier than the children processes (actually, as long as not the last one), do_wp_page() will do COW for parent on this page because refcount(page)>1, then the page seems to get lost to a random child too.. To resolve all these complexity, not sure whether try_to_protect() could enforce VM_DONTCOPY (needs madvise MADV_DONTFORK in the user app), meanwhile make sure mapcount(page)==1 before granting the page to the device, so that this will guarantee this mm owns this page forever, I think? It'll simplify fork() too as a side effect, since VM_DONTCOPY vma go away when fork. -- Peter Xu