From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A212C433B4 for ; Tue, 18 May 2021 21:16:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F1300611BD for ; Tue, 18 May 2021 21:16:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1300611BD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 714B98E0057; Tue, 18 May 2021 17:16:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C4228E002F; Tue, 18 May 2021 17:16:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53D4F8E0057; Tue, 18 May 2021 17:16:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0065.hostedemail.com [216.40.44.65]) by kanga.kvack.org (Postfix) with ESMTP id 25ABD8E002F for ; Tue, 18 May 2021 17:16:45 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A2F908249980 for ; Tue, 18 May 2021 21:16:44 +0000 (UTC) X-FDA: 78155611128.05.386A3DD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf13.hostedemail.com (Postfix) with ESMTP id A0D32E0001AF for ; Tue, 18 May 2021 21:16:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621372603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=S7oIJY1pBUY+y3xOY9sHqrU9CfblvFEvJZJ3pV8M0MfBEz4sV2n52FW0dsscQbRePEPone f/FXagl3vNCkFUasOmL3/j4Tg2cil5PFtD537sMB+9DKgfl46Sl1g7QTwY5gFnUr6rPC2c q9QEtwR6Y8kz+cFxqUOsynomcOSJi6k= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-19-0yCA03L2M3Cybqtns9xLxA-1; Tue, 18 May 2021 17:16:42 -0400 X-MC-Unique: 0yCA03L2M3Cybqtns9xLxA-1 Received: by mail-qv1-f72.google.com with SMTP id f17-20020a0cf3d10000b02901eda24e6b92so8483129qvm.1 for ; Tue, 18 May 2021 14:16:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6yuOc2aL9R33jaIl7mlMHhqH6hbm5A0t+kn6e6OKsWE=; b=rSrehaTDbI+xT+U35sO9f4RUec715sqDsFwoFlubvMLmUGjcBmVfOk5Kk9InHWErB0 Wmxz5WeEZWIic8MZKuZm5NrCmQCrcX4e3++Q1jM/Fig4/ySors1Syu02IYYWP6x1Kvzt EspizhWplAfPya0HeWNi8iyZu+zmjGlmv3tF4BAWYLoBa6m7ZbGWRU4Ua8/3B8WefwxY 2t+X8/qvSWfRx3YuppEHAcvyWwOQkRU9a+aaXpTlo67c/b6L07xStKGwFto1q4QXRNsL dUpc98iUoOu+PX5iN8e4MMBzhwI2egu9omDTNXYQ+IHp/DPEocPxOiUnlLM1RHiCD5u9 NYHA== X-Gm-Message-State: AOAM532Bmc2dXKujK6cRrUpdzNsPqORnNGY7508yDqBMR5oSU99k/pQl lId9H/i/fyJMRR8W75ZBabepiszJX9vgk3zhIHli8YsiX/oHV9ZBpPiwutJ55UsbnPhcGA7VjIV ucWYWiKDuPTE= X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111167qta.172.1621372601484; Tue, 18 May 2021 14:16:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7CflObEdpN93nC2c9rCnQMv5qBohaNF3R0O5eiJLrnt/E6MHpUTkD6EfDzN6tiXMS/VwRag== X-Received: by 2002:ac8:5b81:: with SMTP id a1mr7111124qta.172.1621372601142; Tue, 18 May 2021 14:16:41 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id v66sm13507150qkd.113.2021.05.18.14.16.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:16:40 -0700 (PDT) Date: Tue, 18 May 2021 17:16:38 -0400 From: Peter Xu To: Alistair Popple Cc: linux-mm@kvack.org, nouveau@lists.freedesktop.org, bskeggs@redhat.com, akpm@linux-foundation.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, jhubbard@nvidia.com, rcampbell@nvidia.com, jglisse@redhat.com, jgg@nvidia.com, hch@infradead.org, daniel@ffwll.ch, willy@infradead.org, bsingharora@gmail.com, Christoph Hellwig Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access Message-ID: References: <20210407084238.20443-1-apopple@nvidia.com> <20210407084238.20443-6-apopple@nvidia.com> MIME-Version: 1.0 In-Reply-To: <20210407084238.20443-6-apopple@nvidia.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=S7oIJY1p; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf13.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A0D32E0001AF X-Stat-Signature: d4zxjrqmea6175rp6daut1r1ra7i9ox9 X-HE-Tag: 1621372603-960060 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 07, 2021 at 06:42:35PM +1000, Alistair Popple wrote: [...] > +static bool try_to_protect(struct page *page, struct mm_struct *mm, > + unsigned long address, void *arg) > +{ > + struct ttp_args ttp = { > + .mm = mm, > + .address = address, > + .arg = arg, > + .valid = false, > + }; > + struct rmap_walk_control rwc = { > + .rmap_one = try_to_protect_one, > + .done = page_not_mapped, > + .anon_lock = page_lock_anon_vma_read, > + .arg = &ttp, > + }; > + > + /* > + * Restrict to anonymous pages for now to avoid potential writeback > + * issues. > + */ > + if (!PageAnon(page)) > + return false; > + > + /* > + * During exec, a temporary VMA is setup and later moved. > + * The VMA is moved under the anon_vma lock but not the > + * page tables leading to a race where migration cannot > + * find the migration ptes. Rather than increasing the > + * locking requirements of exec(), migration skips > + * temporary VMAs until after exec() completes. > + */ > + if (!PageKsm(page) && PageAnon(page)) > + rwc.invalid_vma = invalid_migration_vma; > + > + rmap_walk(page, &rwc); > + > + return ttp.valid && !page_mapcount(page); > +} I raised a question in the other thread regarding fork(): https://lore.kernel.org/lkml/YKQjmtMo+YQGx%2FwZ@t490s/ While I suddenly noticed that we may have similar issues even if we fork() before creating the ptes. In that case, we may see multiple read-only ptes pointing to the same page. We will convert all of them into device exclusive read ptes in rmap_walk() above, however how do we guarantee after all COW done in the parent and all the childs processes, the device owned page will be returned to the parent? E.g., if parent accesses the page earlier than the children processes (actually, as long as not the last one), do_wp_page() will do COW for parent on this page because refcount(page)>1, then the page seems to get lost to a random child too.. To resolve all these complexity, not sure whether try_to_protect() could enforce VM_DONTCOPY (needs madvise MADV_DONTFORK in the user app), meanwhile make sure mapcount(page)==1 before granting the page to the device, so that this will guarantee this mm owns this page forever, I think? It'll simplify fork() too as a side effect, since VM_DONTCOPY vma go away when fork. -- Peter Xu