From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39271C77B76 for ; Tue, 4 Apr 2023 13:59:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCEC9900005; Tue, 4 Apr 2023 09:59:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7E1E900003; Tue, 4 Apr 2023 09:59:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF78F900005; Tue, 4 Apr 2023 09:59:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A1127900003 for ; Tue, 4 Apr 2023 09:59:09 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 77121A0962 for ; Tue, 4 Apr 2023 13:59:09 +0000 (UTC) X-FDA: 80643865218.05.D78BACC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP id 8501F100021 for ; Tue, 4 Apr 2023 13:59:07 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Yw+GAfLt; spf=none (imf14.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680616747; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jRN9LMIQY9dyThVaq2oo0Jj1gstfJQImVo8wGgQY6/Q=; b=rX6YTkQVT1g2crD9G7nN/kIXo1cXPrW4gjvVtWakRVuxZVs07ZpWiTmC9JKJcow5WpSKzk LHqcfciDU7YTfezqXFob8GsW2MiiJ65MGENrbQumw81Ryu5niQZYzWZXxP2wAzthTpJXTY FTBV+pMrDwOeIPBQPk0wpIZAiwjKRKk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Yw+GAfLt; spf=none (imf14.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680616747; a=rsa-sha256; cv=none; b=oNGyOwf2dnF0mtyY0sAgw1mbAqPZ0TX+hXM3nW0iwTyDSVDjQuXuV0a8IwIN6KGpvP3nLv O7LU6bVqO/h6UWp0ZBiUKdnJT6dvheEcDZOLIZKoBBNCOZYuiuNT0be9QTcmjdyatbyoPa uEzsCBCJKzU1ibra2C9RbVyeZoT8Hhw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=jRN9LMIQY9dyThVaq2oo0Jj1gstfJQImVo8wGgQY6/Q=; b=Yw+GAfLtiLfrvZGq+OER5Mn4PY +J0pXFeOBou2VBOR4k2CLiBm1tUeMHkv9r0ZhgGnFjFTfK+4BvkUG4x73vi5rXykhGS9VcgC+/FkX mfSd42Q8SVIYoUvx7B34jKjN9JSy7gzP2dNtFDh4TmIUOsJ2bwppQsAvzQvl9rcxQobgu2f+XEzRV X1tipMKhYfhar5o0Mfx0vwkXW0BAGIPHZlC3GVBT1QD5K9g4yiGE9emh9rG0GslClpPdkW6FLbRS8 9wnhkIFz0AOs2gDOASmoWlKVlASpOYQlrQRlKXEvSbe0BoyJ9zmdd6Si5hRfh9pnromJ/fdl3haiu te7v2ZAw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pjhBU-00FPch-6s; Tue, 04 Apr 2023 13:58:52 +0000 From: "Matthew Wilcox (Oracle)" To: Suren Baghdasaryan Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Punit Agrawal Subject: [PATCH 2/6] mm: Move FAULT_FLAG_VMA_LOCK check from handle_mm_fault() Date: Tue, 4 Apr 2023 14:58:46 +0100 Message-Id: <20230404135850.3673404-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230404135850.3673404-1-willy@infradead.org> References: <20230404135850.3673404-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 8501F100021 X-Stat-Signature: gnru9tg1pg1o79qhrxoa59bhuutwstqh X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680616747-527928 X-HE-Meta: U2FsdGVkX1/qKk+0B8lCzBBuncYWf/aWXdYDMxOzDvOWk/yrX+rqQikkMMSvgeDqiUfCYzeSGQr73wuB+xUp5Z6Ws0QM7IMoKBof6ZgZqEt5lOKBqrJzDbIYGGI3DiEPNvWLLtTryHMUkA9zF+IO98VoSfSNzJbONLHQm04XOaZmiU3+GBPhtrU3TjjFzWz9kpcODxYd73PCmN3nL4mw69ii3e/YAhN0XR4irGjvIicY4pbnOYn4sg3Eq/SkJPMZRXfnff2ZAHSkuFXzE8RoVYbm4aY8t9FIme7v3zBzB0GSMYx+L/4UHzZucqqQTrYbYTNi5goP2EP7wty4YGD48sSU8gp3KFRt5xpuCDwv/Ax7wIDlOwhW/YrsimeVx0P4K/oynGcV8Ol4vmIHYS8SVaN94T/LFtottHFf28VgYpIXsEJq6lABF287sjAvUG9SwpWaHyk56HEJDLvDv3Wv7TpNQR5BAiIbT2MFrelwNkCBcPefEhzMNDvi1zdgTGimAB8ncoRjGXfJs7vIi9tB1qD6S2/Wo6+8UiwqMG9VzFGyugqUiTBj0GW8qa3DYXHhaDwT4Q/DjY53uAZ6unlgK1a36vocr+gKX0uhtUkuY9CwfqvplOWh9UTtZua013cT+fbhElJNi5EZmBpCsBNWXaZy28KOdWxm6Vny29Q1Mgmv4o/6hfCTuh3L4q/LjzkBkMRDFmIGZG930E0ScM9PxP3errxRNRMwSv3tbHo66Gla1CFOvRa9hdc2QRqT5eQBXto0Jkbhys6KMsZ0vVGQFDeoSZG+LkSe2uNLbcj1SlcBDJwzpa8JAJnqxUSvhCr6PyzH9NPuPpJfPsDcQBq3QJu2R0sp4ilkrXyXfWSYBS6rQevomsVxKi9xa/H92rZEZnzgF4JsX4pZi3kAzc6e+Ix+co10EP8VoYcg5vze7onMh934NQtJWQYYmsfG1pZD+abjxx1UxjYg/GxxbXj 4zgx2NTP YM0ILyFb5tK6v1AiWoH4eQbrVlWB/7mHExh+FJ1o4k4d7pP3GsFyn59iTwkegxs/YPpml4ah7TvFLHvzA4EKRk8FXxGodb4mxUJr+VNWUhRCSH67y/4x5spGyGR6gp7TBjA5ubYWXV8xx/l2iV/W1D7UJGgMyKUAVWq31VgGazV9DiY1Bbr54MSPNf61pJMUyFsolt6k5BLSkas0zPB7/F4yYiLwqXea0DKwXzs2Gzfcx67bDeK/4c2qT8w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Handle a little more of the page fault path outside the mmap sem. The hugetlb path doesn't need to check whether the VMA is anonymous; the VM_HUGETLB flag is only set on hugetlbfs VMAs. There should be no performance change from the previous commit; this is simply a step to ease bisection of any problems. Signed-off-by: Matthew Wilcox (Oracle) --- mm/hugetlb.c | 4 ++++ mm/memory.c | 14 +++++++------- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index efc443a906fa..39f168e3518f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6052,6 +6052,10 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, int need_wait_lock = 0; unsigned long haddr = address & huge_page_mask(h); + /* TODO: Handle faults under the VMA lock */ + if (flags & FAULT_FLAG_VMA_LOCK) + return VM_FAULT_RETRY; + /* * Serialize hugepage allocation and instantiation, so that we don't * get spurious allocation failures if two CPUs race to instantiate diff --git a/mm/memory.c b/mm/memory.c index f726f85f0081..fc1f0ef9a7a5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4992,10 +4992,10 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) } /* - * By the time we get here, we already hold the mm semaphore - * - * The mmap_lock may have been released depending on flags and our - * return value. See filemap_fault() and __folio_lock_or_retry(). + * On entry, we hold either the VMA lock or the mmap_lock + * (FAULT_FLAG_VMA_LOCK tells you which). If VM_FAULT_RETRY is set in + * the result, the mmap_lock is not held on exit. See filemap_fault() + * and __folio_lock_or_retry(). */ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags) @@ -5014,6 +5014,9 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, p4d_t *p4d; vm_fault_t ret; + if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) + return VM_FAULT_RETRY; + pgd = pgd_offset(mm, address); p4d = p4d_alloc(mm, pgd, address); if (!p4d) @@ -5223,9 +5226,6 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, flags & FAULT_FLAG_REMOTE)) return VM_FAULT_SIGSEGV; - if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) - return VM_FAULT_RETRY; - /* * Enable the memcg OOM handling for faults triggered in user * space. Kernel faults are handled more gracefully. -- 2.39.2