From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 727A0C433DB for ; Thu, 18 Feb 2021 21:55:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E303264EB3 for ; Thu, 18 Feb 2021 21:55:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E303264EB3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 582D86B0005; Thu, 18 Feb 2021 16:55:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 50D9F6B006C; Thu, 18 Feb 2021 16:55:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3606A6B006E; Thu, 18 Feb 2021 16:55:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id 191C06B0005 for ; Thu, 18 Feb 2021 16:55:56 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D9B2F612C for ; Thu, 18 Feb 2021 21:55:55 +0000 (UTC) X-FDA: 77832746670.10.20F400D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf18.hostedemail.com (Postfix) with ESMTP id C983420001DC for ; Thu, 18 Feb 2021 21:55:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613685354; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e4OLUk0Jy8zIZlRHoRelkknb6fgRUan1/BmpVqTVRpo=; b=Eb7tD6ie1ok1pVld6UCZWZzSPuFOlBi05RUkpTKY7Jlof9DyBDdOV8MFRcQvMayA0Vs+0U 9y5CV3dzMJR/bKFzygnpi4lq3UtuHbjFwGsLCxa/vidDBZ1nLRCENMPfaghdCycWSuHtTf /bU7asfAdoGcQpaAbzLmptwEBTxNU0c= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-554-Rd6KrwtqOsuYKlvb22F0Bg-1; Thu, 18 Feb 2021 16:55:53 -0500 X-MC-Unique: Rd6KrwtqOsuYKlvb22F0Bg-1 Received: by mail-qt1-f198.google.com with SMTP id f3so2056839qtx.18 for ; Thu, 18 Feb 2021 13:55:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e4OLUk0Jy8zIZlRHoRelkknb6fgRUan1/BmpVqTVRpo=; b=m39vHWDeTD/N5qpvGa2No1EXU03cARDi/AA4ItoBn2c0dq0tAoVnzda5OuNppOltMH owzumMbplT9edoD6ftLuHuZnm5Mbd9kdbxkNpxz3lLwkQfg4PN3T15V+lAMxl78u50k9 LmkRMunhT03XVGb169R69LhJ3ZX+SFWrQgivwGAouukGeS8DiQDovFJdERXPX/qUwto/ Xmj0KfLH9LRyqVvO4eMcFp9B4DQZOiy52p8qT1wUAi3/gfNJw0U3FbtsDN17TZ20Kr7W u+Mo2GvagM3e5rarBeOV1qOgrpmowoAtNRTBONgA6HZMKgUGfPr1noPfP+4tD/Qlo/+D 8k7w== X-Gm-Message-State: AOAM530Z6YdnnUK5wyVTryD2Syp96p2RypJo8BH7jUCJKLk/FW8txgmf 7skwnoonU83p2HlcQDYOB/qwPQ/0IyX10wbhlZT3s4ZPFSYdMpCHf3vuh9MMbzsD7xfQ20cJQ9u A2zzBoZO9f4k= X-Received: by 2002:a05:620a:227:: with SMTP id u7mr6649929qkm.226.1613685353124; Thu, 18 Feb 2021 13:55:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJzPwEwqxlrFOFzSQ7p6jnGuKozTyPWx/xZeRZ1As4UoBD2Ea1DWzSKkNh01txbsI90r/Aw5MA== X-Received: by 2002:a05:620a:227:: with SMTP id u7mr6649915qkm.226.1613685352896; Thu, 18 Feb 2021 13:55:52 -0800 (PST) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-20-174-93-89-182.dsl.bell.ca. [174.93.89.182]) by smtp.gmail.com with ESMTPSA id c126sm4863324qkg.16.2021.02.18.13.55.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Feb 2021 13:55:52 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Mike Kravetz , Andrew Morton , Mike Rapoport , Andrea Arcangeli , peterx@redhat.com, Axel Rasmussen , "Kirill A . Shutemov" , Matthew Wilcox Subject: [PATCH v3 2/4] hugetlb/userfaultfd: Forbid huge pmd sharing when uffd enabled Date: Thu, 18 Feb 2021 16:55:51 -0500 Message-Id: <20210218215551.10612-1-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210218215434.10203-1-peterx@redhat.com> References: <20210218215434.10203-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="US-ASCII" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C983420001DC X-Stat-Signature: a9nbmjae43qqmsmcb7g8utn17xebmm4y Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=63.128.21.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613685355-827774 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Huge pmd sharing could bring problem to userfaultfd. The thing is that userfaultfd is running its logic based on the special bits on page table entries, however the huge pmd sharing could potentially share page table entries for different address ranges. That could cause issues on either: - When sharing huge pmd page tables for an uffd write protected range, = the newly mapped huge pmd range will also be write protected unexpectedly= , or, - When we try to write protect a range of huge pmd shared range, we'll = first do huge_pmd_unshare() in hugetlb_change_protection(), however that al= so means the UFFDIO_WRITEPROTECT could be silently skipped for the share= d region, which could lead to data loss. Since at it, a few other things are done altogether: - Move want_pmd_share() from mm/hugetlb.c into linux/hugetlb.h, because that's definitely something that arch code would like to use too - ARM64 currently directly check against CONFIG_ARCH_WANT_HUGE_PMD_SHAR= E when trying to share huge pmd. Switch to the want_pmd_share() helper. Since at it, move vma_shareable() from huge_pmd_share() into want_pmd_sha= re(). Reviewed-by: Mike Kravetz Reviewed-by: Axel Rasmussen Signed-off-by: Peter Xu --- arch/arm64/mm/hugetlbpage.c | 3 +-- include/linux/hugetlb.h | 2 ++ include/linux/userfaultfd_k.h | 9 +++++++++ mm/hugetlb.c | 20 ++++++++++++++------ 4 files changed, 26 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 6e3bcffe2837..58987a98e179 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,8 +284,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm= _area_struct *vma, */ ptep =3D pte_alloc_map(mm, pmdp, addr); } else if (sz =3D=3D PMD_SIZE) { - if (IS_ENABLED(CONFIG_ARCH_WANT_HUGE_PMD_SHARE) && - pud_none(READ_ONCE(*pudp))) + if (want_pmd_share(vma, addr) && pud_none(READ_ONCE(*pudp))) ptep =3D huge_pmd_share(mm, vma, addr, pudp); else ptep =3D (pte_t *)pmd_alloc(mm, pudp, addr); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index a6113fa6d21d..bc86f2f516e7 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -950,4 +950,6 @@ static inline __init void hugetlb_cma_check(void) } #endif =20 +bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr); + #endif /* _LINUX_HUGETLB_H */ diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.= h index a8e5f3ea9bb2..c63ccdae3eab 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -52,6 +52,15 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(str= uct vm_area_struct *vma, return vma->vm_userfaultfd_ctx.ctx =3D=3D vm_ctx.ctx; } =20 +/* + * Never enable huge pmd sharing on uffd-wp registered vmas, because uff= d-wp + * protect information is per pgtable entry. + */ +static inline bool uffd_disable_huge_pmd_share(struct vm_area_struct *vm= a) +{ + return vma->vm_flags & VM_UFFD_WP; +} + static inline bool userfaultfd_missing(struct vm_area_struct *vma) { return vma->vm_flags & VM_UFFD_MISSING; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 07bb9bdc3282..8e8e2f3dfe06 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5292,6 +5292,18 @@ static bool vma_shareable(struct vm_area_struct *v= ma, unsigned long addr) return false; } =20 +bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +{ +#ifndef CONFIG_ARCH_WANT_HUGE_PMD_SHARE + return false; +#endif +#ifdef CONFIG_USERFAULTFD + if (uffd_disable_huge_pmd_share(vma)) + return false; +#endif + return vma_shareable(vma, addr); +} + /* * Determine if start,end range within vma could be mapped by shared pmd= . * If yes, adjust start and end to cover range associated with possible @@ -5346,9 +5358,6 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct = vm_area_struct *vma, pte_t *pte; spinlock_t *ptl; =20 - if (!vma_shareable(vma, addr)) - return (pte_t *)pmd_alloc(mm, pud, addr); - i_mmap_assert_locked(mapping); vma_interval_tree_foreach(svma, &mapping->i_mmap, idx, idx) { if (svma =3D=3D vma) @@ -5412,7 +5421,7 @@ int huge_pmd_unshare(struct mm_struct *mm, struct v= m_area_struct *vma, *addr =3D ALIGN(*addr, HPAGE_SIZE * PTRS_PER_PTE) - HPAGE_SIZE; return 1; } -#define want_pmd_share() (1) + #else /* !CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct vma, unsigned long addr, pud_t *pud) @@ -5430,7 +5439,6 @@ void adjust_range_if_pmd_sharing_possible(struct vm= _area_struct *vma, unsigned long *start, unsigned long *end) { } -#define want_pmd_share() (0) #endif /* CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ =20 #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB @@ -5452,7 +5460,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct = vm_area_struct *vma, pte =3D (pte_t *)pud; } else { BUG_ON(sz !=3D PMD_SIZE); - if (want_pmd_share() && pud_none(*pud)) + if (want_pmd_share(vma, addr) && pud_none(*pud)) pte =3D huge_pmd_share(mm, vma, addr, pud); else pte =3D (pte_t *)pmd_alloc(mm, pud, addr); --=20 2.26.2