RE: [PATCH v8 3/3] mm: fix double page fault on arm64 if PTE_AF is cleared

From: "Justin He (Arm Technology China)" <Justin.He@arm.com>
To: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: "Will Deacon" <will@kernel.org>,
	"Mark Rutland" <Mark.Rutland@arm.com>,
	"James Morse" <James.Morse@arm.com>,
	"Marc Zyngier" <maz@kernel.org>,
	"Matthew Wilcox" <willy@infradead.org>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"Suzuki Poulose" <Suzuki.Poulose@arm.com>,
	"Punit Agrawal" <punitagrawal@gmail.com>,
	"Anshuman Khandual" <Anshuman.Khandual@arm.com>,
	"Alex Van Brunt" <avanbrunt@nvidia.com>,
	"Robin Murphy" <Robin.Murphy@arm.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Ralph Campbell" <rcampbell@nvidia.com>,
	"hejianet@gmail.com" <hejianet@gmail.com>,
	"Kaly Xin (Arm Technology China)" <Kaly.Xin@arm.com>,
	nd <nd@arm.com>
Subject: RE: [PATCH v8 3/3] mm: fix double page fault on arm64 if PTE_AF is cleared
Date: Tue, 24 Sep 2019 06:43:06 +0000	[thread overview]
Message-ID: <DB7PR08MB3082BC38536AE16B056AEA05F7840@DB7PR08MB3082.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <20190923170433.GE10192@arrakis.emea.arm.com>

Hi Catalin
Please see an important comment inline, thanks

> -----Original Message-----
> From: Catalin Marinas <catalin.marinas@arm.com>
> Sent: 2019年9月24日 1:05
> To: Justin He (Arm Technology China) <Justin.He@arm.com>
> Cc: Will Deacon <will@kernel.org>; Mark Rutland
> <Mark.Rutland@arm.com>; James Morse <James.Morse@arm.com>; Marc
> Zyngier <maz@kernel.org>; Matthew Wilcox <willy@infradead.org>; Kirill A.
> Shutemov <kirill.shutemov@linux.intel.com>; linux-arm-
> kernel@lists.infradead.org; linux-kernel@vger.kernel.org; linux-
> mm@kvack.org; Suzuki Poulose <Suzuki.Poulose@arm.com>; Punit
> Agrawal <punitagrawal@gmail.com>; Anshuman Khandual
> <Anshuman.Khandual@arm.com>; Alex Van Brunt
> <avanbrunt@nvidia.com>; Robin Murphy <Robin.Murphy@arm.com>;
> Thomas Gleixner <tglx@linutronix.de>; Andrew Morton <akpm@linux-
> foundation.org>; Jérôme Glisse <jglisse@redhat.com>; Ralph Campbell
> <rcampbell@nvidia.com>; hejianet@gmail.com; Kaly Xin (Arm Technology
> China) <Kaly.Xin@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v8 3/3] mm: fix double page fault on arm64 if PTE_AF
> is cleared
> 
> On Sat, Sep 21, 2019 at 09:50:54PM +0800, Jia He wrote:
> > @@ -2151,21 +2163,53 @@ static inline void cow_user_page(struct page
> *dst, struct page *src, unsigned lo
> >  	 * fails, we just zero-fill it. Live with it.
> >  	 */
> >  	if (unlikely(!src)) {
> > -		void *kaddr = kmap_atomic(dst);
> > -		void __user *uaddr = (void __user *)(va & PAGE_MASK);
> > +		void *kaddr;
> > +		pte_t entry;
> > +		void __user *uaddr = (void __user *)(addr & PAGE_MASK);
> >
> > +		/* On architectures with software "accessed" bits, we would
> > +		 * take a double page fault, so mark it accessed here.
> > +		 */
> 
> Nitpick: please follow the kernel coding style for multi-line comments
> (above and the for the rest of the patch):
> 
> 		/*
> 		 * Your multi-line comment.
> 		 */
> 
> > +		if (arch_faults_on_old_pte() && !pte_young(vmf->orig_pte))
> {
> > +			vmf->pte = pte_offset_map_lock(mm, vmf->pmd,
> addr,
> > +						       &vmf->ptl);
> > +			if (likely(pte_same(*vmf->pte, vmf->orig_pte))) {
> > +				entry = pte_mkyoung(vmf->orig_pte);
> > +				if (ptep_set_access_flags(vma, addr,
> > +							  vmf->pte, entry, 0))
> > +					update_mmu_cache(vma, addr, vmf-
> >pte);
> > +			} else {
> > +				/* Other thread has already handled the
> fault
> > +				 * and we don't need to do anything. If it's
> > +				 * not the case, the fault will be triggered
> > +				 * again on the same address.
> > +				 */
> > +				pte_unmap_unlock(vmf->pte, vmf->ptl);
> > +				return false;
> > +			}
> > +			pte_unmap_unlock(vmf->pte, vmf->ptl);
> > +		}
> 
> Another nit, you could rewrite this block slightly to avoid too much
> indentation. Something like (untested):
> 
> 		if (arch_faults_on_old_pte() && !pte_young(vmf->orig_pte))
> {
> 			vmf->pte = pte_offset_map_lock(mm, vmf->pmd,
> addr,
> 						       &vmf->ptl);
> 			if (unlikely(!pte_same(*vmf->pte, vmf->orig_pte))) {
> 				/*
> 				 * Other thread has already handled the fault
> 				 * and we don't need to do anything. If it's
> 				 * not the case, the fault will be triggered
> 				 * again on the same address.
> 				 */
> 				pte_unmap_unlock(vmf->pte, vmf->ptl);
> 				return false;
> 			}
> 			entry = pte_mkyoung(vmf->orig_pte);
> 			if (ptep_set_access_flags(vma, addr,
> 						  vmf->pte, entry, 0))
> 				update_mmu_cache(vma, addr, vmf->pte);
> 			pte_unmap_unlock(vmf->pte, vmf->ptl);
> 		}
> 
> > +
> > +		kaddr = kmap_atomic(dst);
> 
> Since you moved the kmap_atomic() here, could the above
> arch_faults_on_old_pte() run in a preemptible context? I suggested to
> add a WARN_ON in patch 2 to be sure.

Should I move kmap_atomic back to the original line? Thus, we can make sure
that arch_faults_on_old_pte() is in the context of preempt_disabled?
Otherwise, arch_faults_on_old_pte() may cause plenty of warning if I add
a WARN_ON in arch_faults_on_old_pte.  I tested it when I enable the PREEMPT=y
on a ThunderX2 qemu guest.

--
Cheers,
Justin (Jia He)

> 
> >  		/*
> >  		 * This really shouldn't fail, because the page is there
> >  		 * in the page tables. But it might just be unreadable,
> >  		 * in which case we just give up and fill the result with
> >  		 * zeroes.
> >  		 */
> > -		if (__copy_from_user_inatomic(kaddr, uaddr, PAGE_SIZE))
> > +		if (__copy_from_user_inatomic(kaddr, uaddr, PAGE_SIZE)) {
> > +			/* Give a warn in case there can be some obscure
> > +			 * use-case
> > +			 */
> > +			WARN_ON_ONCE(1);
> 
> That's more of a question for the mm guys: at this point we do the
> copying with the ptl released; is there anything else that could have
> made the pte old in the meantime? I think unuse_pte() is only called on
> anonymous vmas, so it shouldn't be the case here.
> 
> >  			clear_page(kaddr);
> > +		}
> >  		kunmap_atomic(kaddr);
> >  		flush_dcache_page(dst);
> >  	} else
> > -		copy_user_highpage(dst, src, va, vma);
> > +		copy_user_highpage(dst, src, addr, vma);
> > +
> > +	return true;
> >  }
> 
> --
> Catalin