From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751922AbdFOBFj (ORCPT ); Wed, 14 Jun 2017 21:05:39 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39160 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751590AbdFOBFh (ORCPT ); Wed, 14 Jun 2017 21:05:37 -0400 Subject: Re: [HELP-NEEDED, PATCH 0/3] Do not loose dirty bit on THP pages To: Will Deacon Cc: "Kirill A. Shutemov" , Andrew Morton , Vlastimil Babka , Vineet Gupta , Russell King , Catalin Marinas , Ralf Baechle , "David S. Miller" , Heiko Carstens , Martin Schwidefsky , Andrea Arcangeli , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, mark.rutland@arm.com References: <20170614135143.25068-1-kirill.shutemov@linux.intel.com> <20170614165513.GD17632@arm.com> From: "Aneesh Kumar K.V" Date: Thu, 15 Jun 2017 06:35:21 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <20170614165513.GD17632@arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 x-cbid: 17061501-0008-0000-0000-00000805A062 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007233; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000212; SDB=6.00874894; UDB=6.00435546; IPR=6.00654986; BA=6.00005421; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015830; XFM=3.00000015; UTC=2017-06-15 01:05:35 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17061501-0009-0000-0000-0000429F83EE Message-Id: <548e33cb-e737-bb39-91a3-f66ee9211262@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-06-14_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706150017 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wednesday 14 June 2017 10:25 PM, Will Deacon wrote: > Hi Aneesh, > > On Wed, Jun 14, 2017 at 08:55:26PM +0530, Aneesh Kumar K.V wrote: >> On Wednesday 14 June 2017 07:21 PM, Kirill A. Shutemov wrote: >>> Vlastimil noted that pmdp_invalidate() is not atomic and we can loose >>> dirty and access bits if CPU sets them after pmdp dereference, but >>> before set_pmd_at(). >>> >>> The bug doesn't lead to user-visible misbehaviour in current kernel, but >>> fixing this would be critical for future work on THP: both huge-ext4 and THP >>> swap out rely on proper dirty tracking. >>> >>> Unfortunately, there's no way to address the issue in a generic way. We need to >>> fix all architectures that support THP one-by-one. >>> >>> All architectures that have THP supported have to provide atomic >>> pmdp_invalidate(). If generic implementation of pmdp_invalidate() is used, >>> architecture needs to provide atomic pmdp_mknonpresent(). >>> >>> I've fixed the issue for x86, but I need help with the rest. >>> >>> So far THP is supported on 8 architectures. Power and S390 already provides >>> atomic pmdp_invalidate(). x86 is fixed by this patches, so 5 architectures >>> left: >>> >>> - arc; >>> - arm; >>> - arm64; >>> - mips; >>> - sparc -- it has custom pmdp_invalidate(), but it's racy too; >>> >>> Please, help me with them. >>> >>> Kirill A. Shutemov (3): >>> x86/mm: Provide pmdp_mknotpresent() helper >>> mm: Do not loose dirty and access bits in pmdp_invalidate() >>> mm, thp: Do not loose dirty bit in __split_huge_pmd_locked() >>> >> >> >> But in __split_huge_pmd_locked() we collected the dirty bit early. So even >> if we made pmdp_invalidate() atomic, if we had marked the pmd pte entry >> dirty after we collected the dirty bit, we still loose it right ? >> >> >> May be we should relook at pmd PTE udpate interface. We really need an >> interface that can update pmd entries such that we don't clear it in >> between. IMHO, we can avoid the pmdp_invalidate() completely, if we can >> switch from a pmd PTE entry to a pointer to PTE page (pgtable_t). We also >> need this interface to avoid the madvise race fixed by > > There's a good chance I'm not following your suggestion here, but it's > probably worth me pointing out that swizzling a page table entry from a > block mapping (e.g. a huge page mapped at the PMD level) to a table entry > (e.g. a pointer to a page of PTEs) can lead to all sorts of horrible > problems on ARM, including amalgamation of TLB entries and fatal aborts. > > So we really need to go via an invalid entry, with appropriate TLB > invalidation before installing the new entry. > I am not suggesting we don't do the invalidate (the need for that is documented in __split_huge_pmd_locked(). I am suggesting we need a new interface, something like Andrea suggested. old_pmd = pmdp_establish(pmd_mknotpresent()); instead of pmdp_invalidate(). We can then use this in scenarios where we want to update pmd PTE entries, where right now we go through a pmdp_clear and set_pmd path. We should really not do that for THP entries. W.r.t pmdp_invalidate() usage, I was wondering whether we can do that early in __split_huge_pmd_locked(). -aneesh