From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 10 Feb 2016 13:35:18 +0100 From: Jan Kara Subject: Re: Another proposal for DAX fault locking Message-ID: <20160210123518.GG12245@quack.suse.cz> References: <20160209172416.GB12245@quack.suse.cz> <87egck4ukx.fsf@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87egck4ukx.fsf@openvz.org> Sender: owner-linux-mm@kvack.org To: Dmitry Monakhov Cc: Jan Kara , Ross Zwisler , linux-nvdimm@lists.01.org, Dave Chinner , linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, linux-fsdevel@vger.kernel.org List-ID: On Wed 10-02-16 15:29:34, Dmitry Monakhov wrote: > Jan Kara writes: > > > Hello, > > > > I was thinking about current issues with DAX fault locking [1] (data > > corruption due to racing faults allocating blocks) and also races which > > currently don't allow us to clear dirty tags in the radix tree due to races > > between faults and cache flushing [2]. Both of these exist because we don't > > have an equivalent of page lock available for DAX. While we have a > > reasonable solution available for problem [1], so far I'm not aware of a > > decent solution for [2]. After briefly discussing the issue with Mel he had > > a bright idea that we could used hashed locks to deal with [2] (and I think > > we can solve [1] with them as well). So my proposal looks as follows: > > > > DAX will have an array of mutexes (the array can be made per device but > > initially a global one should be OK). We will use mutexes in the array as a > > replacement for page lock - we will use hashfn(mapping, index) to get > > particular mutex protecting our offset in the mapping. On fault / page > > mkwrite, we'll grab the mutex similarly to page lock and release it once we > > are done updating page tables. This deals with races in [1]. When flushing > > caches we grab the mutex before clearing writeable bit in page tables > > and clearing dirty bit in the radix tree and drop it after we have flushed > > caches for the pfn. This deals with races in [2]. > > > > Thoughts? > Agree, only small note: > Hash locks has side effect for batch locking due to collision. > Some times we want to lock several pages/entries (migration/defragmentation) > So we will endup with deadlock due to hash collision. Yeah, but at least for the purposes we want the locks for locking just one 'page' is enough. If we ever needed locking more 'pages', we would have to choose a different locking scheme. Honza -- Jan Kara SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750981AbcBJMfJ (ORCPT ); Wed, 10 Feb 2016 07:35:09 -0500 Received: from mx2.suse.de ([195.135.220.15]:44316 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750801AbcBJMfF (ORCPT ); Wed, 10 Feb 2016 07:35:05 -0500 Date: Wed, 10 Feb 2016 13:35:18 +0100 From: Jan Kara To: Dmitry Monakhov Cc: Jan Kara , Ross Zwisler , linux-nvdimm@ml01.01.org, Dave Chinner , linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, linux-fsdevel@vger.kernel.org Subject: Re: Another proposal for DAX fault locking Message-ID: <20160210123518.GG12245@quack.suse.cz> References: <20160209172416.GB12245@quack.suse.cz> <87egck4ukx.fsf@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87egck4ukx.fsf@openvz.org> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed 10-02-16 15:29:34, Dmitry Monakhov wrote: > Jan Kara writes: > > > Hello, > > > > I was thinking about current issues with DAX fault locking [1] (data > > corruption due to racing faults allocating blocks) and also races which > > currently don't allow us to clear dirty tags in the radix tree due to races > > between faults and cache flushing [2]. Both of these exist because we don't > > have an equivalent of page lock available for DAX. While we have a > > reasonable solution available for problem [1], so far I'm not aware of a > > decent solution for [2]. After briefly discussing the issue with Mel he had > > a bright idea that we could used hashed locks to deal with [2] (and I think > > we can solve [1] with them as well). So my proposal looks as follows: > > > > DAX will have an array of mutexes (the array can be made per device but > > initially a global one should be OK). We will use mutexes in the array as a > > replacement for page lock - we will use hashfn(mapping, index) to get > > particular mutex protecting our offset in the mapping. On fault / page > > mkwrite, we'll grab the mutex similarly to page lock and release it once we > > are done updating page tables. This deals with races in [1]. When flushing > > caches we grab the mutex before clearing writeable bit in page tables > > and clearing dirty bit in the radix tree and drop it after we have flushed > > caches for the pfn. This deals with races in [2]. > > > > Thoughts? > Agree, only small note: > Hash locks has side effect for batch locking due to collision. > Some times we want to lock several pages/entries (migration/defragmentation) > So we will endup with deadlock due to hash collision. Yeah, but at least for the purposes we want the locks for locking just one 'page' is enough. If we ever needed locking more 'pages', we would have to choose a different locking scheme. Honza -- Jan Kara SUSE Labs, CR