From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 8AC1B7CA2 for ; Wed, 15 Jun 2016 09:48:09 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 16AC0AC002 for ; Wed, 15 Jun 2016 07:48:05 -0700 (PDT) Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id J8S95Gu914820GTo (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 15 Jun 2016 07:48:03 -0700 (PDT) Date: Wed, 15 Jun 2016 07:48:01 -0700 From: Christoph Hellwig Subject: Re: [PATCH 10/12] NFS: Do not serialise O_DIRECT reads and writes Message-ID: <20160615144801.GB18524@infradead.org> References: <1465931115-30784-3-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-4-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-5-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-6-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-7-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-8-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-9-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-10-git-send-email-trond.myklebust@primarydata.com> <20160615071343.GC4318@infradead.org> <755A2A14-C6A9-4737-8335-0A6785490F6D@primarydata.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <755A2A14-C6A9-4737-8335-0A6785490F6D@primarydata.com> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: xfs-bounces@oss.sgi.com Sender: xfs-bounces@oss.sgi.com To: Trond Myklebust Cc: Christoph Hellwig , "linux-nfs@vger.kernel.org" , xfs@oss.sgi.com On Wed, Jun 15, 2016 at 02:29:42PM +0000, Trond Myklebust wrote: > The locking is actually simpler than XFS. It looks way more complicated. And totally undocumented. > We have 2 I/O modes: buffered I/O and direct I/O. The write lock is there to ensure safe transitions between those 2 modes, but once the mode is set, > we _only_ use shared locks in order to allow parallelism. >>From reading the patch that's not what actually happens - I think you're still taking i_rwsem exclusive for buffered writes, aren't you? Doing that is absolutely mandatory for Posix atomicy requirements, or you'll break tons of of applications. But XFS allows full parallelism for direct reads and writes as long as there is no more pagecache to flush. But if you have pages in the pagecache you need the exclusive lock to prevent against new pagecache pages being added. > >The nice thing is than in 4.7-rc i_mutex has been replaced with a > >rw_mutex so you can just use that in shared mode for direct I/O > >as-is without needing any new lock. > > We would end up serialising reads and writes, since the latter grab an > exclusive lock in generic_file_write(). Why do that if we don???t have to? Looks at the XFS code - no serialization between direct reads and writes as long as no buffered I/O came in inbetween. And don't use generic_file_{read,write}_iter if you want to do direct I/O, unfortunately locking in mm/filemap.c is totally screwed for direct I/O, take a look at XFS which is where direct I/O came from and where we get the locking right. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from bombadil.infradead.org ([198.137.202.9]:42642 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932290AbcFOOsC (ORCPT ); Wed, 15 Jun 2016 10:48:02 -0400 Date: Wed, 15 Jun 2016 07:48:01 -0700 From: Christoph Hellwig To: Trond Myklebust Cc: Christoph Hellwig , "linux-nfs@vger.kernel.org" , xfs@oss.sgi.com Subject: Re: [PATCH 10/12] NFS: Do not serialise O_DIRECT reads and writes Message-ID: <20160615144801.GB18524@infradead.org> References: <1465931115-30784-3-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-4-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-5-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-6-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-7-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-8-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-9-git-send-email-trond.myklebust@primarydata.com> <1465931115-30784-10-git-send-email-trond.myklebust@primarydata.com> <20160615071343.GC4318@infradead.org> <755A2A14-C6A9-4737-8335-0A6785490F6D@primarydata.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <755A2A14-C6A9-4737-8335-0A6785490F6D@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org List-ID: On Wed, Jun 15, 2016 at 02:29:42PM +0000, Trond Myklebust wrote: > The locking is actually simpler than XFS. It looks way more complicated. And totally undocumented. > We have 2 I/O modes: buffered I/O and direct I/O. The write lock is there to ensure safe transitions between those 2 modes, but once the mode is set, > we _only_ use shared locks in order to allow parallelism.