From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> References: <1454829553-29499-1-git-send-email-ross.zwisler@linux.intel.com> <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> Date: Sun, 7 Feb 2016 11:13:51 -0800 Message-ID: Subject: Re: [PATCH 2/2] dax: move writeback calls into the filesystems From: Dan Williams Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org To: Ross Zwisler Cc: "linux-kernel@vger.kernel.org" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Andrew Morton , Dave Chinner , Jan Kara , Matthew Wilcox , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , XFS Developers , jmoyer List-ID: On Sat, Feb 6, 2016 at 11:19 PM, Ross Zwisler wrote: > Previously calls to dax_writeback_mapping_range() for all DAX filesystems > (ext2, ext4 & xfs) were centralized in filemap_write_and_wait_range(). > dax_writeback_mapping_range() needs a struct block_device, and it used to > get that from inode->i_sb->s_bdev. This is correct for normal inodes > mounted on ext2, ext4 and XFS filesystems, but is incorrect for DAX raw > block devices and for XFS real-time files. > > Instead, call dax_writeback_mapping_range() directly from the filesystem or > raw block device fsync/msync code so that they can supply us with a valid > block device. > > It should be noted that this will reduce the number of calls to > dax_writeback_mapping_range() because filemap_write_and_wait_range() is > called in the various filesystems for operations other than just > fsync/msync. Both ext4 & XFS call filemap_write_and_wait_range() outside > of ->fsync for hole punch, truncate, and block relocation > (xfs_shift_file_space() && ext4_collapse_range()/ext4_insert_range()). > > I don't believe that these extra flushes are necessary in the DAX case. In > the page cache case when we have dirty data in the page cache, that data > will be actively lost if we evict a dirty page cache page without flushing > it to media first. For DAX, though, the data will remain consistent with > the physical address to which it was written regardless of whether it's in > the processor cache or not - really the only reason I see to flush is in > response to a fsync or msync so that our data is durable on media in case > of a power loss. The case where we could throw dirty data out of the page > cache and essentially lose writes simply doesn't exist. > > Signed-off-by: Ross Zwisler > --- > fs/block_dev.c | 7 +++++++ > fs/dax.c | 5 ++--- > fs/ext2/file.c | 10 ++++++++++ > fs/ext4/fsync.c | 10 +++++++++- > fs/xfs/xfs_file.c | 12 ++++++++++-- > include/linux/dax.h | 4 ++-- > mm/filemap.c | 6 ------ > 7 files changed, 40 insertions(+), 14 deletions(-) This sprinkling of dax specific fixups outside of vm_operations_struct routines still has me thinking that we are going in the wrong direction for fsync/msync support. If an application is both unaware of DAX and doing mmap I/O it is better served by the page cache where writeback is durable by default. We expect DAX-aware applications to assume responsibility for cpu cache management [1]. Making DAX mmap semantics explicit opt-in solves not only durability support, but also the current problem that DAX gets silently disabled leaving an app to wonder if it really got a direct mapping. DAX also silently picks pud, pmd, or pte mappings which is information an application would really like to know at map time. The proposal: make applications explicitly request DAX semantics with a new MAP_DAX flag and fail if DAX is unavailable. Document that a successful MAP_DAX request mandates that the application assumes responsibility for cpu cache management. Require that all applications that mmap the file agree on MAP_DAX. This also solves the future problem of DAX support on virtually tagged cache architectures where it is difficult for the kernel to know what alias addresses need flushing. [1]: https://github.com/pmem/nvml -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754541AbcBGTNz (ORCPT ); Sun, 7 Feb 2016 14:13:55 -0500 Received: from mail-yw0-f177.google.com ([209.85.161.177]:32981 "EHLO mail-yw0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754121AbcBGTNw (ORCPT ); Sun, 7 Feb 2016 14:13:52 -0500 MIME-Version: 1.0 In-Reply-To: <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> References: <1454829553-29499-1-git-send-email-ross.zwisler@linux.intel.com> <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> Date: Sun, 7 Feb 2016 11:13:51 -0800 Message-ID: Subject: Re: [PATCH 2/2] dax: move writeback calls into the filesystems From: Dan Williams To: Ross Zwisler Cc: "linux-kernel@vger.kernel.org" , "Theodore Ts'o" , Alexander Viro , Andreas Dilger , Andrew Morton , Dave Chinner , Jan Kara , Matthew Wilcox , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , XFS Developers , jmoyer Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Feb 6, 2016 at 11:19 PM, Ross Zwisler wrote: > Previously calls to dax_writeback_mapping_range() for all DAX filesystems > (ext2, ext4 & xfs) were centralized in filemap_write_and_wait_range(). > dax_writeback_mapping_range() needs a struct block_device, and it used to > get that from inode->i_sb->s_bdev. This is correct for normal inodes > mounted on ext2, ext4 and XFS filesystems, but is incorrect for DAX raw > block devices and for XFS real-time files. > > Instead, call dax_writeback_mapping_range() directly from the filesystem or > raw block device fsync/msync code so that they can supply us with a valid > block device. > > It should be noted that this will reduce the number of calls to > dax_writeback_mapping_range() because filemap_write_and_wait_range() is > called in the various filesystems for operations other than just > fsync/msync. Both ext4 & XFS call filemap_write_and_wait_range() outside > of ->fsync for hole punch, truncate, and block relocation > (xfs_shift_file_space() && ext4_collapse_range()/ext4_insert_range()). > > I don't believe that these extra flushes are necessary in the DAX case. In > the page cache case when we have dirty data in the page cache, that data > will be actively lost if we evict a dirty page cache page without flushing > it to media first. For DAX, though, the data will remain consistent with > the physical address to which it was written regardless of whether it's in > the processor cache or not - really the only reason I see to flush is in > response to a fsync or msync so that our data is durable on media in case > of a power loss. The case where we could throw dirty data out of the page > cache and essentially lose writes simply doesn't exist. > > Signed-off-by: Ross Zwisler > --- > fs/block_dev.c | 7 +++++++ > fs/dax.c | 5 ++--- > fs/ext2/file.c | 10 ++++++++++ > fs/ext4/fsync.c | 10 +++++++++- > fs/xfs/xfs_file.c | 12 ++++++++++-- > include/linux/dax.h | 4 ++-- > mm/filemap.c | 6 ------ > 7 files changed, 40 insertions(+), 14 deletions(-) This sprinkling of dax specific fixups outside of vm_operations_struct routines still has me thinking that we are going in the wrong direction for fsync/msync support. If an application is both unaware of DAX and doing mmap I/O it is better served by the page cache where writeback is durable by default. We expect DAX-aware applications to assume responsibility for cpu cache management [1]. Making DAX mmap semantics explicit opt-in solves not only durability support, but also the current problem that DAX gets silently disabled leaving an app to wonder if it really got a direct mapping. DAX also silently picks pud, pmd, or pte mappings which is information an application would really like to know at map time. The proposal: make applications explicitly request DAX semantics with a new MAP_DAX flag and fail if DAX is unavailable. Document that a successful MAP_DAX request mandates that the application assumes responsibility for cpu cache management. Require that all applications that mmap the file agree on MAP_DAX. This also solves the future problem of DAX support on virtually tagged cache architectures where it is difficult for the kernel to know what alias addresses need flushing. [1]: https://github.com/pmem/nvml From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 5D47B29DF5 for ; Sun, 7 Feb 2016 13:13:55 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 21D36304032 for ; Sun, 7 Feb 2016 11:13:55 -0800 (PST) Received: from mail-yw0-f179.google.com (mail-yw0-f179.google.com [209.85.161.179]) by cuda.sgi.com with ESMTP id sZhyZ1DCL2hUom5S (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 07 Feb 2016 11:13:52 -0800 (PST) Received: by mail-yw0-f179.google.com with SMTP id q190so88695217ywd.3 for ; Sun, 07 Feb 2016 11:13:52 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> References: <1454829553-29499-1-git-send-email-ross.zwisler@linux.intel.com> <1454829553-29499-3-git-send-email-ross.zwisler@linux.intel.com> Date: Sun, 7 Feb 2016 11:13:51 -0800 Message-ID: Subject: Re: [PATCH 2/2] dax: move writeback calls into the filesystems From: Dan Williams List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: xfs-bounces@oss.sgi.com Sender: xfs-bounces@oss.sgi.com To: Ross Zwisler Cc: Theodore Ts'o , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" , XFS Developers , Linux MM , jmoyer , Andreas Dilger , Alexander Viro , Jan Kara , linux-fsdevel , Matthew Wilcox , Andrew Morton , linux-ext4 On Sat, Feb 6, 2016 at 11:19 PM, Ross Zwisler wrote: > Previously calls to dax_writeback_mapping_range() for all DAX filesystems > (ext2, ext4 & xfs) were centralized in filemap_write_and_wait_range(). > dax_writeback_mapping_range() needs a struct block_device, and it used to > get that from inode->i_sb->s_bdev. This is correct for normal inodes > mounted on ext2, ext4 and XFS filesystems, but is incorrect for DAX raw > block devices and for XFS real-time files. > > Instead, call dax_writeback_mapping_range() directly from the filesystem or > raw block device fsync/msync code so that they can supply us with a valid > block device. > > It should be noted that this will reduce the number of calls to > dax_writeback_mapping_range() because filemap_write_and_wait_range() is > called in the various filesystems for operations other than just > fsync/msync. Both ext4 & XFS call filemap_write_and_wait_range() outside > of ->fsync for hole punch, truncate, and block relocation > (xfs_shift_file_space() && ext4_collapse_range()/ext4_insert_range()). > > I don't believe that these extra flushes are necessary in the DAX case. In > the page cache case when we have dirty data in the page cache, that data > will be actively lost if we evict a dirty page cache page without flushing > it to media first. For DAX, though, the data will remain consistent with > the physical address to which it was written regardless of whether it's in > the processor cache or not - really the only reason I see to flush is in > response to a fsync or msync so that our data is durable on media in case > of a power loss. The case where we could throw dirty data out of the page > cache and essentially lose writes simply doesn't exist. > > Signed-off-by: Ross Zwisler > --- > fs/block_dev.c | 7 +++++++ > fs/dax.c | 5 ++--- > fs/ext2/file.c | 10 ++++++++++ > fs/ext4/fsync.c | 10 +++++++++- > fs/xfs/xfs_file.c | 12 ++++++++++-- > include/linux/dax.h | 4 ++-- > mm/filemap.c | 6 ------ > 7 files changed, 40 insertions(+), 14 deletions(-) This sprinkling of dax specific fixups outside of vm_operations_struct routines still has me thinking that we are going in the wrong direction for fsync/msync support. If an application is both unaware of DAX and doing mmap I/O it is better served by the page cache where writeback is durable by default. We expect DAX-aware applications to assume responsibility for cpu cache management [1]. Making DAX mmap semantics explicit opt-in solves not only durability support, but also the current problem that DAX gets silently disabled leaving an app to wonder if it really got a direct mapping. DAX also silently picks pud, pmd, or pte mappings which is information an application would really like to know at map time. The proposal: make applications explicitly request DAX semantics with a new MAP_DAX flag and fail if DAX is unavailable. Document that a successful MAP_DAX request mandates that the application assumes responsibility for cpu cache management. Require that all applications that mmap the file agree on MAP_DAX. This also solves the future problem of DAX support on virtually tagged cache architectures where it is difficult for the kernel to know what alias addresses need flushing. [1]: https://github.com/pmem/nvml _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs