* [PATCH 0/3] Allow fscache to work on BTRFS @ 2015-04-20 5:27 NeilBrown 2015-04-20 5:27 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap NeilBrown ` (5 more replies) 0 siblings, 6 replies; 20+ messages in thread From: NeilBrown @ 2015-04-20 5:27 UTC (permalink / raw) To: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba Cc: linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs The following three patches allow fs to "cachefiles" in a BTRFS filesystem. The first is a minor cleanup to cachefiles. The second is the main change - it teaches cachefile to use lseek(SEEK_DATA) to find allocated blocks in a file, rather than bmap. The third patch simply enables this for btrfs. Thanks, NeilBrown --- NeilBrown (3): cachefiles: perform test on s_blocksize when opening cache file. fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. btrfs: set FS_SUPPORTS_SEEK_HOLE flag. fs/btrfs/super.c | 3 + fs/cachefiles/namei.c | 13 ++++- fs/cachefiles/rdwr.c | 125 ++++++++++++++++++++++++++++++------------------- include/linux/fs.h | 1 4 files changed, 88 insertions(+), 54 deletions(-) -- Signature ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown @ 2015-04-20 5:27 ` NeilBrown 2015-04-20 6:08 ` Christoph Hellwig 2015-04-20 5:27 ` [PATCH 1/3] cachefiles: perform test on s_blocksize when opening cache file NeilBrown ` (4 subsequent siblings) 5 siblings, 1 reply; 20+ messages in thread From: NeilBrown @ 2015-04-20 5:27 UTC (permalink / raw) To: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba Cc: linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs cachefiles currently uses 'bmap' to determine if a given block in a file has been cached, or not. Not all filesystems support bmap, particularly BTRFS. SEEK_DATA can be used to determine if a block in a file has been allocated, but not all filesystems support this reliably. On filesystems without explicit report, SEEK_DATA will report anything below i_size to be valid data. So: - add a file_system_type flag which confirms that SEEK_DATA and SEEK_HOLE will reliably report holes, - change cachefiles to use vfs_lseek if FS_SUPPORTS_SEEK_HOLE is set, and only use ->bmap if it isn't. Subsequent patch will set flag for btrfs. Other filesystems could usefully have FS_SUPPORTS_SEEK_HOLE set, but I'll leave that to the relevant maintainers to decide. Signed-off-by: NeilBrown <neilb@suse.de> --- fs/cachefiles/namei.c | 15 ++++-- fs/cachefiles/rdwr.c | 119 +++++++++++++++++++++++++++++++------------------ include/linux/fs.h | 1 3 files changed, 86 insertions(+), 49 deletions(-) diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c index 5404afcdee98..5d5e56c645ec 100644 --- a/fs/cachefiles/namei.c +++ b/fs/cachefiles/namei.c @@ -643,14 +643,17 @@ lookup_again: /* open a file interface onto a data file */ if (object->type != FSCACHE_COOKIE_TYPE_INDEX) { if (d_is_reg(object->dentry)) { - const struct address_space_operations *aops; ret = -EPERM; - aops = object->dentry->d_inode->i_mapping->a_ops; - if (!aops->bmap) - goto check_error; - if (object->dentry->d_sb->s_blocksize > PAGE_SIZE) - goto check_error; + if (!(object->dentry->d_sb->s_type->fs_flags + & FS_SUPPORTS_SEEK_HOLE)) { + const struct address_space_operations *aops; + aops = object->dentry->d_inode->i_mapping->a_ops; + if (!aops->bmap) + goto check_error; + if (object->dentry->d_sb->s_blocksize > PAGE_SIZE) + goto check_error; + } object->backer = object->dentry; } else { diff --git a/fs/cachefiles/rdwr.c b/fs/cachefiles/rdwr.c index 9bd44ff48cc0..7c7cbfae7b19 100644 --- a/fs/cachefiles/rdwr.c +++ b/fs/cachefiles/rdwr.c @@ -394,9 +394,8 @@ int cachefiles_read_or_alloc_page(struct fscache_retrieval *op, struct cachefiles_object *object; struct cachefiles_cache *cache; struct inode *inode; - sector_t block0, block; - unsigned shift; int ret; + bool have_data; object = container_of(op->op.object, struct cachefiles_object, fscache); @@ -410,31 +409,47 @@ int cachefiles_read_or_alloc_page(struct fscache_retrieval *op, inode = object->backer->d_inode; ASSERT(S_ISREG(inode->i_mode)); - ASSERT(inode->i_mapping->a_ops->bmap); ASSERT(inode->i_mapping->a_ops->readpages); - /* calculate the shift required to use bmap */ - shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; - op->op.flags &= FSCACHE_OP_KEEP_FLAGS; op->op.flags |= FSCACHE_OP_ASYNC; op->op.processor = cachefiles_read_copier; - /* we assume the absence or presence of the first block is a good - * enough indication for the page as a whole - * - TODO: don't use bmap() for this as it is _not_ actually good - * enough for this as it doesn't indicate errors, but it's all we've - * got for the moment - */ - block0 = page->index; - block0 <<= shift; - - block = inode->i_mapping->a_ops->bmap(inode->i_mapping, block0); - _debug("%llx -> %llx", - (unsigned long long) block0, - (unsigned long long) block); - - if (block) { + if (inode->i_sb->s_type->fs_flags & FS_SUPPORTS_SEEK_HOLE) { + /* Use llseek */ + struct path path; + struct file *file; + loff_t addr; + path.mnt = cache->mnt; + path.dentry = object->backer; + file = dentry_open(&path, O_RDONLY, cache->cache_cred); + if (IS_ERR(file)) + goto enobufs; + addr = page->index; + addr <<= PAGE_SHIFT; + have_data = (addr == vfs_llseek(file, addr, SEEK_DATA)); + filp_close(file, NULL); + } else { + /* we assume the absence or presence of the first block is a good + * enough indication for the page as a whole + * - TODO: don't use bmap() for this as it is _not_ actually good + * enough for this as it doesn't indicate errors, but it's all we've + * got for the moment + */ + /* calculate the shift required to use bmap */ + unsigned shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; + sector_t block0, block; + + block0 = page->index; + block0 <<= shift; + + block = inode->i_mapping->a_ops->bmap(inode->i_mapping, block0); + _debug("%llx -> %llx", + (unsigned long long) block0, + (unsigned long long) block); + have_data = (block != 0); + } + if (have_data) { /* submit the apparently valid page to the backing fs to be * read from disk */ ret = cachefiles_read_backing_file_one(object, op, page); @@ -683,7 +698,7 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, struct pagevec pagevec; struct inode *inode; struct page *page, *_n; - unsigned shift, nrbackpages; + unsigned nrbackpages; int ret, ret2, space; object = container_of(op->op.object, @@ -704,11 +719,8 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, inode = object->backer->d_inode; ASSERT(S_ISREG(inode->i_mode)); - ASSERT(inode->i_mapping->a_ops->bmap); ASSERT(inode->i_mapping->a_ops->readpages); - /* calculate the shift required to use bmap */ - shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; pagevec_init(&pagevec, 0); @@ -721,24 +733,45 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, ret = space ? -ENODATA : -ENOBUFS; list_for_each_entry_safe(page, _n, pages, lru) { - sector_t block0, block; - - /* we assume the absence or presence of the first block is a - * good enough indication for the page as a whole - * - TODO: don't use bmap() for this as it is _not_ actually - * good enough for this as it doesn't indicate errors, but - * it's all we've got for the moment - */ - block0 = page->index; - block0 <<= shift; - - block = inode->i_mapping->a_ops->bmap(inode->i_mapping, - block0); - _debug("%llx -> %llx", - (unsigned long long) block0, - (unsigned long long) block); - - if (block) { + bool have_data; + + if (inode->i_sb->s_type->fs_flags & FS_SUPPORTS_SEEK_HOLE) { + /* Use llseek */ + struct path path; + struct file *file; + loff_t addr; + + path.mnt = cache->mnt; + path.dentry = object->backer; + file = dentry_open(&path, O_RDONLY, cache->cache_cred); + if (IS_ERR(file)) + goto all_enobufs; + addr = page->index; + addr <<= PAGE_SHIFT; + have_date = (addr == vfs_llseek(file, addr, SEEK_DATA)); + filp_close(file, NULL); + } else { + /* we assume the absence or presence of the first block is a + * good enough indication for the page as a whole + * - TODO: don't use bmap() for this as it is _not_ actually + * good enough for this as it doesn't indicate errors, but + * it's all we've got for the moment + */ + /* calculate the shift required to use bmap */ + unsigned shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; + sector_t block0, block; + + block0 = page->index; + block0 <<= shift; + + block = inode->i_mapping->a_ops->bmap(inode->i_mapping, + block0); + _debug("%llx -> %llx", + (unsigned long long) block0, + (unsigned long long) block); + have_data = (block != 0); + } + if (have_data) { /* we have data - add it to the list to give to the * backing fs */ list_move(&page->lru, &backpages); diff --git a/include/linux/fs.h b/include/linux/fs.h index f4131e8ead74..ae28d175eeb4 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1863,6 +1863,7 @@ struct file_system_type { #define FS_HAS_SUBTYPE 4 #define FS_USERNS_MOUNT 8 /* Can be mounted by userns root */ #define FS_USERNS_DEV_MOUNT 16 /* A userns mount does not imply MNT_NODEV */ +#define FS_SUPPORTS_SEEK_HOLE 32 /* SEEK_HOLE/SEEK_DATA reliably detect holes */ #define FS_RENAME_DOES_D_MOVE 32768 /* FS will handle d_move() during rename() internally. */ struct dentry *(*mount) (struct file_system_type *, int, const char *, void *); ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 5:27 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap NeilBrown @ 2015-04-20 6:08 ` Christoph Hellwig 2015-04-20 6:27 ` NeilBrown 0 siblings, 1 reply; 20+ messages in thread From: Christoph Hellwig @ 2015-04-20 6:08 UTC (permalink / raw) To: NeilBrown Cc: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs Please just usse SEEK_HOLE/DATA support unconditioanlly. ->bmap is a horrible hack that is completely unsafe. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 6:08 ` Christoph Hellwig @ 2015-04-20 6:27 ` NeilBrown 2015-04-20 9:45 ` Christoph Hellwig 0 siblings, 1 reply; 20+ messages in thread From: NeilBrown @ 2015-04-20 6:27 UTC (permalink / raw) To: Christoph Hellwig Cc: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 395 bytes --] On Sun, 19 Apr 2015 23:08:18 -0700 Christoph Hellwig <hch@infradead.org> wrote: > Please just usse SEEK_HOLE/DATA support unconditioanlly. ->bmap is a > horrible hack that is completely unsafe. > A worthwhile goal, but I certainly wouldn't consider pursuing it until what I have submitted so far as been accepted - let's not reject "good" while waiting for "perfect". NeilBrown [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 6:27 ` NeilBrown @ 2015-04-20 9:45 ` Christoph Hellwig 2015-04-21 23:06 ` NeilBrown 0 siblings, 1 reply; 20+ messages in thread From: Christoph Hellwig @ 2015-04-20 9:45 UTC (permalink / raw) To: NeilBrown Cc: Christoph Hellwig, David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs On Mon, Apr 20, 2015 at 04:27:00PM +1000, NeilBrown wrote: > A worthwhile goal, but I certainly wouldn't consider pursuing it until what I > have submitted so far as been accepted - let's not reject "good" while > waiting for "perfect". It's still broken. You add conditional flag for the almost right (almost because the flag in the filesystem type needs to go) while leaving the broken option th default. So what you propose here is not good, it's at best just as bad as the old version because you don't remove broken code but add a lot more clutter at the same time. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 9:45 ` Christoph Hellwig @ 2015-04-21 23:06 ` NeilBrown 0 siblings, 0 replies; 20+ messages in thread From: NeilBrown @ 2015-04-21 23:06 UTC (permalink / raw) To: Christoph Hellwig Cc: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1387 bytes --] On Mon, 20 Apr 2015 02:45:39 -0700 Christoph Hellwig <hch@infradead.org> wrote: > On Mon, Apr 20, 2015 at 04:27:00PM +1000, NeilBrown wrote: > > A worthwhile goal, but I certainly wouldn't consider pursuing it until what I > > have submitted so far as been accepted - let's not reject "good" while > > waiting for "perfect". > > It's still broken. You add conditional flag for the almost right > (almost because the flag in the filesystem type needs to go) Why does it have to go? I suspect you have a reason, but I can't read your mind. > while > leaving the broken option th default. You say it is broken, and yet people are using it and are having a degree of success. Surely the appropriate process is: - introduce a "better" option - examine each relevant filesystem and transition over to use the new option. - remove the "not so good" option. I'm still at step 1. > So what you propose here is not > good, it's at best just as bad as the old version because you don't > remove broken code but add a lot more clutter at the same time. What I propose is measurably better because it works with BTRFS now, and there seems to be a reasonable path towards making to generally better if someone cares enough to examine each filesystem. So I still claim you are pushing back against "good" because you want "perfect". NeilBrown [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/3] cachefiles: perform test on s_blocksize when opening cache file. 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown 2015-04-20 5:27 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap NeilBrown @ 2015-04-20 5:27 ` NeilBrown 2015-04-20 5:27 ` [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag NeilBrown ` (3 subsequent siblings) 5 siblings, 0 replies; 20+ messages in thread From: NeilBrown @ 2015-04-20 5:27 UTC (permalink / raw) To: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba Cc: linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs cachefiles requires that s_blocksize in the cache is not greater than PAGE_SIZE, and performs the check every time a block is accessed. Move the test to the place where the file is "opened", where other file-validity tests are performed. Signed-off-by: NeilBrown <neilb@suse.de> --- fs/cachefiles/namei.c | 2 ++ fs/cachefiles/rdwr.c | 6 ------ 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c index 1e51714eb33e..5404afcdee98 100644 --- a/fs/cachefiles/namei.c +++ b/fs/cachefiles/namei.c @@ -649,6 +649,8 @@ lookup_again: aops = object->dentry->d_inode->i_mapping->a_ops; if (!aops->bmap) goto check_error; + if (object->dentry->d_sb->s_blocksize > PAGE_SIZE) + goto check_error; object->backer = object->dentry; } else { diff --git a/fs/cachefiles/rdwr.c b/fs/cachefiles/rdwr.c index c6cd8d7a4eef..9bd44ff48cc0 100644 --- a/fs/cachefiles/rdwr.c +++ b/fs/cachefiles/rdwr.c @@ -414,9 +414,6 @@ int cachefiles_read_or_alloc_page(struct fscache_retrieval *op, ASSERT(inode->i_mapping->a_ops->readpages); /* calculate the shift required to use bmap */ - if (inode->i_sb->s_blocksize > PAGE_SIZE) - goto enobufs; - shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; op->op.flags &= FSCACHE_OP_KEEP_FLAGS; @@ -711,9 +708,6 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, ASSERT(inode->i_mapping->a_ops->readpages); /* calculate the shift required to use bmap */ - if (inode->i_sb->s_blocksize > PAGE_SIZE) - goto all_enobufs; - shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; pagevec_init(&pagevec, 0); ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown 2015-04-20 5:27 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap NeilBrown 2015-04-20 5:27 ` [PATCH 1/3] cachefiles: perform test on s_blocksize when opening cache file NeilBrown @ 2015-04-20 5:27 ` NeilBrown 2015-04-20 19:48 ` Chris Mason 2015-04-20 8:47 ` David Howells ` (2 subsequent siblings) 5 siblings, 1 reply; 20+ messages in thread From: NeilBrown @ 2015-04-20 5:27 UTC (permalink / raw) To: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba Cc: linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs This allows fscache to cachefiles in a btrfs filesystem. Signed-off-by: NeilBrown <neilb@suse.de> --- fs/btrfs/super.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 05fef198ff94..d3c5d2b40f8e 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1904,7 +1904,8 @@ static struct file_system_type btrfs_fs_type = { .name = "btrfs", .mount = btrfs_mount, .kill_sb = btrfs_kill_super, - .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA, + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | + FS_SUPPORTS_SEEK_HOLE, }; MODULE_ALIAS_FS("btrfs"); ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 5:27 ` [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag NeilBrown @ 2015-04-20 19:48 ` Chris Mason 0 siblings, 0 replies; 20+ messages in thread From: Chris Mason @ 2015-04-20 19:48 UTC (permalink / raw) To: NeilBrown, David Howells, Al Viro, Josef Bacik, David Sterba Cc: linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs On 04/20/2015 01:27 AM, NeilBrown wrote: > This allows fscache to cachefiles in a btrfs filesystem. Thanks for working on this Neil. Signed-off-by: Chris Mason <clm@fb.com> -chris > > Signed-off-by: NeilBrown <neilb@suse.de> > --- > fs/btrfs/super.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c > index 05fef198ff94..d3c5d2b40f8e 100644 > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1904,7 +1904,8 @@ static struct file_system_type btrfs_fs_type = { > .name = "btrfs", > .mount = btrfs_mount, > .kill_sb = btrfs_kill_super, > - .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA, > + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | > + FS_SUPPORTS_SEEK_HOLE, > }; > MODULE_ALIAS_FS("btrfs"); > > > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown ` (2 preceding siblings ...) 2015-04-20 5:27 ` [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag NeilBrown @ 2015-04-20 8:47 ` David Howells 2015-04-20 9:33 ` NeilBrown 2015-04-20 9:46 ` David Howells 2015-04-20 11:21 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap David Howells 2015-04-20 15:59 ` [RFC][PATCH] cachefiles: Make better use of SEEK_DATA/SEEK_HOLE David Howells 5 siblings, 2 replies; 20+ messages in thread From: David Howells @ 2015-04-20 8:47 UTC (permalink / raw) To: NeilBrown Cc: dhowells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs NeilBrown <neilb@suse.de> wrote: > + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | > + FS_SUPPORTS_SEEK_HOLE, I must be missing something: warthog>git merge linus/master Already up-to-date. warthog>stg id 09d51602cf84a1264946711dd4ea0dddbac599a1 warthog>grep -r FS_SUPPORTS_SEEK_HOLE include/ warthog1>git grep FS_SUPPORTS_SEEK_HOLE warthog1> David ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 8:47 ` David Howells @ 2015-04-20 9:33 ` NeilBrown 2015-04-20 9:46 ` David Howells 1 sibling, 0 replies; 20+ messages in thread From: NeilBrown @ 2015-04-20 9:33 UTC (permalink / raw) To: David Howells Cc: Chris Mason, Al Viro, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 545 bytes --] On Mon, 20 Apr 2015 09:47:42 +0100 David Howells <dhowells@redhat.com> wrote: > NeilBrown <neilb@suse.de> wrote: > > > + .fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | > > + FS_SUPPORTS_SEEK_HOLE, > > I must be missing something: > > warthog>git merge linus/master > Already up-to-date. > warthog>stg id > 09d51602cf84a1264946711dd4ea0dddbac599a1 > warthog>grep -r FS_SUPPORTS_SEEK_HOLE include/ > warthog1>git grep FS_SUPPORTS_SEEK_HOLE > warthog1> > Missing patch 2 of the 3-patch series? NeilBrown [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 8:47 ` David Howells 2015-04-20 9:33 ` NeilBrown @ 2015-04-20 9:46 ` David Howells 2015-04-20 9:48 ` Christoph Hellwig 1 sibling, 1 reply; 20+ messages in thread From: David Howells @ 2015-04-20 9:46 UTC (permalink / raw) To: NeilBrown Cc: dhowells, Chris Mason, Al Viro, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs NeilBrown <neilb@suse.de> wrote: > Missing patch 2 of the 3-patch series? Yes. :-) Do ext4 and xfs support this, do you know? David ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 9:46 ` David Howells @ 2015-04-20 9:48 ` Christoph Hellwig 2015-04-20 12:58 ` Al Viro 2015-04-27 5:41 ` NeilBrown 0 siblings, 2 replies; 20+ messages in thread From: Christoph Hellwig @ 2015-04-20 9:48 UTC (permalink / raw) To: David Howells Cc: NeilBrown, Chris Mason, Al Viro, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs On Mon, Apr 20, 2015 at 10:46:49AM +0100, David Howells wrote: > NeilBrown <neilb@suse.de> wrote: > > > Missing patch 2 of the 3-patch series? > > Yes. :-) > > Do ext4 and xfs support this, do you know? Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 > > David ---end quoted text--- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 9:48 ` Christoph Hellwig @ 2015-04-20 12:58 ` Al Viro 2015-04-21 8:43 ` Christoph Hellwig 2015-04-27 5:41 ` NeilBrown 1 sibling, 1 reply; 20+ messages in thread From: Al Viro @ 2015-04-20 12:58 UTC (permalink / raw) To: Christoph Hellwig Cc: David Howells, NeilBrown, Chris Mason, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs On Mon, Apr 20, 2015 at 02:48:55AM -0700, Christoph Hellwig wrote: > On Mon, Apr 20, 2015 at 10:46:49AM +0100, David Howells wrote: > > NeilBrown <neilb@suse.de> wrote: > > > > > Missing patch 2 of the 3-patch series? > > > > Yes. :-) > > > > Do ext4 and xfs support this, do you know? > > Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 Er... Nominally, gfs2 supports it. By treating all files as "there's a hole starting at EOF". Same as ext2 or even minix... ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 12:58 ` Al Viro @ 2015-04-21 8:43 ` Christoph Hellwig 2015-04-21 10:23 ` Hugh Dickins 0 siblings, 1 reply; 20+ messages in thread From: Christoph Hellwig @ 2015-04-21 8:43 UTC (permalink / raw) To: Al Viro Cc: Christoph Hellwig, David Howells, NeilBrown, Chris Mason, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs On Mon, Apr 20, 2015 at 01:58:35PM +0100, Al Viro wrote: > > > Do ext4 and xfs support this, do you know? > > > > Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 > > Er... Nominally, gfs2 supports it. By treating all files as "there's a > hole starting at EOF". Same as ext2 or even minix... Yeah remove gfs2 from the list. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-21 8:43 ` Christoph Hellwig @ 2015-04-21 10:23 ` Hugh Dickins 0 siblings, 0 replies; 20+ messages in thread From: Hugh Dickins @ 2015-04-21 10:23 UTC (permalink / raw) To: NeilBrown Cc: Christoph Hellwig, Al Viro, David Howells, Chris Mason, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs On Tue, 21 Apr 2015, Christoph Hellwig wrote: > On Mon, Apr 20, 2015 at 01:58:35PM +0100, Al Viro wrote: > > > > Do ext4 and xfs support this, do you know? > > > > > > Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 > > > > Er... Nominally, gfs2 supports it. By treating all files as "there's a > > hole starting at EOF". Same as ext2 or even minix... > > Yeah remove gfs2 from the list. But please add tmpfs to the list - though I doubt you're interested in caching its files. Hugh ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-20 9:48 ` Christoph Hellwig 2015-04-20 12:58 ` Al Viro @ 2015-04-27 5:41 ` NeilBrown 2015-04-27 13:43 ` Christoph Hellwig 1 sibling, 1 reply; 20+ messages in thread From: NeilBrown @ 2015-04-27 5:41 UTC (permalink / raw) To: Christoph Hellwig Cc: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1071 bytes --] On Mon, 20 Apr 2015 02:48:55 -0700 Christoph Hellwig <hch@infradead.org> wrote: > On Mon, Apr 20, 2015 at 10:46:49AM +0100, David Howells wrote: > > NeilBrown <neilb@suse.de> wrote: > > > > > Missing patch 2 of the 3-patch series? > > > > Yes. :-) > > > > Do ext4 and xfs support this, do you know? > > Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 Are you sure about NFSv4.2? I see that it *can* report holes, but is there any guarantee that if you create a new file and write only the 5th block, then READ_PLUS will reliably report that the first 4 block are holes?? Because if it doesn't guarantee that, then NFSv4.2 doesn't fit the with the others where SEEK_HOLE reliable reports holes. On the other hand if NFSv4.2 *does* guarantee that then the current READ_PLUS server patches are broken because they just use vfs_llseek and assume that trust what it says. It would be really nice if SEEK_{DATA,HOLE} either reported holes reliably or returned ENXIO, but I guess there was a goo reason not to do that. Thanks, NeilBrown [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag. 2015-04-27 5:41 ` NeilBrown @ 2015-04-27 13:43 ` Christoph Hellwig 0 siblings, 0 replies; 20+ messages in thread From: Christoph Hellwig @ 2015-04-27 13:43 UTC (permalink / raw) To: NeilBrown Cc: David Howells, Chris Mason, Al Viro, Josef Bacik, David Sterba, Dave Chinner, linux-kernel, linux-fsdevel, linux-btrfs On Mon, Apr 27, 2015 at 03:41:33PM +1000, NeilBrown wrote: > Are you sure about NFSv4.2? > > I see that it *can* report holes, but is there any guarantee that if you > create a new file and write only the 5th block, then READ_PLUS will reliably > report that the first 4 block are holes?? FYI, I'm talking about SEEK here, not READ_PLUS but the issue is the same. > Because if it doesn't guarantee that, then NFSv4.2 doesn't fit the with the > others where SEEK_HOLE reliable reports holes. > On the other hand if NFSv4.2 *does* guarantee that then the current READ_PLUS > server patches are broken because they just use vfs_llseek and assume that > trust what it says. There is no quality of implementation guarantee in NFS, just like there isn't any in Linux. It's very hard to have any hard guarantees without leaking specific implementation details like a block size. > It would be really nice if SEEK_{DATA,HOLE} either reported holes reliably or > returned ENXIO, but I guess there was a goo reason not to do that. It would hav been useful, but we went with the Solaris way of reporting a giant hole. Solaris at least has a pathconf value telling you if real SEEK_{DATA,HOLE} are supported, but with Linus' hatred of that syscall we only have a bad emulation in glibc that isn't of much help here. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap. 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown ` (3 preceding siblings ...) 2015-04-20 8:47 ` David Howells @ 2015-04-20 11:21 ` David Howells 2015-04-20 15:59 ` [RFC][PATCH] cachefiles: Make better use of SEEK_DATA/SEEK_HOLE David Howells 5 siblings, 0 replies; 20+ messages in thread From: David Howells @ 2015-04-20 11:21 UTC (permalink / raw) To: NeilBrown Cc: dhowells, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs NeilBrown <neilb@suse.de> wrote: > @@ -721,24 +733,45 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, We can actually do better than this what you've done here for cachefiles_read_or_alloc_pages(). We can use SEEK_DATA to check the beginning of a run of pages and then SEEK_HOLE to see how many pages are covered. David ^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC][PATCH] cachefiles: Make better use of SEEK_DATA/SEEK_HOLE 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown ` (4 preceding siblings ...) 2015-04-20 11:21 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap David Howells @ 2015-04-20 15:59 ` David Howells 5 siblings, 0 replies; 20+ messages in thread From: David Howells @ 2015-04-20 15:59 UTC (permalink / raw) Cc: dhowells, NeilBrown, Chris Mason, Al Viro, Josef Bacik, David Sterba, linux-cachefs, Dave Chinner, linux-kernel, Christoph Hellwig, linux-fsdevel, linux-btrfs Here's a test patch that makes better use of SEEK_DATA/SEEK_HOLE in cachefiles_read_or_alloc_pages() by caching data/hole information to use on the subsequent pages in the list. Note that the pages list needs to be transited in reverse for this to work as it seems that the list passed to the fs is reverse-sorted. What I see in the kernel output when reading a 16MB file is: [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: 0 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: 200000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: 400000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: 600000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: 800000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: a00000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: c00000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_pages({OBJ5,1},,512,,) [cat ] SEEK_DATA: e00000 [cat ] SEEK_HOLE: 1000000 [cat ] <== cachefiles_read_or_alloc_pages() = 0 [nr=0 empty] [cat ] ==> cachefiles_read_or_alloc_page({ffff880037ffac00},{1000},,,) [cat ] <== cachefiles_read_or_alloc_page() = -61 It doesn't quite work, though - after invalidating the file, I see lots of: [cat ] SEEK_DATA: fffffffffffffffa That is ENXIO. I'm not sure what that portends yet. David --- diff --git a/fs/cachefiles/bind.c b/fs/cachefiles/bind.c index fbb08e97438d..17612f09d4b3 100644 --- a/fs/cachefiles/bind.c +++ b/fs/cachefiles/bind.c @@ -129,7 +129,8 @@ static int cachefiles_daemon_add_cache(struct cachefiles_cache *cache) !root->d_inode->i_op->setxattr || !root->d_inode->i_op->getxattr || !root->d_sb->s_op->statfs || - !root->d_sb->s_op->sync_fs) + !root->d_sb->s_op->sync_fs || + !root->d_sb->s_type->fs_flags & FS_SUPPORTS_SEEK_HOLE) goto error_unsupported; ret = -EROFS; diff --git a/fs/cachefiles/rdwr.c b/fs/cachefiles/rdwr.c index 7c7cbfae7b19..375588be715c 100644 --- a/fs/cachefiles/rdwr.c +++ b/fs/cachefiles/rdwr.c @@ -402,7 +402,7 @@ int cachefiles_read_or_alloc_page(struct fscache_retrieval *op, cache = container_of(object->fscache.cache, struct cachefiles_cache, cache); - _enter("{%p},{%lx},,,", object, page->index); + kenter("{%p},{%lx},,,", object, page->index); if (!object->backer) goto enobufs; @@ -462,12 +462,12 @@ int cachefiles_read_or_alloc_page(struct fscache_retrieval *op, goto enobufs; } - _leave(" = %d", ret); + kleave(" = %d", ret); return ret; enobufs: fscache_retrieval_complete(op, 1); - _leave(" = -ENOBUFS"); + kleave(" = -ENOBUFS"); return -ENOBUFS; } @@ -698,7 +698,10 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, struct pagevec pagevec; struct inode *inode; struct page *page, *_n; + struct file *file; + struct path path; unsigned nrbackpages; + loff_t pre_hole, from, to; int ret, ret2, space; object = container_of(op->op.object, @@ -706,7 +709,7 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, cache = container_of(object->fscache.cache, struct cachefiles_cache, cache); - _enter("{OBJ%x,%d},,%d,,", + kenter("{OBJ%x,%d},,%d,,", object->fscache.debug_id, atomic_read(&op->op.usage), *nr_pages); @@ -731,47 +734,34 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, INIT_LIST_HEAD(&backpages); nrbackpages = 0; + path.mnt = cache->mnt; + path.dentry = object->backer; + file = dentry_open(&path, O_RDWR, cache->cache_cred); + if (IS_ERR(file)) + goto all_enobufs; + pre_hole = from = to = 0; + ret = space ? -ENODATA : -ENOBUFS; - list_for_each_entry_safe(page, _n, pages, lru) { - bool have_data; - - if (inode->i_sb->s_type->fs_flags & FS_SUPPORTS_SEEK_HOLE) { - /* Use llseek */ - struct path path; - struct file *file; - loff_t addr; - - path.mnt = cache->mnt; - path.dentry = object->backer; - file = dentry_open(&path, O_RDONLY, cache->cache_cred); - if (IS_ERR(file)) + list_for_each_entry_safe_reverse(page, _n, pages, lru) { + loff_t addr; + + /* Determine whether the page is present */ + addr = page->index; + addr <<= PAGE_SHIFT; + if (addr < pre_hole || addr >= to) { + pre_hole = addr; + from = vfs_llseek(file, pre_hole, SEEK_DATA); + kdebug("SEEK_DATA: %llx", from); + if (IS_ERR_VALUE(from)) + goto all_enobufs; + + to = vfs_llseek(file, from, SEEK_HOLE); + kdebug("SEEK_HOLE: %llx", to); + if (IS_ERR_VALUE(to)) goto all_enobufs; - addr = page->index; - addr <<= PAGE_SHIFT; - have_date = (addr == vfs_llseek(file, addr, SEEK_DATA)); - filp_close(file, NULL); - } else { - /* we assume the absence or presence of the first block is a - * good enough indication for the page as a whole - * - TODO: don't use bmap() for this as it is _not_ actually - * good enough for this as it doesn't indicate errors, but - * it's all we've got for the moment - */ - /* calculate the shift required to use bmap */ - unsigned shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits; - sector_t block0, block; - - block0 = page->index; - block0 <<= shift; - - block = inode->i_mapping->a_ops->bmap(inode->i_mapping, - block0); - _debug("%llx -> %llx", - (unsigned long long) block0, - (unsigned long long) block); - have_data = (block != 0); } - if (have_data) { + + if (addr >= from && addr <= to - PAGE_SIZE) { /* we have data - add it to the list to give to the * backing fs */ list_move(&page->lru, &backpages); @@ -786,6 +776,8 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, } } + filp_close(file, NULL); + if (pagevec_count(&pagevec) > 0) fscache_mark_pages_cached(op, &pagevec); @@ -800,12 +792,13 @@ int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op, ret = ret2; } - _leave(" = %d [nr=%u%s]", + kleave(" = %d [nr=%u%s]", ret, *nr_pages, list_empty(pages) ? " empty" : ""); return ret; all_enobufs: fscache_retrieval_complete(op, *nr_pages); + kleave(" = -ENOBUFS [all]"); return -ENOBUFS; } diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 821f22dbe825..686bd0db1d80 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -89,7 +89,7 @@ static struct file_system_type ext2_fs_type = { .name = "ext2", .mount = ext4_mount, .kill_sb = kill_block_super, - .fs_flags = FS_REQUIRES_DEV, + .fs_flags = FS_REQUIRES_DEV | FS_SUPPORTS_SEEK_HOLE, }; MODULE_ALIAS_FS("ext2"); MODULE_ALIAS("ext2"); ^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2015-04-27 13:44 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-04-20 5:27 [PATCH 0/3] Allow fscache to work on BTRFS NeilBrown 2015-04-20 5:27 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap NeilBrown 2015-04-20 6:08 ` Christoph Hellwig 2015-04-20 6:27 ` NeilBrown 2015-04-20 9:45 ` Christoph Hellwig 2015-04-21 23:06 ` NeilBrown 2015-04-20 5:27 ` [PATCH 1/3] cachefiles: perform test on s_blocksize when opening cache file NeilBrown 2015-04-20 5:27 ` [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag NeilBrown 2015-04-20 19:48 ` Chris Mason 2015-04-20 8:47 ` David Howells 2015-04-20 9:33 ` NeilBrown 2015-04-20 9:46 ` David Howells 2015-04-20 9:48 ` Christoph Hellwig 2015-04-20 12:58 ` Al Viro 2015-04-21 8:43 ` Christoph Hellwig 2015-04-21 10:23 ` Hugh Dickins 2015-04-27 5:41 ` NeilBrown 2015-04-27 13:43 ` Christoph Hellwig 2015-04-20 11:21 ` [PATCH 2/3] fscache/cachefiles: optionally use SEEK_DATA instead of ->bmap David Howells 2015-04-20 15:59 ` [RFC][PATCH] cachefiles: Make better use of SEEK_DATA/SEEK_HOLE David Howells
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).