From: David Wysochanski <dwysocha@redhat.com>
To: David Howells <dhowells@redhat.com>
Cc: Jeff Layton <jlayton@kernel.org>,
ceph-devel@vger.kernel.org, linux-cachefs@redhat.com,
idryomov@gmail.com
Subject: Re: [Linux-cachefs] [RFC PATCH v2 09/11] ceph: convert readpages to fscache_read_helper
Date: Mon, 10 Aug 2020 09:50:44 -0400 [thread overview]
Message-ID: <CALF+zO=K8iE5y7_5MPS4Zg+stUmY4FQobop1DnsS71Dpn_YpOg@mail.gmail.com> (raw)
In-Reply-To: <526038.1597054155@warthog.procyon.org.uk>
On Mon, Aug 10, 2020 at 6:09 AM David Howells <dhowells@redhat.com> wrote:
>
> David Wysochanski <dwysocha@redhat.com> wrote:
>
> > Looks like fscache_shape_request() overrides any 'max_pages' value (actually
> > it is cachefiles_shape_request) , so it's unclear why the netfs would pass
> > in a 'max_pages' if it is not honored - seems like a bug maybe or it's not
> > obvious
>
> I think the problem is that cachefiles_shape_request() is applying the limit
> too early. It's using it to cut down the number of pages in the original
> request (only applicable to readpages), but then the shaping to fit cache
> granules can exceed that, so it needs to be applied later also.
>
> Does the attached patch help?
>
> David
> ---
> diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
> index 2bfba2e41c39..ce05cf1d9a6e 100644
> --- a/fs/cachefiles/content-map.c
> +++ b/fs/cachefiles/content-map.c
> @@ -134,7 +134,8 @@ void cachefiles_shape_request(struct fscache_object *obj,
> _enter("{%lx,%lx,%x},%llx,%d",
> start, end, max_pages, i_size, shape->for_write);
>
> - if (start >= CACHEFILES_SIZE_LIMIT / PAGE_SIZE) {
> + if (start >= CACHEFILES_SIZE_LIMIT / PAGE_SIZE ||
> + max_pages < CACHEFILES_GRAN_PAGES) {
> shape->to_be_done = FSCACHE_READ_FROM_SERVER;
> return;
> }
> @@ -144,10 +145,6 @@ void cachefiles_shape_request(struct fscache_object *obj,
> if (shape->i_size > CACHEFILES_SIZE_LIMIT)
> i_size = CACHEFILES_SIZE_LIMIT;
>
> - max_pages = round_down(max_pages, CACHEFILES_GRAN_PAGES);
> - if (end - start > max_pages)
> - end = start + max_pages;
> -
> granule = start / CACHEFILES_GRAN_PAGES;
> if (granule / 8 >= object->content_map_size) {
> cachefiles_expand_content_map(object, i_size);
> @@ -185,6 +182,10 @@ void cachefiles_shape_request(struct fscache_object *obj,
> start = round_down(start, CACHEFILES_GRAN_PAGES);
> end = round_up(end, CACHEFILES_GRAN_PAGES);
>
> + /* Trim to the maximum size the netfs supports */
> + if (end - start > max_pages)
> + end = round_down(start + max_pages, CACHEFILES_GRAN_PAGES);
> +
> /* But trim to the end of the file and the starting page */
> eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
> if (eof <= shape->proposed_start)
>
I tried this and got the same panic - I think i_size is the culprit
(it is larger than max_pages). I'll send you a larger trace offline
with cachefiles/fscache debugging enabled if that helps, but below is
some custom tracing that may be enough because it shows before / after
shaping values.
Here's outline of the test (smaller rsize and readahead for simplicity):
# ./t1_rsize_lt_read.sh 4.1
Setting NFS vers=4.1
Mon 10 Aug 2020 09:34:18 AM EDT: 1. On NFS client, install and enable
cachefilesd
Mon 10 Aug 2020 09:34:18 AM EDT: 2. On NFS client, mount -o
vers=4.1,fsc,rsize=8192 127.0.0.1:/export/dir1 /mnt/dir1
Mon 10 Aug 2020 09:34:18 AM EDT: 3. On NFS client, dd if=/dev/zero
of=/mnt/dir1/file1.bin bs=16384 count=1
Mon 10 Aug 2020 09:34:18 AM EDT: 4. On NFS client, echo 3 >
/proc/sys/vm/drop_caches
Mon 10 Aug 2020 09:34:19 AM EDT: 5. On NFS client, ./nfs-readahead.sh
set /mnt/dir1 16384
Mon 10 Aug 2020 09:34:19 AM EDT: 6. On NFS client, dd
if=/mnt/dir1/file1.bin of=/dev/null
Mon 10 Aug 2020 09:34:19 AM EDT: 7. On NFS client, echo 3 >
/proc/sys/vm/drop_caches
Mon 10 Aug 2020 09:34:19 AM EDT: 8. On NFS client, dd
if=/mnt/dir1/file1.bin of=/dev/null
Console with custom nfs tracing
[ 62.955355] t1_rsize_lt_rea (4840): drop_caches: 3
[ 63.028786] fs/nfs/fscache.c:480 before read_helper_page_list pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 pages
ffffb4b4c0fafca8 max_pages 2
[ 63.028804] fs/fscache/read_helper.c:347 pid 4882
fscache_read_helper before shape req ffff8902f50b5800 req->nr_pages 0
shape.actual_nr_pages 48 shape.proposed_nr_pages 4
[ 63.037231] fs/fscache/read_helper.c:353 pid 4882
fscache_read_helper after shape req ffff8902f50b5800 req->nr_pages 0
shape.actual_nr_pages 4 shape.proposed_nr_pages 4
[ 63.043421] fs/fscache/read_helper.c:531 pid 4882
fscache_read_helper before while req ffff8902f50b5800 req->nr_pages 1
shape.actual_nr_pages 4 shape.proposed_nr_pages 4
[ 63.049498] fs/fscache/read_helper.c:531 pid 4882
fscache_read_helper before while req ffff8902f50b5800 req->nr_pages 2
shape.actual_nr_pages 4 shape.proposed_nr_pages 4
[ 63.063708] fs/fscache/read_helper.c:531 pid 4882
fscache_read_helper before while req ffff8902f50b5800 req->nr_pages 3
shape.actual_nr_pages 4 shape.proposed_nr_pages 4
[ 63.070114] fs/fscache/read_helper.c:531 pid 4882
fscache_read_helper before while req ffff8902f50b5800 req->nr_pages 4
shape.actual_nr_pages 4 shape.proposed_nr_pages 4
[ 63.076438] fs/nfs/fscache.c:369 enter nfs_issue_op pid 4882 inode
ffff8902b4a2a828 cache ffff8902f50b5800 start 0 last 3
[ 63.082964] fs/nfs/fscache.c:379 before readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f08741a00
[ 63.087591] fs/nfs/fscache.c:382 after readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f08741a00 cache.error 0
[ 63.093058] fs/nfs/fscache.c:379 before readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f08288680
[ 63.098927] fs/nfs/fscache.c:382 after readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f08288680 cache.error 0
[ 63.104507] fs/nfs/fscache.c:379 before readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f082816c0
[ 63.110922] fs/nfs/fscache.c:382 after readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f082816c0 cache.error 0
[ 63.111973] fs/nfs/fscache.c:523 pid 233 before io_done inode
ffff8902b4a2a828 bytes 8192 &req->cache ffff8902f50b5800 cache.pos 0
cache.len 16384
[ 63.115407] fs/nfs/fscache.c:379 before readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f067e8f40
[ 63.126337] fs/nfs/fscache.c:382 after readpage_async_filler pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 page
fffff42f067e8f40 cache.error 0
[ 63.131411] fs/nfs/fscache.c:388 exit nfs_issue_op pid 4882 inode
ffff8902b4a2a828 cache ffff8902f50b5800
[ 63.131955] fs/nfs/fscache.c:523 pid 233 before io_done inode
ffff8902b4a2a828 bytes 8192 &req->cache ffff8902f50b5800 cache.pos 0
cache.len 16384
[ 63.137012] fs/nfs/fscache.c:484 after read_helper_page_list pid
4882 inode ffff8902b4a2a828 cache ffff8902f50b5800 cache.pos 0
cache.len 16384 cache.nr_pages 4 pages ffffb4b4c0fafca8 ret 0
[ 63.140922] page:fffff42f08741a00 refcount:2 mapcount:0
mapping:00000000727f3adc index:0x0
[ 63.141091] mapping->aops:nfs_file_aops [nfs] dentry name:"file1.bin"
[ 63.146475] fs/nfs/fscache.c:490 outside while(!list_empty(pages))
read_helper_page_list pid 4882 inode ffff8902b4a2a828 cache
ffff8902f50b5800 cache.pos 0 cache.len 16384 cache.nr_pages 4
[ 63.146740] fs/fscache/read_helper.c:347 pid 4882
fscache_read_helper before shape req ffff8902f50b5800 req->nr_pages 0
shape.actual_nr_pages 3227832042 shape.proposed_nr_pages 1
[ 63.153662] flags: 0x17ffffc0000006(referenced|uptodate)
[ 63.153699] raw: 0017ffffc0000006 dead000000000100 dead000000000122
ffff8902b4a2a9a0
[ 63.168174] fs/fscache/read_helper.c:353 pid 4882
fscache_read_helper after shape req ffff8902f50b5800 req->nr_pages 0
shape.actual_nr_pages 5 shape.proposed_nr_pages 1
[ 63.193131] raw: 0000000000000000 0000000000000000 00000001ffffffff
ffff8902ecfe8000
[ 63.203785] page dumped because: VM_BUG_ON_PAGE(!PageLocked(page))
[ 63.206372] page->mem_cgroup:ffff8902ecfe8000
[ 63.208333] ------------[ cut here ]------------
[ 63.211081] kernel BUG at mm/filemap.c:1290!
[ 63.213152] invalid opcode: 0000 [#1] SMP PTI
next prev parent reply other threads:[~2020-08-10 13:51 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-31 13:04 [RFC PATCH v2 00/11] ceph: convert to new FSCache API Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 01/11] ceph: break out writeback of incompatible snap context to separate function Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 02/11] ceph: don't call ceph_update_writeable_page from page_mkwrite Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 03/11] ceph: fold ceph_sync_readpages into ceph_readpage Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 04/11] ceph: fold ceph_sync_writepages into writepage_nounlock Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 05/11] ceph: fold ceph_update_writeable_page into ceph_write_begin Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 06/11] ceph: conversion to new fscache API Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 07/11] ceph: convert readpage to fscache read helper Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 08/11] ceph: plug write_begin into " Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 09/11] ceph: convert readpages to fscache_read_helper Jeff Layton
2020-08-09 15:09 ` [Linux-cachefs] " David Wysochanski
2020-08-10 11:09 ` Jeff Layton
2020-08-10 12:24 ` David Wysochanski
2020-08-09 18:06 ` David Wysochanski
2020-08-10 10:09 ` David Howells
2020-08-10 13:50 ` David Wysochanski [this message]
2020-08-10 18:55 ` David Wysochanski
2020-07-31 13:04 ` [RFC PATCH v2 10/11] ceph: add fscache writeback support Jeff Layton
2020-07-31 13:04 ` [RFC PATCH v2 11/11] ceph: re-enable fscache support Jeff Layton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CALF+zO=K8iE5y7_5MPS4Zg+stUmY4FQobop1DnsS71Dpn_YpOg@mail.gmail.com' \
--to=dwysocha@redhat.com \
--cc=ceph-devel@vger.kernel.org \
--cc=dhowells@redhat.com \
--cc=idryomov@gmail.com \
--cc=jlayton@kernel.org \
--cc=linux-cachefs@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.