* [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter
@ 2020-05-04 17:07 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:07 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: Matthew Wilcox, dhowells-H+wXaHxf7aLQT0dZR+AlfA
Here's a set of patches that massively overhauls the object lifecycle
management and the I/O API of the local caching for network filesystems code
for a reduction of about 3000 LoC and a 1000 line Documentation reduction.
The ability to use async DIO to pass data to/from the cache gives a huge speed
bonus and uses less memory.
So far, only AFS has been ported to use this. This series disabled fscache
support in all other filesystems that use it. I'm working on NFS, but it's
tricky and I could use some help.
The following parts have been removed:
- The object state machine
- The I/O operation manager
- All non-transient references from fscache to the netfs's data
- All non-transient callbacks from fscache to the netfs
- The backing page I/O monitoring
- The tracking of netfs pages that fscache knows about
- The tracking of netfs pages that need writing to the cache
- The use of bmap to work out if a page is stored in the cache
- The copy of data to/from backing pages to netfs pages.
Instead, the I/O to the cache is driven much more from the netfs. There are a
number of aspects to the I/O API:
(1) The lowest level I/O primitives just take an iov_iter and start async
DIO on the cache objects. The caller gets a callback upon completion.
The PG_fscache bit is now just used to indicate that there's a write
to the cache in progress. The cache will keep track in xattrs as to
what areas of the backing file are occupied.
- struct fscache_io_request
- fscache_read() and fscache_write().
(2) The cache may require that the I/O fulfil certain alignment and length
granularity constraints. This, and whether the cache contains the
desired data, can be queried.
- struct fscache_extent
- fscache_shape_extent()
(3) The cookie that's obtained when an inode is set up must be 'used' when a
file is opened (with an indication as to whether it might be modified)
and 'unused' when it is done with. At the point of unuse, the auxdata
and file size can be specified.
- fscache_use_cookie(), fscache_unuse_cookie()
(4) The cookie can be invalidated at any time, and new auxiliary data and a
new size provided. Any in-progress I/O will either cause new I/O to
wait, or a replacement tmpfile will be created and the in-progress I/O
will just be abandoned. The on-disk auxdata (in xattrs, say) are updated
lazily.
- fscache_invalidate()
(5) A helper API for reads is provided to combine the (1), (2) above and
read_cache_pages() and also do read-(re)issue to the network in the case
that the data isn't present or the cache fails. This requires that an
operation descriptor be allocated and given some operations. This needs
to be used for ->readpage(), ->readpages() and prefetching for
->write_begin().
- struct fscache_io_request
- fscache_read_helper()
I've also simplified the cookie management API to remove struct
fscache_cookie_def. Instead, the pertinent details are supplied when a cookie
is created and the file size, key and auxdata are stored in the cookie.
Callbacks and backpointers are simply removed.
I've added some pieces outside of the API also:
(1) An inode flag to mark a backing cachefile as being in use by the kernel.
This prevents multiple caches mounted in the same directory from fighting
over the same files. It can also be extended to exclude other kernel
users (such as swap) and could also be used to prevent userspace
interfering with the file.
(2) A new I/O iterator class, ITER_MAPPING, that iterates over the specified
byte range of a mapping. The caller is required to make sure that the
pages don't evaporate under the callee (eg. pinning them by PG_locked,
PG_writeback, PG_fscache or usage count).
It may make sense to make this more generic and just take an xarray that
is known to be laden with pages. The one tricky bit there is that the
iteration routines call find_get_page_contig() - though the only thing
the mapping is used for is to find the xarray containing the pages.
This is better than an ITER_BVEC as no allocation of bio_vec structs is
required since the xarray holds pointers to all the pages involved.
(3) Wait and unlock functions for PG_fscache. These are in the core, so no
need to call into fscache for it.
So, to be done hopefully before the next merge window:
(1) Handling CONFIG_FSCACHE=n. fscache_read_helper() is a bit intrusive,
since it acts as the middle manager for switching between reading from
the cache and reading from the network. I need a no-fscache version as
well.
Possibly a chunk of this can/will get absorbed into the VM, since the
request shaping conflicts with the readahead code. Especially with
Matthew Wilcox intending to rewrite the VM interface.
(2) Need to investigate whether I need to be lazy, but only to an extent,
about closing up backing files.
(3) Make NFS, CIFS, Ceph, 9P work with it. I'm hoping that Jeff Layton will
do Ceph. As mentioned, I'm having a crack at NFS, but it's evolved a bit
since I last looked at it and it might be easier if I can palm that off
to someone more current in the NFS I/O code.
And beyond that:
(4) Put in support for versioned monolithic objects (eg. AFS directories).
(5) Currently it cachefiles only caches large files up to 1GiB. File data
beyond that isn't cached. The problem here is that I'm using an xattr to
hold the content map, and xattrs may be limited in size and I've limited
myself to using a 512 byte xattr. I can easily make it cache a
non-sparse file of any size with no map, but as soon as it becomes
sparse, I need a different strategy.
(6) Change the indexing strategy so that the culling mechanism is brought
into the kernel, rather than doing that in userspace, and use an index
table of files with a LRU list.
These patches can be found also on:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=fscache-iter
David
---
David Howells (61):
afs: Make afs_zap_data() static
iov_iter: Add ITER_MAPPING
vm: Add wait/unlock functions for PG_fscache
vfs: Export rw_verify_area() for use by cachefiles
vfs: Provide S_CACHE_FILE inode flag
afs: Disable use of the fscache I/O routines
fscache: Add a cookie debug ID and use that in traces
fscache: Procfile to display cookies
fscache: Temporarily disable network filesystems' use of fscache
fscache: Remove the old I/O API
fscache: Remove the netfs data from the cookie
fscache: Remove struct fscache_cookie_def
fscache: Remove store_limit* from struct fscache_object
fscache: Remove fscache_check_consistency()
fscache: Remove fscache_attr_changed()
fscache: Remove obsolete stats
fscache: Remove old I/O tracepoints
fscache: Temporarily disable fscache_invalidate()
fscache: Remove the I/O operation manager
cachefiles: Remove tree of active files and use S_CACHE_FILE inode flag
fscache: Provide a simple thread pool for running ops asynchronously
fscache: Replace the object management state machine
fscache: Rewrite the I/O API based on iov_iter
fscache: Remove fscache_wait_on_invalidate()
fscache: Keep track of size of a file last set independently on the server
fscache, cachefiles: Fix disabled histogram warnings
fscache: Recast assertion in terms of cookie not being an index
cachefiles: Remove some redundant checks on unsigned values
cachefiles: trace: Log coherency checks
cachefiles: Split cachefiles_drop_object() up a bit
cachefiles: Implement new fscache I/O backend API
cachefiles: Merge object->backer into object->dentry
cachefiles: Implement a content-present indicator and bitmap
cachefiles: Implement extent shaper
cachefiles: Round the cachefile size up to DIO block size
cachefiles: Implement read and write parts of new I/O API
cachefiles: Add I/O tracepoints
fscache: Add read helper
fscache: Display cache-specific data in /proc/fs/fscache/objects
fscache: Remove more obsolete stats
fscache: New stats
fscache, cachefiles: Rewrite invalidation
fscache: Implement "will_modify" parameter on fscache_use_cookie()
fscache: Provide resize operation
fscache: Remove the update operation
cachefiles: Shape write requests
afs: Remove afs_zero_fid as it's not used
afs: Move key to afs_read struct
afs: Don't truncate iter during data fetch
afs: Set up the iov_iter before calling afs_extract_data()
afs: Use ITER_MAPPING for writing
afs: Interpose struct fscache_io_request into struct afs_read
afs: Note the amount transferred in fetch-data delivery
afs: Wait on PG_fscache before modifying/releasing a page
afs: Use new fscache I/O API
afs: Copy local writes to the cache when writing to the server
afs: Invoke fscache_resize_cookie() when handling ATTR_SIZE for setattr
fscache: Rewrite the main document
fscache: Remove the obsolete API bits from the documentation
fscache: Document the new netfs API
fscache: Document the rewritten cache backend API
Documentation/filesystems/caching/backend-api.txt | 520 ++-------
Documentation/filesystems/caching/fscache.txt | 51 -
Documentation/filesystems/caching/netfs-api.txt | 926 ++++++----------
Documentation/filesystems/caching/object.txt | 240 ----
Documentation/filesystems/caching/operations.txt | 213 ----
fs/9p/Kconfig | 2
fs/afs/cache.c | 54 -
fs/afs/cell.c | 9
fs/afs/dir.c | 242 +++-
fs/afs/file.c | 558 ++++-----
fs/afs/fsclient.c | 127 +-
fs/afs/inode.c | 45 -
fs/afs/internal.h | 64 +
fs/afs/rxrpc.c | 112 --
fs/afs/volume.c | 9
fs/afs/write.c | 306 +++--
fs/afs/yfsclient.c | 99 +-
fs/cachefiles/Makefile | 3
fs/cachefiles/bind.c | 11
fs/cachefiles/content-map.c | 489 ++++++++
fs/cachefiles/daemon.c | 10
fs/cachefiles/interface.c | 558 +++++----
fs/cachefiles/internal.h | 140 +-
fs/cachefiles/io.c | 279 +++++
fs/cachefiles/main.c | 12
fs/cachefiles/namei.c | 506 +++------
fs/cachefiles/rdwr.c | 974 ----------------
fs/cachefiles/xattr.c | 263 ++--
fs/ceph/Kconfig | 2
fs/cifs/Kconfig | 2
fs/fscache/Kconfig | 4
fs/fscache/Makefile | 8
fs/fscache/cache.c | 136 +-
fs/fscache/cookie.c | 768 +++++--------
fs/fscache/dispatcher.c | 150 +++
fs/fscache/fsdef.c | 56 -
fs/fscache/histogram.c | 2
fs/fscache/internal.h | 254 +---
fs/fscache/io.c | 204 +++
fs/fscache/main.c | 34 -
fs/fscache/netfs.c | 10
fs/fscache/obj.c | 343 ++++++
fs/fscache/object-list.c | 129 --
fs/fscache/object.c | 1133 -------------------
fs/fscache/object_bits.c | 120 ++
fs/fscache/operation.c | 633 -----------
fs/fscache/page.c | 1248 ---------------------
fs/fscache/proc.c | 13
fs/fscache/read_helper.c | 580 ++++++++++
fs/fscache/stats.c | 251 +---
fs/internal.h | 5
fs/nfs/Kconfig | 2
fs/nfs/fscache-index.c | 4
fs/read_write.c | 1
include/linux/fs.h | 2
include/linux/fscache-cache.h | 507 ++-------
include/linux/fscache-obsolete.h | 13
include/linux/fscache.h | 784 +++++--------
include/linux/pagemap.h | 14
include/linux/uio.h | 11
include/net/af_rxrpc.h | 2
include/trace/events/afs.h | 51 -
include/trace/events/cachefiles.h | 285 ++++-
include/trace/events/fscache.h | 439 +------
lib/iov_iter.c | 280 ++++-
mm/filemap.c | 18
net/rxrpc/recvmsg.c | 9
67 files changed, 5655 insertions(+), 9674 deletions(-)
delete mode 100644 Documentation/filesystems/caching/operations.txt
create mode 100644 fs/cachefiles/content-map.c
create mode 100644 fs/cachefiles/io.c
delete mode 100644 fs/cachefiles/rdwr.c
create mode 100644 fs/fscache/dispatcher.c
create mode 100644 fs/fscache/io.c
create mode 100644 fs/fscache/obj.c
delete mode 100644 fs/fscache/object.c
create mode 100644 fs/fscache/object_bits.c
delete mode 100644 fs/fscache/operation.c
delete mode 100644 fs/fscache/page.c
create mode 100644 fs/fscache/read_helper.c
create mode 100644 include/linux/fscache-obsolete.h
^ permalink raw reply [flat|nested] 92+ messages in thread* [RFC PATCH 01/61] afs: Make afs_zap_data() static
2020-05-04 17:07 ` David Howells
(?)
@ 2020-05-04 17:07 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:07 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Make afs_zap_data() static as it's only used in the file in which it is
defined.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/inode.c | 2 +-
fs/afs/internal.h | 1 -
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 281470fe1183..67ccfbab683c 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -568,7 +568,7 @@ struct inode *afs_iget(struct super_block *sb, struct key *key,
* mark the data attached to an inode as obsolete due to a write on the server
* - might also want to ditch all the outstanding writes and dirty pages
*/
-void afs_zap_data(struct afs_vnode *vnode)
+static void afs_zap_data(struct afs_vnode *vnode)
{
_enter("{%llx:%llu}", vnode->fid.vid, vnode->fid.vnode);
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 80255513e230..03487f1bef07 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1017,7 +1017,6 @@ extern struct inode *afs_iget(struct super_block *, struct key *,
struct afs_iget_data *, struct afs_status_cb *,
struct afs_cb_interest *,
struct afs_vnode *);
-extern void afs_zap_data(struct afs_vnode *);
extern bool afs_check_validity(struct afs_vnode *);
extern int afs_validate(struct afs_vnode *, struct key *);
extern int afs_getattr(const struct path *, struct kstat *, u32, unsigned int);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 02/61] iov_iter: Add ITER_MAPPING
2020-05-04 17:07 ` David Howells
@ 2020-05-04 17:07 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:07 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: Matthew Wilcox, dhowells, Matthew Wilcox, Alexander Viro,
linux-afs, linux-nfs, linux-cifs, ceph-devel, v9fs-developer,
linux-fsdevel, linux-kernel
Add an iterator, ITER_MAPPING, that walks through a set of pages attached
to an address_space, starting at a given page and offset and walking for
the specified amount of bytes.
The caller must guarantee that the pages are all present and they must be
locked using PG_locked, PG_writeback or PG_fscache to prevent them from
going away or being migrated whilst they're being accessed.
This is useful for copying data from socket buffers to inodes in network
filesystems and for transferring data between those inodes and the cache
using direct I/O.
Whilst it is true that ITER_BVEC could be used instead, that would require
a bio_vec array to be allocated to refer to all the pages - which should be
redundant if inode->i_pages also points to all these pages.
This could also be turned into an ITER_XARRAY, taking and xarray pointer
instead of a mapping pointer. It would be mostly trivial, except for the
use of find_get_pages_contig() by iov_iter_get_pages*().
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Matthew Wilcox <willy@infradead.org>
---
include/linux/uio.h | 11 ++
lib/iov_iter.c | 280 +++++++++++++++++++++++++++++++++++++++++++++++----
2 files changed, 268 insertions(+), 23 deletions(-)
diff --git a/include/linux/uio.h b/include/linux/uio.h
index 9576fd8158d7..a0321a740f51 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,6 +11,7 @@
#include <uapi/linux/uio.h>
struct page;
+struct address_space;
struct pipe_inode_info;
struct kvec {
@@ -25,6 +26,7 @@ enum iter_type {
ITER_BVEC = 16,
ITER_PIPE = 32,
ITER_DISCARD = 64,
+ ITER_MAPPING = 128,
};
struct iov_iter {
@@ -40,6 +42,7 @@ struct iov_iter {
const struct iovec *iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
+ struct address_space *mapping;
struct pipe_inode_info *pipe;
};
union {
@@ -48,6 +51,7 @@ struct iov_iter {
unsigned int head;
unsigned int start_head;
};
+ loff_t mapping_start;
};
};
@@ -81,6 +85,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
+static inline bool iov_iter_is_mapping(const struct iov_iter *i)
+{
+ return iov_iter_type(i) == ITER_MAPPING;
+}
+
static inline unsigned char iov_iter_rw(const struct iov_iter *i)
{
return i->type & (READ | WRITE);
@@ -222,6 +231,8 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_
void iov_iter_pipe(struct iov_iter *i, unsigned int direction, struct pipe_inode_info *pipe,
size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_mapping(struct iov_iter *i, unsigned int direction, struct address_space *mapping,
+ loff_t start, size_t count);
ssize_t iov_iter_get_pages(struct iov_iter *i, struct page **pages,
size_t maxsize, unsigned maxpages, size_t *start);
ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, struct page ***pages,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 51595bf3af85..8b26128e85c2 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -74,7 +74,38 @@
} \
}
-#define iterate_all_kinds(i, n, v, I, B, K) { \
+#define iterate_mapping(i, n, __v, skip, STEP) { \
+ struct page *page; \
+ size_t wanted = n, seg, offset; \
+ loff_t start = i->mapping_start + skip; \
+ pgoff_t index = start >> PAGE_SHIFT; \
+ \
+ XA_STATE(xas, &i->mapping->i_pages, index); \
+ \
+ rcu_read_lock(); \
+ for (page = xas_load(&xas); page; page = xas_next(&xas)) { \
+ if (xas_retry(&xas, page)) \
+ continue; \
+ if (WARN_ON(xa_is_value(page))) \
+ break; \
+ if (WARN_ON(PageHuge(page))) \
+ break; \
+ __v.bv_page = find_subpage(page, xas.xa_index); \
+ offset = (i->mapping_start + skip) & ~PAGE_MASK; \
+ seg = PAGE_SIZE - offset; \
+ __v.bv_offset = offset; \
+ __v.bv_len = min(n, seg); \
+ (void)(STEP); \
+ n -= __v.bv_len; \
+ skip += __v.bv_len; \
+ if (n == 0) \
+ break; \
+ } \
+ rcu_read_unlock(); \
+ n = wanted - n; \
+}
+
+#define iterate_all_kinds(i, n, v, I, B, K, M) { \
if (likely(n)) { \
size_t skip = i->iov_offset; \
if (unlikely(i->type & ITER_BVEC)) { \
@@ -86,6 +117,9 @@
struct kvec v; \
iterate_kvec(i, n, v, kvec, skip, (K)) \
} else if (unlikely(i->type & ITER_DISCARD)) { \
+ } else if (unlikely(i->type & ITER_MAPPING)) { \
+ struct bio_vec v; \
+ iterate_mapping(i, n, v, skip, (M)); \
} else { \
const struct iovec *iov; \
struct iovec v; \
@@ -94,7 +128,7 @@
} \
}
-#define iterate_and_advance(i, n, v, I, B, K) { \
+#define iterate_and_advance(i, n, v, I, B, K, M) { \
if (unlikely(i->count < n)) \
n = i->count; \
if (i->count) { \
@@ -119,6 +153,9 @@
i->kvec = kvec; \
} else if (unlikely(i->type & ITER_DISCARD)) { \
skip += n; \
+ } else if (unlikely(i->type & ITER_MAPPING)) { \
+ struct bio_vec v; \
+ iterate_mapping(i, n, v, skip, (M)) \
} else { \
const struct iovec *iov; \
struct iovec v; \
@@ -628,7 +665,9 @@ size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
memcpy_to_page(v.bv_page, v.bv_offset,
(from += v.bv_len) - v.bv_len, v.bv_len),
- memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
+ memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
+ memcpy_to_page(v.bv_page, v.bv_offset,
+ (from += v.bv_len) - v.bv_len, v.bv_len)
)
return bytes;
@@ -746,6 +785,15 @@ size_t _copy_to_iter_mcsafe(const void *addr, size_t bytes, struct iov_iter *i)
bytes = curr_addr - s_addr - rem;
return bytes;
}
+ }),
+ ({
+ rem = memcpy_mcsafe_to_page(v.bv_page, v.bv_offset,
+ (from += v.bv_len) - v.bv_len, v.bv_len);
+ if (rem) {
+ curr_addr = (unsigned long) from;
+ bytes = curr_addr - s_addr - rem;
+ return bytes;
+ }
})
)
@@ -767,7 +815,9 @@ size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i)
copyin((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -793,7 +843,9 @@ bool _copy_from_iter_full(void *addr, size_t bytes, struct iov_iter *i)
0;}),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
iov_iter_advance(i, bytes);
@@ -813,7 +865,9 @@ size_t _copy_from_iter_nocache(void *addr, size_t bytes, struct iov_iter *i)
v.iov_base, v.iov_len),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -848,7 +902,9 @@ size_t _copy_from_iter_flushcache(void *addr, size_t bytes, struct iov_iter *i)
memcpy_page_flushcache((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
memcpy_flushcache((to += v.iov_len) - v.iov_len, v.iov_base,
- v.iov_len)
+ v.iov_len),
+ memcpy_page_flushcache((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -872,7 +928,9 @@ bool _copy_from_iter_full_nocache(void *addr, size_t bytes, struct iov_iter *i)
0;}),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
iov_iter_advance(i, bytes);
@@ -909,7 +967,7 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes,
{
if (unlikely(!page_copy_sane(page, offset, bytes)))
return 0;
- if (i->type & (ITER_BVEC|ITER_KVEC)) {
+ if (i->type & (ITER_BVEC | ITER_KVEC | ITER_MAPPING)) {
void *kaddr = kmap_atomic(page);
size_t wanted = copy_to_iter(kaddr + offset, bytes, i);
kunmap_atomic(kaddr);
@@ -932,7 +990,7 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes,
WARN_ON(1);
return 0;
}
- if (i->type & (ITER_BVEC|ITER_KVEC)) {
+ if (i->type & (ITER_BVEC | ITER_KVEC | ITER_MAPPING)) {
void *kaddr = kmap_atomic(page);
size_t wanted = _copy_from_iter(kaddr + offset, bytes, i);
kunmap_atomic(kaddr);
@@ -976,7 +1034,8 @@ size_t iov_iter_zero(size_t bytes, struct iov_iter *i)
iterate_and_advance(i, bytes, v,
clear_user(v.iov_base, v.iov_len),
memzero_page(v.bv_page, v.bv_offset, v.bv_len),
- memset(v.iov_base, 0, v.iov_len)
+ memset(v.iov_base, 0, v.iov_len),
+ memzero_page(v.bv_page, v.bv_offset, v.bv_len)
)
return bytes;
@@ -1000,7 +1059,9 @@ size_t iov_iter_copy_from_user_atomic(struct page *page,
copyin((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
memcpy_from_page((p += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((p += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
kunmap_atomic(kaddr);
return bytes;
@@ -1071,7 +1132,13 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
i->count -= size;
return;
}
- iterate_and_advance(i, size, v, 0, 0, 0)
+ if (unlikely(iov_iter_is_mapping(i))) {
+ /* We really don't want to fetch pages if we can avoid it */
+ i->iov_offset += size;
+ i->count -= size;
+ return;
+ }
+ iterate_and_advance(i, size, v, 0, 0, 0, 0)
}
EXPORT_SYMBOL(iov_iter_advance);
@@ -1115,7 +1182,12 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
return;
}
unroll -= i->iov_offset;
- if (iov_iter_is_bvec(i)) {
+ if (iov_iter_is_mapping(i)) {
+ BUG(); /* We should never go beyond the start of the specified
+ * range since we might then be straying into pages that
+ * aren't pinned.
+ */
+ } else if (iov_iter_is_bvec(i)) {
const struct bio_vec *bvec = i->bvec;
while (1) {
size_t n = (--bvec)->bv_len;
@@ -1152,9 +1224,9 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
return i->count; // it is a silly place, anyway
if (i->nr_segs == 1)
return i->count;
- if (unlikely(iov_iter_is_discard(i)))
+ if (unlikely(iov_iter_is_discard(i) || iov_iter_is_mapping(i)))
return i->count;
- else if (iov_iter_is_bvec(i))
+ if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
else
return min(i->count, i->iov->iov_len - i->iov_offset);
@@ -1202,6 +1274,32 @@ void iov_iter_pipe(struct iov_iter *i, unsigned int direction,
}
EXPORT_SYMBOL(iov_iter_pipe);
+/**
+ * iov_iter_mapping - Initialise an I/O iterator to use the pages in a mapping
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @mapping: The mapping to access.
+ * @start: The start file position.
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the pages attached to an
+ * inode or to inject data into those pages. The pages *must* be prevented
+ * from evaporation, either by taking a ref on them or locking them by the
+ * caller.
+ */
+void iov_iter_mapping(struct iov_iter *i, unsigned int direction,
+ struct address_space *mapping,
+ loff_t start, size_t count)
+{
+ BUG_ON(direction & ~1);
+ i->type = ITER_MAPPING | (direction & (READ | WRITE));
+ i->mapping = mapping;
+ i->mapping_start = start;
+ i->count = count;
+ i->iov_offset = 0;
+}
+EXPORT_SYMBOL(iov_iter_mapping);
+
/**
* iov_iter_discard - Initialise an I/O iterator that discards data
* @i: The iterator to initialise.
@@ -1235,7 +1333,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
iterate_all_kinds(i, size, v,
(res |= (unsigned long)v.iov_base | v.iov_len, 0),
res |= v.bv_offset | v.bv_len,
- res |= (unsigned long)v.iov_base | v.iov_len
+ res |= (unsigned long)v.iov_base | v.iov_len,
+ res |= v.bv_offset | v.bv_len
)
return res;
}
@@ -1257,7 +1356,9 @@ unsigned long iov_iter_gap_alignment(const struct iov_iter *i)
(res |= (!res ? 0 : (unsigned long)v.bv_offset) |
(size != v.bv_len ? size : 0)),
(res |= (!res ? 0 : (unsigned long)v.iov_base) |
- (size != v.iov_len ? size : 0))
+ (size != v.iov_len ? size : 0)),
+ (res |= (!res ? 0 : (unsigned long)v.bv_offset) |
+ (size != v.bv_len ? size : 0))
);
return res;
}
@@ -1307,6 +1408,46 @@ static ssize_t pipe_get_pages(struct iov_iter *i,
return __pipe_get_pages(i, min(maxsize, capacity), pages, iter_head, start);
}
+static ssize_t iter_mapping_get_pages(struct iov_iter *i,
+ struct page **pages, size_t maxsize,
+ unsigned maxpages, size_t *_start_offset)
+{
+ unsigned nr, offset;
+ pgoff_t index, count;
+ size_t size = maxsize, actual;
+ loff_t pos;
+
+ if (!size || !maxpages)
+ return 0;
+
+ pos = i->mapping_start + i->iov_offset;
+ index = pos >> PAGE_SHIFT;
+ offset = pos & ~PAGE_MASK;
+ *_start_offset = offset;
+
+ count = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ count += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ count++;
+ }
+
+ if (count > maxpages)
+ count = maxpages;
+
+ nr = find_get_pages_contig(i->mapping, index, count, pages);
+ if (nr == 0)
+ return 0;
+
+ actual = PAGE_SIZE * nr;
+ actual -= offset;
+ if (nr == count && size > 0)
+ actual -= PAGE_SIZE - size;
+ return actual;
+}
+
ssize_t iov_iter_get_pages(struct iov_iter *i,
struct page **pages, size_t maxsize, unsigned maxpages,
size_t *start)
@@ -1316,6 +1457,8 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
if (unlikely(iov_iter_is_pipe(i)))
return pipe_get_pages(i, pages, maxsize, maxpages, start);
+ if (unlikely(iov_iter_is_mapping(i)))
+ return iter_mapping_get_pages(i, pages, maxsize, maxpages, start);
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
@@ -1342,7 +1485,8 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
return v.bv_len;
}),({
return -EFAULT;
- })
+ }),
+ 0
)
return 0;
}
@@ -1386,6 +1530,49 @@ static ssize_t pipe_get_pages_alloc(struct iov_iter *i,
return n;
}
+static ssize_t iter_mapping_get_pages_alloc(struct iov_iter *i,
+ struct page ***pages, size_t maxsize,
+ size_t *_start_offset)
+{
+ struct page **p;
+ unsigned nr, offset;
+ pgoff_t index, count;
+ size_t size = maxsize, actual;
+ loff_t pos;
+
+ if (!size)
+ return 0;
+
+ pos = i->mapping_start + i->iov_offset;
+ index = pos >> PAGE_SHIFT;
+ offset = pos & ~PAGE_MASK;
+ *_start_offset = offset;
+
+ count = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ count += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ count++;
+ }
+
+ p = get_pages_array(count);
+ if (!p)
+ return -ENOMEM;
+ *pages = p;
+
+ nr = find_get_pages_contig(i->mapping, index, count, p);
+ if (nr == 0)
+ return 0;
+
+ actual = PAGE_SIZE * nr;
+ actual -= offset;
+ if (nr == count && size > 0)
+ actual -= PAGE_SIZE - size;
+ return actual;
+}
+
ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
size_t *start)
@@ -1397,6 +1584,8 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
if (unlikely(iov_iter_is_pipe(i)))
return pipe_get_pages_alloc(i, pages, maxsize, start);
+ if (unlikely(iov_iter_is_mapping(i)))
+ return iter_mapping_get_pages_alloc(i, pages, maxsize, start);
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
@@ -1429,7 +1618,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
return v.bv_len;
}),({
return -EFAULT;
- })
+ }), 0
)
return 0;
}
@@ -1468,6 +1657,14 @@ size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum,
v.iov_base, v.iov_len,
sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck(p + v.bv_offset,
+ (to += v.bv_len) - v.bv_len,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1510,6 +1707,14 @@ bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum,
v.iov_base, v.iov_len,
sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck(p + v.bv_offset,
+ (to += v.bv_len) - v.bv_len,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1556,6 +1761,14 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *csump,
(from += v.iov_len) - v.iov_len,
v.iov_len, sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck((from += v.bv_len) - v.bv_len,
+ p + v.bv_offset,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1605,6 +1818,21 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
npages = pipe_space_for_user(iter_head, pipe->tail, pipe);
if (npages >= maxpages)
return maxpages;
+ } else if (unlikely(iov_iter_is_mapping(i))) {
+ unsigned offset;
+
+ offset = (i->mapping_start + i->iov_offset) & ~PAGE_MASK;
+
+ npages = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ npages += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ npages++;
+ }
+ if (npages >= maxpages)
+ return maxpages;
} else iterate_all_kinds(i, size, v, ({
unsigned long p = (unsigned long)v.iov_base;
npages += DIV_ROUND_UP(p + v.iov_len, PAGE_SIZE)
@@ -1621,7 +1849,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
- p / PAGE_SIZE;
if (npages >= maxpages)
return maxpages;
- })
+ }),
+ 0
)
return npages;
}
@@ -1634,7 +1863,7 @@ const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags)
WARN_ON(1);
return NULL;
}
- if (unlikely(iov_iter_is_discard(new)))
+ if (unlikely(iov_iter_is_discard(new) || iov_iter_is_mapping(new)))
return NULL;
if (iov_iter_is_bvec(new))
return new->bvec = kmemdup(new->bvec,
@@ -1746,7 +1975,12 @@ int iov_iter_for_each_range(struct iov_iter *i, size_t bytes,
kunmap(v.bv_page);
err;}), ({
w = v;
- err = f(&w, context);})
+ err = f(&w, context);}), ({
+ w.iov_base = kmap(v.bv_page) + v.bv_offset;
+ w.iov_len = v.bv_len;
+ err = f(&w, context);
+ kunmap(v.bv_page);
+ err;})
)
return err;
}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 02/61] iov_iter: Add ITER_MAPPING
@ 2020-05-04 17:07 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:07 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: Matthew Wilcox, dhowells
Add an iterator, ITER_MAPPING, that walks through a set of pages attached
to an address_space, starting at a given page and offset and walking for
the specified amount of bytes.
The caller must guarantee that the pages are all present and they must be
locked using PG_locked, PG_writeback or PG_fscache to prevent them from
going away or being migrated whilst they're being accessed.
This is useful for copying data from socket buffers to inodes in network
filesystems and for transferring data between those inodes and the cache
using direct I/O.
Whilst it is true that ITER_BVEC could be used instead, that would require
a bio_vec array to be allocated to refer to all the pages - which should be
redundant if inode->i_pages also points to all these pages.
This could also be turned into an ITER_XARRAY, taking and xarray pointer
instead of a mapping pointer. It would be mostly trivial, except for the
use of find_get_pages_contig() by iov_iter_get_pages*().
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Matthew Wilcox <willy@infradead.org>
---
include/linux/uio.h | 11 ++
lib/iov_iter.c | 280 +++++++++++++++++++++++++++++++++++++++++++++++----
2 files changed, 268 insertions(+), 23 deletions(-)
diff --git a/include/linux/uio.h b/include/linux/uio.h
index 9576fd8158d7..a0321a740f51 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,6 +11,7 @@
#include <uapi/linux/uio.h>
struct page;
+struct address_space;
struct pipe_inode_info;
struct kvec {
@@ -25,6 +26,7 @@ enum iter_type {
ITER_BVEC = 16,
ITER_PIPE = 32,
ITER_DISCARD = 64,
+ ITER_MAPPING = 128,
};
struct iov_iter {
@@ -40,6 +42,7 @@ struct iov_iter {
const struct iovec *iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
+ struct address_space *mapping;
struct pipe_inode_info *pipe;
};
union {
@@ -48,6 +51,7 @@ struct iov_iter {
unsigned int head;
unsigned int start_head;
};
+ loff_t mapping_start;
};
};
@@ -81,6 +85,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
+static inline bool iov_iter_is_mapping(const struct iov_iter *i)
+{
+ return iov_iter_type(i) == ITER_MAPPING;
+}
+
static inline unsigned char iov_iter_rw(const struct iov_iter *i)
{
return i->type & (READ | WRITE);
@@ -222,6 +231,8 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_
void iov_iter_pipe(struct iov_iter *i, unsigned int direction, struct pipe_inode_info *pipe,
size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_mapping(struct iov_iter *i, unsigned int direction, struct address_space *mapping,
+ loff_t start, size_t count);
ssize_t iov_iter_get_pages(struct iov_iter *i, struct page **pages,
size_t maxsize, unsigned maxpages, size_t *start);
ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, struct page ***pages,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 51595bf3af85..8b26128e85c2 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -74,7 +74,38 @@
} \
}
-#define iterate_all_kinds(i, n, v, I, B, K) { \
+#define iterate_mapping(i, n, __v, skip, STEP) { \
+ struct page *page; \
+ size_t wanted = n, seg, offset; \
+ loff_t start = i->mapping_start + skip; \
+ pgoff_t index = start >> PAGE_SHIFT; \
+ \
+ XA_STATE(xas, &i->mapping->i_pages, index); \
+ \
+ rcu_read_lock(); \
+ for (page = xas_load(&xas); page; page = xas_next(&xas)) { \
+ if (xas_retry(&xas, page)) \
+ continue; \
+ if (WARN_ON(xa_is_value(page))) \
+ break; \
+ if (WARN_ON(PageHuge(page))) \
+ break; \
+ __v.bv_page = find_subpage(page, xas.xa_index); \
+ offset = (i->mapping_start + skip) & ~PAGE_MASK; \
+ seg = PAGE_SIZE - offset; \
+ __v.bv_offset = offset; \
+ __v.bv_len = min(n, seg); \
+ (void)(STEP); \
+ n -= __v.bv_len; \
+ skip += __v.bv_len; \
+ if (n == 0) \
+ break; \
+ } \
+ rcu_read_unlock(); \
+ n = wanted - n; \
+}
+
+#define iterate_all_kinds(i, n, v, I, B, K, M) { \
if (likely(n)) { \
size_t skip = i->iov_offset; \
if (unlikely(i->type & ITER_BVEC)) { \
@@ -86,6 +117,9 @@
struct kvec v; \
iterate_kvec(i, n, v, kvec, skip, (K)) \
} else if (unlikely(i->type & ITER_DISCARD)) { \
+ } else if (unlikely(i->type & ITER_MAPPING)) { \
+ struct bio_vec v; \
+ iterate_mapping(i, n, v, skip, (M)); \
} else { \
const struct iovec *iov; \
struct iovec v; \
@@ -94,7 +128,7 @@
} \
}
-#define iterate_and_advance(i, n, v, I, B, K) { \
+#define iterate_and_advance(i, n, v, I, B, K, M) { \
if (unlikely(i->count < n)) \
n = i->count; \
if (i->count) { \
@@ -119,6 +153,9 @@
i->kvec = kvec; \
} else if (unlikely(i->type & ITER_DISCARD)) { \
skip += n; \
+ } else if (unlikely(i->type & ITER_MAPPING)) { \
+ struct bio_vec v; \
+ iterate_mapping(i, n, v, skip, (M)) \
} else { \
const struct iovec *iov; \
struct iovec v; \
@@ -628,7 +665,9 @@ size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
memcpy_to_page(v.bv_page, v.bv_offset,
(from += v.bv_len) - v.bv_len, v.bv_len),
- memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
+ memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
+ memcpy_to_page(v.bv_page, v.bv_offset,
+ (from += v.bv_len) - v.bv_len, v.bv_len)
)
return bytes;
@@ -746,6 +785,15 @@ size_t _copy_to_iter_mcsafe(const void *addr, size_t bytes, struct iov_iter *i)
bytes = curr_addr - s_addr - rem;
return bytes;
}
+ }),
+ ({
+ rem = memcpy_mcsafe_to_page(v.bv_page, v.bv_offset,
+ (from += v.bv_len) - v.bv_len, v.bv_len);
+ if (rem) {
+ curr_addr = (unsigned long) from;
+ bytes = curr_addr - s_addr - rem;
+ return bytes;
+ }
})
)
@@ -767,7 +815,9 @@ size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i)
copyin((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -793,7 +843,9 @@ bool _copy_from_iter_full(void *addr, size_t bytes, struct iov_iter *i)
0;}),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
iov_iter_advance(i, bytes);
@@ -813,7 +865,9 @@ size_t _copy_from_iter_nocache(void *addr, size_t bytes, struct iov_iter *i)
v.iov_base, v.iov_len),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -848,7 +902,9 @@ size_t _copy_from_iter_flushcache(void *addr, size_t bytes, struct iov_iter *i)
memcpy_page_flushcache((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
memcpy_flushcache((to += v.iov_len) - v.iov_len, v.iov_base,
- v.iov_len)
+ v.iov_len),
+ memcpy_page_flushcache((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
return bytes;
@@ -872,7 +928,9 @@ bool _copy_from_iter_full_nocache(void *addr, size_t bytes, struct iov_iter *i)
0;}),
memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((to += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((to += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
iov_iter_advance(i, bytes);
@@ -909,7 +967,7 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes,
{
if (unlikely(!page_copy_sane(page, offset, bytes)))
return 0;
- if (i->type & (ITER_BVEC|ITER_KVEC)) {
+ if (i->type & (ITER_BVEC | ITER_KVEC | ITER_MAPPING)) {
void *kaddr = kmap_atomic(page);
size_t wanted = copy_to_iter(kaddr + offset, bytes, i);
kunmap_atomic(kaddr);
@@ -932,7 +990,7 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes,
WARN_ON(1);
return 0;
}
- if (i->type & (ITER_BVEC|ITER_KVEC)) {
+ if (i->type & (ITER_BVEC | ITER_KVEC | ITER_MAPPING)) {
void *kaddr = kmap_atomic(page);
size_t wanted = _copy_from_iter(kaddr + offset, bytes, i);
kunmap_atomic(kaddr);
@@ -976,7 +1034,8 @@ size_t iov_iter_zero(size_t bytes, struct iov_iter *i)
iterate_and_advance(i, bytes, v,
clear_user(v.iov_base, v.iov_len),
memzero_page(v.bv_page, v.bv_offset, v.bv_len),
- memset(v.iov_base, 0, v.iov_len)
+ memset(v.iov_base, 0, v.iov_len),
+ memzero_page(v.bv_page, v.bv_offset, v.bv_len)
)
return bytes;
@@ -1000,7 +1059,9 @@ size_t iov_iter_copy_from_user_atomic(struct page *page,
copyin((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
memcpy_from_page((p += v.bv_len) - v.bv_len, v.bv_page,
v.bv_offset, v.bv_len),
- memcpy((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len)
+ memcpy((p += v.iov_len) - v.iov_len, v.iov_base, v.iov_len),
+ memcpy_from_page((p += v.bv_len) - v.bv_len, v.bv_page,
+ v.bv_offset, v.bv_len)
)
kunmap_atomic(kaddr);
return bytes;
@@ -1071,7 +1132,13 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
i->count -= size;
return;
}
- iterate_and_advance(i, size, v, 0, 0, 0)
+ if (unlikely(iov_iter_is_mapping(i))) {
+ /* We really don't want to fetch pages if we can avoid it */
+ i->iov_offset += size;
+ i->count -= size;
+ return;
+ }
+ iterate_and_advance(i, size, v, 0, 0, 0, 0)
}
EXPORT_SYMBOL(iov_iter_advance);
@@ -1115,7 +1182,12 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
return;
}
unroll -= i->iov_offset;
- if (iov_iter_is_bvec(i)) {
+ if (iov_iter_is_mapping(i)) {
+ BUG(); /* We should never go beyond the start of the specified
+ * range since we might then be straying into pages that
+ * aren't pinned.
+ */
+ } else if (iov_iter_is_bvec(i)) {
const struct bio_vec *bvec = i->bvec;
while (1) {
size_t n = (--bvec)->bv_len;
@@ -1152,9 +1224,9 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
return i->count; // it is a silly place, anyway
if (i->nr_segs == 1)
return i->count;
- if (unlikely(iov_iter_is_discard(i)))
+ if (unlikely(iov_iter_is_discard(i) || iov_iter_is_mapping(i)))
return i->count;
- else if (iov_iter_is_bvec(i))
+ if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
else
return min(i->count, i->iov->iov_len - i->iov_offset);
@@ -1202,6 +1274,32 @@ void iov_iter_pipe(struct iov_iter *i, unsigned int direction,
}
EXPORT_SYMBOL(iov_iter_pipe);
+/**
+ * iov_iter_mapping - Initialise an I/O iterator to use the pages in a mapping
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @mapping: The mapping to access.
+ * @start: The start file position.
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the pages attached to an
+ * inode or to inject data into those pages. The pages *must* be prevented
+ * from evaporation, either by taking a ref on them or locking them by the
+ * caller.
+ */
+void iov_iter_mapping(struct iov_iter *i, unsigned int direction,
+ struct address_space *mapping,
+ loff_t start, size_t count)
+{
+ BUG_ON(direction & ~1);
+ i->type = ITER_MAPPING | (direction & (READ | WRITE));
+ i->mapping = mapping;
+ i->mapping_start = start;
+ i->count = count;
+ i->iov_offset = 0;
+}
+EXPORT_SYMBOL(iov_iter_mapping);
+
/**
* iov_iter_discard - Initialise an I/O iterator that discards data
* @i: The iterator to initialise.
@@ -1235,7 +1333,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
iterate_all_kinds(i, size, v,
(res |= (unsigned long)v.iov_base | v.iov_len, 0),
res |= v.bv_offset | v.bv_len,
- res |= (unsigned long)v.iov_base | v.iov_len
+ res |= (unsigned long)v.iov_base | v.iov_len,
+ res |= v.bv_offset | v.bv_len
)
return res;
}
@@ -1257,7 +1356,9 @@ unsigned long iov_iter_gap_alignment(const struct iov_iter *i)
(res |= (!res ? 0 : (unsigned long)v.bv_offset) |
(size != v.bv_len ? size : 0)),
(res |= (!res ? 0 : (unsigned long)v.iov_base) |
- (size != v.iov_len ? size : 0))
+ (size != v.iov_len ? size : 0)),
+ (res |= (!res ? 0 : (unsigned long)v.bv_offset) |
+ (size != v.bv_len ? size : 0))
);
return res;
}
@@ -1307,6 +1408,46 @@ static ssize_t pipe_get_pages(struct iov_iter *i,
return __pipe_get_pages(i, min(maxsize, capacity), pages, iter_head, start);
}
+static ssize_t iter_mapping_get_pages(struct iov_iter *i,
+ struct page **pages, size_t maxsize,
+ unsigned maxpages, size_t *_start_offset)
+{
+ unsigned nr, offset;
+ pgoff_t index, count;
+ size_t size = maxsize, actual;
+ loff_t pos;
+
+ if (!size || !maxpages)
+ return 0;
+
+ pos = i->mapping_start + i->iov_offset;
+ index = pos >> PAGE_SHIFT;
+ offset = pos & ~PAGE_MASK;
+ *_start_offset = offset;
+
+ count = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ count += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ count++;
+ }
+
+ if (count > maxpages)
+ count = maxpages;
+
+ nr = find_get_pages_contig(i->mapping, index, count, pages);
+ if (nr == 0)
+ return 0;
+
+ actual = PAGE_SIZE * nr;
+ actual -= offset;
+ if (nr == count && size > 0)
+ actual -= PAGE_SIZE - size;
+ return actual;
+}
+
ssize_t iov_iter_get_pages(struct iov_iter *i,
struct page **pages, size_t maxsize, unsigned maxpages,
size_t *start)
@@ -1316,6 +1457,8 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
if (unlikely(iov_iter_is_pipe(i)))
return pipe_get_pages(i, pages, maxsize, maxpages, start);
+ if (unlikely(iov_iter_is_mapping(i)))
+ return iter_mapping_get_pages(i, pages, maxsize, maxpages, start);
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
@@ -1342,7 +1485,8 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
return v.bv_len;
}),({
return -EFAULT;
- })
+ }),
+ 0
)
return 0;
}
@@ -1386,6 +1530,49 @@ static ssize_t pipe_get_pages_alloc(struct iov_iter *i,
return n;
}
+static ssize_t iter_mapping_get_pages_alloc(struct iov_iter *i,
+ struct page ***pages, size_t maxsize,
+ size_t *_start_offset)
+{
+ struct page **p;
+ unsigned nr, offset;
+ pgoff_t index, count;
+ size_t size = maxsize, actual;
+ loff_t pos;
+
+ if (!size)
+ return 0;
+
+ pos = i->mapping_start + i->iov_offset;
+ index = pos >> PAGE_SHIFT;
+ offset = pos & ~PAGE_MASK;
+ *_start_offset = offset;
+
+ count = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ count += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ count++;
+ }
+
+ p = get_pages_array(count);
+ if (!p)
+ return -ENOMEM;
+ *pages = p;
+
+ nr = find_get_pages_contig(i->mapping, index, count, p);
+ if (nr == 0)
+ return 0;
+
+ actual = PAGE_SIZE * nr;
+ actual -= offset;
+ if (nr == count && size > 0)
+ actual -= PAGE_SIZE - size;
+ return actual;
+}
+
ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
size_t *start)
@@ -1397,6 +1584,8 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
if (unlikely(iov_iter_is_pipe(i)))
return pipe_get_pages_alloc(i, pages, maxsize, start);
+ if (unlikely(iov_iter_is_mapping(i)))
+ return iter_mapping_get_pages_alloc(i, pages, maxsize, start);
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
@@ -1429,7 +1618,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
return v.bv_len;
}),({
return -EFAULT;
- })
+ }), 0
)
return 0;
}
@@ -1468,6 +1657,14 @@ size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum,
v.iov_base, v.iov_len,
sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck(p + v.bv_offset,
+ (to += v.bv_len) - v.bv_len,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1510,6 +1707,14 @@ bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum,
v.iov_base, v.iov_len,
sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck(p + v.bv_offset,
+ (to += v.bv_len) - v.bv_len,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1556,6 +1761,14 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *csump,
(from += v.iov_len) - v.iov_len,
v.iov_len, sum, off);
off += v.iov_len;
+ }), ({
+ char *p = kmap_atomic(v.bv_page);
+ next = csum_partial_copy_nocheck((from += v.bv_len) - v.bv_len,
+ p + v.bv_offset,
+ v.bv_len, 0);
+ kunmap_atomic(p);
+ sum = csum_block_add(sum, next, off);
+ off += v.bv_len;
})
)
*csum = sum;
@@ -1605,6 +1818,21 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
npages = pipe_space_for_user(iter_head, pipe->tail, pipe);
if (npages >= maxpages)
return maxpages;
+ } else if (unlikely(iov_iter_is_mapping(i))) {
+ unsigned offset;
+
+ offset = (i->mapping_start + i->iov_offset) & ~PAGE_MASK;
+
+ npages = 1;
+ if (size > PAGE_SIZE - offset) {
+ size -= PAGE_SIZE - offset;
+ npages += size >> PAGE_SHIFT;
+ size &= ~PAGE_MASK;
+ if (size)
+ npages++;
+ }
+ if (npages >= maxpages)
+ return maxpages;
} else iterate_all_kinds(i, size, v, ({
unsigned long p = (unsigned long)v.iov_base;
npages += DIV_ROUND_UP(p + v.iov_len, PAGE_SIZE)
@@ -1621,7 +1849,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
- p / PAGE_SIZE;
if (npages >= maxpages)
return maxpages;
- })
+ }),
+ 0
)
return npages;
}
@@ -1634,7 +1863,7 @@ const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags)
WARN_ON(1);
return NULL;
}
- if (unlikely(iov_iter_is_discard(new)))
+ if (unlikely(iov_iter_is_discard(new) || iov_iter_is_mapping(new)))
return NULL;
if (iov_iter_is_bvec(new))
return new->bvec = kmemdup(new->bvec,
@@ -1746,7 +1975,12 @@ int iov_iter_for_each_range(struct iov_iter *i, size_t bytes,
kunmap(v.bv_page);
err;}), ({
w = v;
- err = f(&w, context);})
+ err = f(&w, context);}), ({
+ w.iov_base = kmap(v.bv_page) + v.bv_offset;
+ w.iov_len = v.bv_len;
+ err = f(&w, context);
+ kunmap(v.bv_page);
+ err;})
)
return err;
}
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 03/61] vm: Add wait/unlock functions for PG_fscache
2020-05-04 17:07 ` David Howells
` (2 preceding siblings ...)
(?)
@ 2020-05-04 17:07 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:07 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Add functions to unlock and wait for unlock of PG_fscache analogously with
those for PG_lock.
Signed-off-by: David Howells <dhowells@redhat.com>
---
include/linux/pagemap.h | 14 ++++++++++++++
mm/filemap.c | 18 ++++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index a8f7bd8ea1c6..6b90226e6ef9 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -461,6 +461,7 @@ extern int __lock_page_killable(struct page *page);
extern int __lock_page_or_retry(struct page *page, struct mm_struct *mm,
unsigned int flags);
extern void unlock_page(struct page *page);
+extern void unlock_page_fscache(struct page *page);
/*
* Return true if the page was successfully locked
@@ -535,6 +536,19 @@ static inline int wait_on_page_locked_killable(struct page *page)
return wait_on_page_bit_killable(compound_head(page), PG_locked);
}
+/**
+ * wait_on_page_fscache - Wait for PG_fscache to be cleared on a page
+ * @page: The page
+ *
+ * Wait for the fscache mark to be removed from a page, usually signifying the
+ * completion of a write from that page to the cache.
+ */
+static inline void wait_on_page_fscache(struct page *page)
+{
+ if (PagePrivate2(page))
+ wait_on_page_bit(compound_head(page), PG_fscache);
+}
+
extern void put_and_wait_on_page_locked(struct page *page);
void wait_on_page_writeback(struct page *page);
diff --git a/mm/filemap.c b/mm/filemap.c
index 23a051a7ef0f..609db7977687 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1296,6 +1296,24 @@ void unlock_page(struct page *page)
}
EXPORT_SYMBOL(unlock_page);
+/**
+ * unlock_page_fscache - Unlock a page pinned with PG_fscache
+ * @page: The page
+ *
+ * Unlocks the page and wakes up sleepers in wait_on_page_fscache(). Also
+ * wakes those waiting for the lock and writeback bits because the wakeup
+ * mechanism is shared. But that's OK - those sleepers will just go back to
+ * sleep.
+ */
+void unlock_page_fscache(struct page *page)
+{
+ page = compound_head(page);
+ VM_BUG_ON_PAGE(!PagePrivate2(page), page);
+ clear_bit_unlock(PG_fscache, &page->flags);
+ wake_up_page_bit(page, PG_fscache);
+}
+EXPORT_SYMBOL(unlock_page_fscache);
+
/**
* end_page_writeback - end writeback against a page
* @page: the page
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 04/61] vfs: Export rw_verify_area() for use by cachefiles
2020-05-04 17:07 ` David Howells
` (3 preceding siblings ...)
(?)
@ 2020-05-04 17:08 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Export rw_verify_area() for so that cachefiles can use it before issuing
call_read_iter() and call_write_iter() to effect async DIO operations
against the cache.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/internal.h | 5 -----
fs/read_write.c | 1 +
include/linux/fs.h | 1 +
3 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/fs/internal.h b/fs/internal.h
index aa5d45524e87..fade86c4e4b4 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -159,11 +159,6 @@ extern char *simple_dname(struct dentry *, char *, int);
extern void dput_to_list(struct dentry *, struct list_head *);
extern void shrink_dentry_list(struct list_head *);
-/*
- * read_write.c
- */
-extern int rw_verify_area(int, struct file *, const loff_t *, size_t);
-
/*
* pipe.c
*/
diff --git a/fs/read_write.c b/fs/read_write.c
index bbfa9b12b15e..eb18270a1e14 100644
--- a/fs/read_write.c
+++ b/fs/read_write.c
@@ -400,6 +400,7 @@ int rw_verify_area(int read_write, struct file *file, const loff_t *ppos, size_t
return security_file_permission(file,
read_write == READ ? MAY_READ : MAY_WRITE);
}
+EXPORT_SYMBOL(rw_verify_area);
static ssize_t new_sync_read(struct file *filp, char __user *buf, size_t len, loff_t *ppos)
{
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 4f6f59b4f22a..2d2704aff9df 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2895,6 +2895,7 @@ extern int notify_change(struct dentry *, struct iattr *, struct inode **);
extern int inode_permission(struct inode *, int);
extern int generic_permission(struct inode *, int);
extern int __check_sticky(struct inode *dir, struct inode *inode);
+extern int rw_verify_area(int, struct file *, const loff_t *, size_t);
static inline bool execute_ok(struct inode *inode)
{
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 05/61] vfs: Provide S_CACHE_FILE inode flag
@ 2020-05-04 17:08 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Provide an S_CACHE_FILE inode flag that cachefiles can set to ward off
other kernel services and drivers (including itself) from using its cache
files.
Signed-off-by: David Howells <dhowells@redhat.com>
---
include/linux/fs.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 2d2704aff9df..728eae950dc3 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2003,6 +2003,7 @@ struct super_operations {
#define S_ENCRYPTED 16384 /* Encrypted file (using fs/crypto/) */
#define S_CASEFOLD 32768 /* Casefolded file */
#define S_VERITY 65536 /* Verity file (using fs/verity/) */
+#define S_CACHE_FILE 0x20000 /* File is in use as cache file (eg. fs/cachefiles) */
/*
* Note that nosuid etc flags are inode-specific: setting some file-system
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 05/61] vfs: Provide S_CACHE_FILE inode flag
@ 2020-05-04 17:08 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Provide an S_CACHE_FILE inode flag that cachefiles can set to ward off
other kernel services and drivers (including itself) from using its cache
files.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
include/linux/fs.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 2d2704aff9df..728eae950dc3 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2003,6 +2003,7 @@ struct super_operations {
#define S_ENCRYPTED 16384 /* Encrypted file (using fs/crypto/) */
#define S_CASEFOLD 32768 /* Casefolded file */
#define S_VERITY 65536 /* Verity file (using fs/verity/) */
+#define S_CACHE_FILE 0x20000 /* File is in use as cache file (eg. fs/cachefiles) */
/*
* Note that nosuid etc flags are inode-specific: setting some file-system
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 06/61] afs: Disable use of the fscache I/O routines
2020-05-04 17:07 ` David Howells
` (5 preceding siblings ...)
(?)
@ 2020-05-04 17:08 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Disable use of the fscache I/O routined by the AFS filesystem. It's about to
transition to passing iov_iters down and fscache is about to have its I/O path
to use iov_iter, so all that needs to change.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/file.c | 196 ++++++++++----------------------------------------------
fs/afs/write.c | 10 ---
2 files changed, 35 insertions(+), 171 deletions(-)
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 8415733f7bc1..f0cca37f2da8 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -202,24 +202,6 @@ void afs_put_read(struct afs_read *req)
}
}
-#ifdef CONFIG_AFS_FSCACHE
-/*
- * deal with notification that a page was read from the cache
- */
-static void afs_file_readpage_read_complete(struct page *page,
- void *data,
- int error)
-{
- _enter("%p,%p,%d", page, data, error);
-
- /* if the read completes with an error, we just unlock the page and let
- * the VM reissue the readpage */
- if (!error)
- SetPageUptodate(page);
- unlock_page(page);
-}
-#endif
-
/*
* Fetch file data from the volume.
*/
@@ -285,89 +267,46 @@ int afs_page_filler(void *data, struct page *page)
if (test_bit(AFS_VNODE_DELETED, &vnode->flags))
goto error;
- /* is it cached? */
-#ifdef CONFIG_AFS_FSCACHE
- ret = fscache_read_or_alloc_page(vnode->cache,
- page,
- afs_file_readpage_read_complete,
- NULL,
- GFP_KERNEL);
-#else
- ret = -ENOBUFS;
-#endif
- switch (ret) {
- /* read BIO submitted (page in cache) */
- case 0:
- break;
-
- /* page not yet cached */
- case -ENODATA:
- _debug("cache said ENODATA");
- goto go_on;
-
- /* page will not be cached */
- case -ENOBUFS:
- _debug("cache said ENOBUFS");
-
- /* fall through */
- default:
- go_on:
- req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
- if (!req)
- goto enomem;
-
- /* We request a full page. If the page is a partial one at the
- * end of the file, the server will return a short read and the
- * unmarshalling code will clear the unfilled space.
- */
- refcount_set(&req->usage, 1);
- req->pos = (loff_t)page->index << PAGE_SHIFT;
- req->len = PAGE_SIZE;
- req->nr_pages = 1;
- req->pages = req->array;
- req->pages[0] = page;
- get_page(page);
-
- /* read the contents of the file from the server into the
- * page */
- ret = afs_fetch_data(vnode, key, req);
- afs_put_read(req);
-
- if (ret < 0) {
- if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
- }
-
-#ifdef CONFIG_AFS_FSCACHE
- fscache_uncache_page(vnode->cache, page);
-#endif
- BUG_ON(PageFsCache(page));
-
- if (ret == -EINTR ||
- ret == -ENOMEM ||
- ret == -ERESTARTSYS ||
- ret == -EAGAIN)
- goto error;
- goto io_error;
- }
+ req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
+ if (!req)
+ goto enomem;
+
+ /* We request a full page. If the page is a partial one at the
+ * end of the file, the server will return a short read and the
+ * unmarshalling code will clear the unfilled space.
+ */
+ refcount_set(&req->usage, 1);
+ req->pos = (loff_t)page->index << PAGE_SHIFT;
+ req->len = PAGE_SIZE;
+ req->nr_pages = 1;
+ req->pages = req->array;
+ req->pages[0] = page;
+ get_page(page);
- SetPageUptodate(page);
+ /* read the contents of the file from the server into the
+ * page */
+ ret = afs_fetch_data(vnode, key, req);
+ afs_put_read(req);
- /* send the page to the cache */
-#ifdef CONFIG_AFS_FSCACHE
- if (PageFsCache(page) &&
- fscache_write_page(vnode->cache, page, vnode->status.size,
- GFP_KERNEL) != 0) {
- fscache_uncache_page(vnode->cache, page);
- BUG_ON(PageFsCache(page));
+ if (ret < 0) {
+ if (ret == -ENOENT) {
+ _debug("got NOENT from server"
+ " - marking file deleted and stale");
+ set_bit(AFS_VNODE_DELETED, &vnode->flags);
+ ret = -ESTALE;
}
-#endif
- unlock_page(page);
+
+ if (ret == -EINTR ||
+ ret == -ENOMEM ||
+ ret == -ERESTARTSYS ||
+ ret == -EAGAIN)
+ goto error;
+ goto io_error;
}
+ SetPageUptodate(page);
+ unlock_page(page);
+
_leave(" = 0");
return 0;
@@ -413,23 +352,10 @@ static int afs_readpage(struct file *file, struct page *page)
*/
static void afs_readpages_page_done(struct afs_read *req)
{
-#ifdef CONFIG_AFS_FSCACHE
- struct afs_vnode *vnode = req->vnode;
-#endif
struct page *page = req->pages[req->index];
req->pages[req->index] = NULL;
SetPageUptodate(page);
-
- /* send the page to the cache */
-#ifdef CONFIG_AFS_FSCACHE
- if (PageFsCache(page) &&
- fscache_write_page(vnode->cache, page, vnode->status.size,
- GFP_KERNEL) != 0) {
- fscache_uncache_page(vnode->cache, page);
- BUG_ON(PageFsCache(page));
- }
-#endif
unlock_page(page);
put_page(page);
}
@@ -488,9 +414,6 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
index = page->index;
if (add_to_page_cache_lru(page, mapping, index,
readahead_gfp_mask(mapping))) {
-#ifdef CONFIG_AFS_FSCACHE
- fscache_uncache_page(vnode->cache, page);
-#endif
put_page(page);
break;
}
@@ -523,9 +446,6 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
for (i = 0; i < req->nr_pages; i++) {
page = req->pages[i];
if (page) {
-#ifdef CONFIG_AFS_FSCACHE
- fscache_uncache_page(vnode->cache, page);
-#endif
SetPageError(page);
unlock_page(page);
}
@@ -557,37 +477,6 @@ static int afs_readpages(struct file *file, struct address_space *mapping,
}
/* attempt to read as many of the pages as possible */
-#ifdef CONFIG_AFS_FSCACHE
- ret = fscache_read_or_alloc_pages(vnode->cache,
- mapping,
- pages,
- &nr_pages,
- afs_file_readpage_read_complete,
- NULL,
- mapping_gfp_mask(mapping));
-#else
- ret = -ENOBUFS;
-#endif
-
- switch (ret) {
- /* all pages are being read from the cache */
- case 0:
- BUG_ON(!list_empty(pages));
- BUG_ON(nr_pages != 0);
- _leave(" = 0 [reading all]");
- return 0;
-
- /* there were pages that couldn't be read from the cache */
- case -ENODATA:
- case -ENOBUFS:
- break;
-
- /* other error */
- default:
- _leave(" = %d", ret);
- return ret;
- }
-
while (!list_empty(pages)) {
ret = afs_readpages_one(file, mapping, pages);
if (ret < 0)
@@ -615,14 +504,6 @@ static void afs_invalidatepage(struct page *page, unsigned int offset,
/* we clean up only if the entire page is being invalidated */
if (offset == 0 && length == PAGE_SIZE) {
-#ifdef CONFIG_AFS_FSCACHE
- if (PageFsCache(page)) {
- struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
- fscache_wait_on_page_write(vnode->cache, page);
- fscache_uncache_page(vnode->cache, page);
- }
-#endif
-
if (PagePrivate(page)) {
priv = page_private(page);
trace_afs_page_dirty(vnode, tracepoint_string("inval"),
@@ -650,13 +531,6 @@ static int afs_releasepage(struct page *page, gfp_t gfp_flags)
/* deny if page is being written to the cache and the caller hasn't
* elected to wait */
-#ifdef CONFIG_AFS_FSCACHE
- if (!fscache_maybe_release_page(vnode->cache, page, gfp_flags)) {
- _leave(" = F [cache busy]");
- return 0;
- }
-#endif
-
if (PagePrivate(page)) {
priv = page_private(page);
trace_afs_page_dirty(vnode, tracepoint_string("rel"),
diff --git a/fs/afs/write.c b/fs/afs/write.c
index cb76566763db..25786237485a 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -788,9 +788,6 @@ vm_fault_t afs_page_mkwrite(struct vm_fault *vmf)
/* Wait for the page to be written to the cache before we allow it to
* be modified. We then assume the entire page will need writing back.
*/
-#ifdef CONFIG_AFS_FSCACHE
- fscache_wait_on_page_write(vnode->cache, vmf->page);
-#endif
if (PageWriteback(vmf->page) &&
wait_on_page_bit_killable(vmf->page, PG_writeback) < 0)
@@ -875,12 +872,5 @@ int afs_launder_page(struct page *page)
page->index, priv);
set_page_private(page, 0);
ClearPagePrivate(page);
-
-#ifdef CONFIG_AFS_FSCACHE
- if (PageFsCache(page)) {
- fscache_wait_on_page_write(vnode->cache, page);
- fscache_uncache_page(vnode->cache, page);
- }
-#endif
return ret;
}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 07/61] fscache: Add a cookie debug ID and use that in traces
2020-05-04 17:07 ` David Howells
` (6 preceding siblings ...)
(?)
@ 2020-05-04 17:08 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Add a cookie debug ID and use that in traces and in procfiles rather than
displaying the (hashed) pointer to the cookie. This is easier to correlate
and we don't lose anything when interpreting oops output since that shows
unhashed addresses and registers that aren't comparable to the hashed
values.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/cookie.c | 29 ++++++---
fs/fscache/fsdef.c | 1
fs/fscache/object-list.c | 14 ++--
include/linux/fscache.h | 1
include/trace/events/cachefiles.h | 68 +++++++++++-----------
include/trace/events/fscache.h | 116 +++++++++++++++++++------------------
6 files changed, 121 insertions(+), 108 deletions(-)
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 0ce39658a620..b94e3995011e 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -29,21 +29,29 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
static void fscache_print_cookie(struct fscache_cookie *cookie, char prefix)
{
- struct hlist_node *object;
+ struct fscache_object *object;
+ struct hlist_node *o;
const u8 *k;
unsigned loop;
- pr_err("%c-cookie c=%p [p=%p fl=%lx nc=%u na=%u]\n",
- prefix, cookie, cookie->parent, cookie->flags,
+ pr_err("%c-cookie c=%08x [p=%08x fl=%lx nc=%u na=%u]\n",
+ prefix,
+ cookie->debug_id,
+ cookie->parent ? cookie->parent->debug_id : 0,
+ cookie->flags,
atomic_read(&cookie->n_children),
atomic_read(&cookie->n_active));
- pr_err("%c-cookie d=%p n=%p\n",
- prefix, cookie->def, cookie->netfs_data);
+ pr_err("%c-cookie d=%p{%s} n=%p\n",
+ prefix,
+ cookie->def,
+ cookie->def ? cookie->def->name : "?",
+ cookie->netfs_data);
- object = READ_ONCE(cookie->backing_objects.first);
- if (object)
- pr_err("%c-cookie o=%p\n",
- prefix, hlist_entry(object, struct fscache_object, cookie_link));
+ o = READ_ONCE(cookie->backing_objects.first);
+ if (o) {
+ object = hlist_entry(o, struct fscache_object, cookie_link);
+ pr_err("%c-cookie o=%u\n", prefix, object->debug_id);
+ }
pr_err("%c-key=[%u] '", prefix, cookie->key_len);
k = (cookie->key_len <= sizeof(cookie->inline_key)) ?
@@ -129,6 +137,8 @@ static long fscache_compare_cookie(const struct fscache_cookie *a,
return memcmp(ka, kb, a->key_len);
}
+static atomic_t fscache_cookie_debug_id = ATOMIC_INIT(1);
+
/*
* Allocate a cookie.
*/
@@ -163,6 +173,7 @@ struct fscache_cookie *fscache_alloc_cookie(
atomic_set(&cookie->usage, 1);
atomic_set(&cookie->n_children, 0);
+ cookie->debug_id = atomic_inc_return(&fscache_cookie_debug_id);
/* We keep the active count elevated until relinquishment to prevent an
* attempt to wake up every time the object operations queue quiesces.
diff --git a/fs/fscache/fsdef.c b/fs/fscache/fsdef.c
index 09ed8795ad86..5f8f6fe243fe 100644
--- a/fs/fscache/fsdef.c
+++ b/fs/fscache/fsdef.c
@@ -45,6 +45,7 @@ static struct fscache_cookie_def fscache_fsdef_index_def = {
};
struct fscache_cookie fscache_fsdef_index = {
+ .debug_id = 1,
.usage = ATOMIC_INIT(1),
.n_active = ATOMIC_INIT(1),
.lock = __SPIN_LOCK_UNLOCKED(fscache_fsdef_index.lock),
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index e106a1a1600d..1a0dc32c0a33 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -170,7 +170,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
" EM EV FL S"
- " | NETFS_COOKIE_DEF TY FL NETFS_DATA");
+ " | COOKIE NETFS_COOKIE_DEF TY FL NETFS_DATA");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ");
@@ -189,7 +189,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 2) {
seq_puts(m, "======== ======== ==== ===== === === === == ====="
" == == == ="
- " | ================ == == ================");
+ " | ======== ================ == === ================");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ================");
@@ -231,9 +231,9 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
}
seq_printf(m,
- "%8x %8x %s %5u %3u %3u %3u %2u %5u %2lx %2lx %2lx %1x | ",
+ "%08x %08x %s %5u %3u %3u %3u %2u %5u %2lx %2lx %2lx %1x | ",
obj->debug_id,
- obj->parent ? obj->parent->debug_id : -1,
+ obj->parent ? obj->parent->debug_id : UINT_MAX,
obj->state->short_name,
obj->n_children,
obj->n_ops,
@@ -246,7 +246,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
obj->flags,
work_busy(&obj->work));
- if (fscache_use_cookie(obj)) {
+ if (obj->cookie) {
uint16_t keylen = 0, auxlen = 0;
switch (cookie->type) {
@@ -263,7 +263,8 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
break;
}
- seq_printf(m, "%-16s %s %2lx %16p",
+ seq_printf(m, "%08x %-16s %s %3lx %16p",
+ cookie->debug_id,
cookie->def->name,
type,
cookie->flags,
@@ -292,7 +293,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
}
seq_puts(m, "\n");
- fscache_unuse_cookie(obj);
} else {
seq_puts(m, "<no_netfs>\n");
}
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index ad044c0cb1f3..0229bb80b73c 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -134,6 +134,7 @@ struct fscache_cookie {
atomic_t usage; /* number of users of this cookie */
atomic_t n_children; /* number of children of this cookie */
atomic_t n_active; /* number of active users of netfs ptrs */
+ unsigned int debug_id;
spinlock_t lock;
spinlock_t stores_lock; /* lock on page store tree */
struct hlist_head backing_objects; /* object(s) backing this file/index */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 5d9de24cb9c0..9a448fe9355d 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -78,20 +78,20 @@ TRACE_EVENT(cachefiles_ref,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, obj )
+ __field(unsigned int, cookie )
__field(enum cachefiles_obj_ref_trace, why )
__field(int, usage )
),
TP_fast_assign(
- __entry->obj = obj;
- __entry->cookie = cookie;
+ __entry->obj = obj->fscache.debug_id;
+ __entry->cookie = cookie->debug_id;
__entry->usage = usage;
__entry->why = why;
),
- TP_printk("c=%p o=%p u=%d %s",
+ TP_printk("c=%08x o=%08x u=%d %s",
__entry->cookie, __entry->obj, __entry->usage,
__print_symbolic(__entry->why, cachefiles_obj_ref_traces))
);
@@ -104,18 +104,18 @@ TRACE_EVENT(cachefiles_lookup,
TP_ARGS(obj, de, inode),
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(struct inode *, inode )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->inode = inode;
),
- TP_printk("o=%p d=%p i=%p",
+ TP_printk("o=%08x d=%p i=%p",
__entry->obj, __entry->de, __entry->inode)
);
@@ -126,18 +126,18 @@ TRACE_EVENT(cachefiles_mkdir,
TP_ARGS(obj, de, ret),
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(int, ret )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->ret = ret;
),
- TP_printk("o=%p d=%p r=%u",
+ TP_printk("o=%08x d=%p r=%u",
__entry->obj, __entry->de, __entry->ret)
);
@@ -148,18 +148,18 @@ TRACE_EVENT(cachefiles_create,
TP_ARGS(obj, de, ret),
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(int, ret )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->ret = ret;
),
- TP_printk("o=%p d=%p r=%u",
+ TP_printk("o=%08x d=%p r=%u",
__entry->obj, __entry->de, __entry->ret)
);
@@ -172,18 +172,18 @@ TRACE_EVENT(cachefiles_unlink,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(enum fscache_why_object_killed, why )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->why = why;
),
- TP_printk("o=%p d=%p w=%s",
+ TP_printk("o=%08x d=%p w=%s",
__entry->obj, __entry->de,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
);
@@ -198,20 +198,20 @@ TRACE_EVENT(cachefiles_rename,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(struct dentry *, to )
__field(enum fscache_why_object_killed, why )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->to = to;
__entry->why = why;
),
- TP_printk("o=%p d=%p t=%p w=%s",
+ TP_printk("o=%08x d=%p t=%p w=%s",
__entry->obj, __entry->de, __entry->to,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
);
@@ -224,16 +224,16 @@ TRACE_EVENT(cachefiles_mark_active,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
),
- TP_printk("o=%p d=%p",
+ TP_printk("o=%08x d=%p",
__entry->obj, __entry->de)
);
@@ -246,22 +246,22 @@ TRACE_EVENT(cachefiles_wait_active,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
+ __field(unsigned int, xobj )
__field(struct dentry *, de )
- __field(struct cachefiles_object *, xobj )
__field(u16, flags )
__field(u16, fsc_flags )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
- __entry->xobj = xobj;
+ __entry->xobj = xobj->fscache.debug_id;
__entry->flags = xobj->flags;
__entry->fsc_flags = xobj->fscache.flags;
),
- TP_printk("o=%p d=%p wo=%p wf=%x wff=%x",
+ TP_printk("o=%08x d=%p wo=%08x wf=%x wff=%x",
__entry->obj, __entry->de, __entry->xobj,
__entry->flags, __entry->fsc_flags)
);
@@ -275,18 +275,18 @@ TRACE_EVENT(cachefiles_mark_inactive,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(struct inode *, inode )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->inode = inode;
),
- TP_printk("o=%p d=%p i=%p",
+ TP_printk("o=%08x d=%p i=%p",
__entry->obj, __entry->de, __entry->inode)
);
@@ -299,18 +299,18 @@ TRACE_EVENT(cachefiles_mark_buried,
/* Note that obj may be NULL */
TP_STRUCT__entry(
- __field(struct cachefiles_object *, obj )
+ __field(unsigned int, obj )
__field(struct dentry *, de )
__field(enum fscache_why_object_killed, why )
),
TP_fast_assign(
- __entry->obj = obj;
+ __entry->obj = obj->fscache.debug_id;
__entry->de = de;
__entry->why = why;
),
- TP_printk("o=%p d=%p w=%s",
+ TP_printk("o=%08x d=%p w=%s",
__entry->obj, __entry->de,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
);
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index d16fe6ed78a2..0b9e058aba4d 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -167,8 +167,8 @@ TRACE_EVENT(fscache_cookie,
TP_ARGS(cookie, where, usage),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_cookie *, parent )
+ __field(unsigned int, cookie )
+ __field(unsigned int, parent )
__field(enum fscache_cookie_trace, where )
__field(int, usage )
__field(int, n_children )
@@ -177,8 +177,8 @@ TRACE_EVENT(fscache_cookie,
),
TP_fast_assign(
- __entry->cookie = cookie;
- __entry->parent = cookie->parent;
+ __entry->cookie = cookie->debug_id;
+ __entry->parent = cookie->parent ? cookie->parent->debug_id : 0;
__entry->where = where;
__entry->usage = usage;
__entry->n_children = atomic_read(&cookie->n_children);
@@ -186,7 +186,7 @@ TRACE_EVENT(fscache_cookie,
__entry->flags = cookie->flags;
),
- TP_printk("%s c=%p u=%d p=%p Nc=%d Na=%d f=%02x",
+ TP_printk("%s c=%08x u=%d p=%08x Nc=%d Na=%d f=%02x",
__print_symbolic(__entry->where, fscache_cookie_traces),
__entry->cookie, __entry->usage,
__entry->parent, __entry->n_children, __entry->n_active,
@@ -199,17 +199,17 @@ TRACE_EVENT(fscache_netfs,
TP_ARGS(netfs),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
__array(char, name, 8 )
),
TP_fast_assign(
- __entry->cookie = netfs->primary_index;
+ __entry->cookie = netfs->primary_index->debug_id;
strncpy(__entry->name, netfs->name, 8);
__entry->name[7] = 0;
),
- TP_printk("c=%p n=%s",
+ TP_printk("c=%08x n=%s",
__entry->cookie, __entry->name)
);
@@ -219,8 +219,8 @@ TRACE_EVENT(fscache_acquire,
TP_ARGS(cookie),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_cookie *, parent )
+ __field(unsigned int, cookie )
+ __field(unsigned int, parent )
__array(char, name, 8 )
__field(int, p_usage )
__field(int, p_n_children )
@@ -228,8 +228,8 @@ TRACE_EVENT(fscache_acquire,
),
TP_fast_assign(
- __entry->cookie = cookie;
- __entry->parent = cookie->parent;
+ __entry->cookie = cookie->debug_id;
+ __entry->parent = cookie->parent->debug_id;
__entry->p_usage = atomic_read(&cookie->parent->usage);
__entry->p_n_children = atomic_read(&cookie->parent->n_children);
__entry->p_flags = cookie->parent->flags;
@@ -237,7 +237,7 @@ TRACE_EVENT(fscache_acquire,
__entry->name[7] = 0;
),
- TP_printk("c=%p p=%p pu=%d pc=%d pf=%02x n=%s",
+ TP_printk("c=%08x p=%08x pu=%d pc=%d pf=%02x n=%s",
__entry->cookie, __entry->parent, __entry->p_usage,
__entry->p_n_children, __entry->p_flags, __entry->name)
);
@@ -248,8 +248,8 @@ TRACE_EVENT(fscache_relinquish,
TP_ARGS(cookie, retire),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_cookie *, parent )
+ __field(unsigned int, cookie )
+ __field(unsigned int, parent )
__field(int, usage )
__field(int, n_children )
__field(int, n_active )
@@ -258,8 +258,8 @@ TRACE_EVENT(fscache_relinquish,
),
TP_fast_assign(
- __entry->cookie = cookie;
- __entry->parent = cookie->parent;
+ __entry->cookie = cookie->debug_id;
+ __entry->parent = cookie->parent->debug_id;
__entry->usage = atomic_read(&cookie->usage);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
@@ -267,7 +267,7 @@ TRACE_EVENT(fscache_relinquish,
__entry->retire = retire;
),
- TP_printk("c=%p u=%d p=%p Nc=%d Na=%d f=%02x r=%u",
+ TP_printk("c=%08x u=%d p=%08x Nc=%d Na=%d f=%02x r=%u",
__entry->cookie, __entry->usage,
__entry->parent, __entry->n_children, __entry->n_active,
__entry->flags, __entry->retire)
@@ -279,7 +279,7 @@ TRACE_EVENT(fscache_enable,
TP_ARGS(cookie),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
__field(int, usage )
__field(int, n_children )
__field(int, n_active )
@@ -287,14 +287,14 @@ TRACE_EVENT(fscache_enable,
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->usage = atomic_read(&cookie->usage);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
__entry->flags = cookie->flags;
),
- TP_printk("c=%p u=%d Nc=%d Na=%d f=%02x",
+ TP_printk("c=%08x u=%d Nc=%d Na=%d f=%02x",
__entry->cookie, __entry->usage,
__entry->n_children, __entry->n_active, __entry->flags)
);
@@ -305,7 +305,7 @@ TRACE_EVENT(fscache_disable,
TP_ARGS(cookie),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
__field(int, usage )
__field(int, n_children )
__field(int, n_active )
@@ -313,14 +313,14 @@ TRACE_EVENT(fscache_disable,
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->usage = atomic_read(&cookie->usage);
__entry->n_children = atomic_read(&cookie->n_children);
__entry->n_active = atomic_read(&cookie->n_active);
__entry->flags = cookie->flags;
),
- TP_printk("c=%p u=%d Nc=%d Na=%d f=%02x",
+ TP_printk("c=%08x u=%d Nc=%d Na=%d f=%02x",
__entry->cookie, __entry->usage,
__entry->n_children, __entry->n_active, __entry->flags)
);
@@ -333,8 +333,8 @@ TRACE_EVENT(fscache_osm,
TP_ARGS(object, state, wait, oob, event_num),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_object *, object )
+ __field(unsigned int, cookie )
+ __field(unsigned int, object )
__array(char, state, 8 )
__field(bool, wait )
__field(bool, oob )
@@ -342,15 +342,15 @@ TRACE_EVENT(fscache_osm,
),
TP_fast_assign(
- __entry->cookie = object->cookie;
- __entry->object = object;
+ __entry->cookie = object->cookie->debug_id;
+ __entry->object = object->debug_id;
__entry->wait = wait;
__entry->oob = oob;
__entry->event_num = event_num;
memcpy(__entry->state, state->short_name, 8);
),
- TP_printk("c=%p o=%p %s %s%sev=%d",
+ TP_printk("c=%08x o=%08d %s %s%sev=%d",
__entry->cookie,
__entry->object,
__entry->state,
@@ -370,18 +370,18 @@ TRACE_EVENT(fscache_page,
TP_ARGS(cookie, page, why),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
__field(pgoff_t, page )
__field(enum fscache_page_trace, why )
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->page = page->index;
__entry->why = why;
),
- TP_printk("c=%p %s pg=%lx",
+ TP_printk("c=%08x %s pg=%lx",
__entry->cookie,
__print_symbolic(__entry->why, fscache_page_traces),
__entry->page)
@@ -394,20 +394,20 @@ TRACE_EVENT(fscache_check_page,
TP_ARGS(cookie, page, val, n),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
__field(void *, page )
__field(void *, val )
__field(int, n )
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->page = page;
__entry->val = val;
__entry->n = n;
),
- TP_printk("c=%p pg=%p val=%p n=%d",
+ TP_printk("c=%08x pg=%p val=%p n=%d",
__entry->cookie, __entry->page, __entry->val, __entry->n)
);
@@ -417,14 +417,14 @@ TRACE_EVENT(fscache_wake_cookie,
TP_ARGS(cookie),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
),
- TP_printk("c=%p", __entry->cookie)
+ TP_printk("c=%08x", __entry->cookie)
);
TRACE_EVENT(fscache_op,
@@ -434,18 +434,18 @@ TRACE_EVENT(fscache_op,
TP_ARGS(cookie, op, why),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_operation *, op )
+ __field(unsigned int, cookie )
+ __field(unsigned int, op )
__field(enum fscache_op_trace, why )
),
TP_fast_assign(
- __entry->cookie = cookie;
- __entry->op = op;
+ __entry->cookie = cookie->debug_id;
+ __entry->op = op->debug_id;
__entry->why = why;
),
- TP_printk("c=%p op=%p %s",
+ TP_printk("c=%08x op=%08x %s",
__entry->cookie, __entry->op,
__print_symbolic(__entry->why, fscache_op_traces))
);
@@ -457,20 +457,20 @@ TRACE_EVENT(fscache_page_op,
TP_ARGS(cookie, page, op, what),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
+ __field(unsigned int, op )
__field(pgoff_t, page )
- __field(struct fscache_operation *, op )
__field(enum fscache_page_op_trace, what )
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->page = page ? page->index : 0;
- __entry->op = op;
+ __entry->op = op->debug_id;
__entry->what = what;
),
- TP_printk("c=%p %s pg=%lx op=%p",
+ TP_printk("c=%08x %s pg=%lx op=%08x",
__entry->cookie,
__print_symbolic(__entry->what, fscache_page_op_traces),
__entry->page, __entry->op)
@@ -483,20 +483,20 @@ TRACE_EVENT(fscache_wrote_page,
TP_ARGS(cookie, page, op, ret),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
+ __field(unsigned int, cookie )
+ __field(unsigned int, op )
__field(pgoff_t, page )
- __field(struct fscache_operation *, op )
__field(int, ret )
),
TP_fast_assign(
- __entry->cookie = cookie;
+ __entry->cookie = cookie->debug_id;
__entry->page = page->index;
- __entry->op = op;
+ __entry->op = op->debug_id;
__entry->ret = ret;
),
- TP_printk("c=%p pg=%lx op=%p ret=%d",
+ TP_printk("c=%08x pg=%lx op=%08x ret=%d",
__entry->cookie, __entry->page, __entry->op, __entry->ret)
);
@@ -507,22 +507,22 @@ TRACE_EVENT(fscache_gang_lookup,
TP_ARGS(cookie, op, results, n, store_limit),
TP_STRUCT__entry(
- __field(struct fscache_cookie *, cookie )
- __field(struct fscache_operation *, op )
+ __field(unsigned int, cookie )
+ __field(unsigned int, op )
__field(pgoff_t, results0 )
__field(int, n )
__field(pgoff_t, store_limit )
),
TP_fast_assign(
- __entry->cookie = cookie;
- __entry->op = op;
+ __entry->cookie = cookie->debug_id;
+ __entry->op = op->debug_id;
__entry->results0 = results[0] ? ((struct page *)results[0])->index : (pgoff_t)-1;
__entry->n = n;
__entry->store_limit = store_limit;
),
- TP_printk("c=%p op=%p r0=%lx n=%d sl=%lx",
+ TP_printk("c=%08x op=%08x r0=%lx n=%d sl=%lx",
__entry->cookie, __entry->op, __entry->results0, __entry->n,
__entry->store_limit)
);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 08/61] fscache: Procfile to display cookies
2020-05-04 17:07 ` David Howells
` (7 preceding siblings ...)
(?)
@ 2020-05-04 17:08 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Add /proc/fs/fscache/cookies to display active cookies.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/cookie.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++
fs/fscache/internal.h | 1
fs/fscache/proc.c | 7 +++
include/linux/fscache.h | 1
4 files changed, 112 insertions(+)
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index b94e3995011e..477c5fb349ee 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -19,6 +19,8 @@ static atomic_t fscache_object_debug_id = ATOMIC_INIT(0);
#define fscache_cookie_hash_shift 15
static struct hlist_bl_head fscache_cookie_hash[1 << fscache_cookie_hash_shift];
+static LIST_HEAD(fscache_cookies);
+static DEFINE_RWLOCK(fscache_cookies_lock);
static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie,
loff_t object_size);
@@ -65,6 +67,9 @@ void fscache_free_cookie(struct fscache_cookie *cookie)
{
if (cookie) {
BUG_ON(!hlist_empty(&cookie->backing_objects));
+ write_lock(&fscache_cookies_lock);
+ list_del(&cookie->proc_link);
+ write_unlock(&fscache_cookies_lock);
if (cookie->aux_len > sizeof(cookie->inline_aux))
kfree(cookie->aux);
if (cookie->key_len > sizeof(cookie->inline_key))
@@ -192,6 +197,10 @@ struct fscache_cookie *fscache_alloc_cookie(
/* radix tree insertion won't use the preallocation pool unless it's
* told it may not wait */
INIT_RADIX_TREE(&cookie->stores, GFP_NOFS & ~__GFP_DIRECT_RECLAIM);
+
+ write_lock(&fscache_cookies_lock);
+ list_add_tail(&cookie->proc_link, &fscache_cookies);
+ write_unlock(&fscache_cookies_lock);
return cookie;
nomem:
@@ -969,3 +978,97 @@ int __fscache_check_consistency(struct fscache_cookie *cookie,
return -ESTALE;
}
EXPORT_SYMBOL(__fscache_check_consistency);
+
+/*
+ * Generate a list of extant cookies in /proc/fs/fscache/cookies
+ */
+static int fscache_cookies_seq_show(struct seq_file *m, void *v)
+{
+ struct fscache_cookie *cookie;
+ unsigned int keylen = 0, auxlen = 0;
+ char _type[3], *type;
+ u8 *p;
+
+ if (v == &fscache_cookies) {
+ seq_puts(m,
+ "COOKIE PARENT USAGE CHILD ACT TY FL DEF NETFS_DATA\n"
+ "======== ======== ===== ===== === == === ================ ==========\n"
+ );
+ return 0;
+ }
+
+ cookie = list_entry(v, struct fscache_cookie, proc_link);
+
+ switch (cookie->type) {
+ case 0:
+ type = "IX";
+ break;
+ case 1:
+ type = "DT";
+ break;
+ default:
+ snprintf(_type, sizeof(_type), "%02u",
+ cookie->type);
+ type = _type;
+ break;
+ }
+
+ seq_printf(m,
+ "%08x %08x %5u %5u %3u %s %03lx %-16s %px",
+ cookie->debug_id,
+ cookie->parent ? cookie->parent->debug_id : 0,
+ atomic_read(&cookie->usage),
+ atomic_read(&cookie->n_children),
+ atomic_read(&cookie->n_active),
+ type,
+ cookie->flags,
+ cookie->def->name,
+ cookie->netfs_data);
+
+ keylen = cookie->key_len;
+ auxlen = cookie->aux_len;
+
+ if (keylen > 0 || auxlen > 0) {
+ seq_puts(m, " ");
+ p = keylen <= sizeof(cookie->inline_key) ?
+ cookie->inline_key : cookie->key;
+ for (; keylen > 0; keylen--)
+ seq_printf(m, "%02x", *p++);
+ if (auxlen > 0) {
+ seq_puts(m, ", ");
+ p = auxlen <= sizeof(cookie->inline_aux) ?
+ cookie->inline_aux : cookie->aux;
+ for (; auxlen > 0; auxlen--)
+ seq_printf(m, "%02x", *p++);
+ }
+ }
+
+ seq_puts(m, "\n");
+ return 0;
+}
+
+static void *fscache_cookies_seq_start(struct seq_file *m, loff_t *_pos)
+ __acquires(fscache_cookies_lock)
+{
+ read_lock(&fscache_cookies_lock);
+ return seq_list_start_head(&fscache_cookies, *_pos);
+}
+
+static void *fscache_cookies_seq_next(struct seq_file *m, void *v, loff_t *_pos)
+{
+ return seq_list_next(v, &fscache_cookies, _pos);
+}
+
+static void fscache_cookies_seq_stop(struct seq_file *m, void *v)
+ __releases(rcu)
+{
+ read_unlock(&fscache_cookies_lock);
+}
+
+
+const struct seq_operations fscache_cookies_seq_ops = {
+ .start = fscache_cookies_seq_start,
+ .next = fscache_cookies_seq_next,
+ .stop = fscache_cookies_seq_stop,
+ .show = fscache_cookies_seq_show,
+};
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 08e91efbce53..4b535c2dae4a 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -45,6 +45,7 @@ extern struct fscache_cache *fscache_select_cache_for_object(
* cookie.c
*/
extern struct kmem_cache *fscache_cookie_jar;
+extern const struct seq_operations fscache_cookies_seq_ops;
extern void fscache_free_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
diff --git a/fs/fscache/proc.c b/fs/fscache/proc.c
index 90a7bc22f7e1..da51fdfc8641 100644
--- a/fs/fscache/proc.c
+++ b/fs/fscache/proc.c
@@ -21,6 +21,10 @@ int __init fscache_proc_init(void)
if (!proc_mkdir("fs/fscache", NULL))
goto error_dir;
+ if (!proc_create_seq("fs/fscache/cookies", S_IFREG | 0444, NULL,
+ &fscache_cookies_seq_ops))
+ goto error_cookies;
+
#ifdef CONFIG_FSCACHE_STATS
if (!proc_create_single("fs/fscache/stats", S_IFREG | 0444, NULL,
fscache_stats_show))
@@ -53,6 +57,8 @@ int __init fscache_proc_init(void)
remove_proc_entry("fs/fscache/stats", NULL);
error_stats:
#endif
+ remove_proc_entry("fs/fscache/cookies", NULL);
+error_cookies:
remove_proc_entry("fs/fscache", NULL);
error_dir:
_leave(" = -ENOMEM");
@@ -73,5 +79,6 @@ void fscache_proc_cleanup(void)
#ifdef CONFIG_FSCACHE_STATS
remove_proc_entry("fs/fscache/stats", NULL);
#endif
+ remove_proc_entry("fs/fscache/cookies", NULL);
remove_proc_entry("fs/fscache", NULL);
}
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 0229bb80b73c..38ec80217899 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -141,6 +141,7 @@ struct fscache_cookie {
const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
struct hlist_bl_node hash_link; /* Link in hash table */
+ struct list_head proc_link; /* Link in proc list */
void *netfs_data; /* back pointer to netfs */
struct radix_tree_root stores; /* pages to be stored on this cookie */
#define FSCACHE_COOKIE_PENDING_TAG 0 /* pages tag: pending write to cache */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 09/61] fscache: Temporarily disable network filesystems' use of fscache
2020-05-04 17:07 ` David Howells
` (8 preceding siblings ...)
(?)
@ 2020-05-04 17:08 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Temporarily disable the use of fscache by the various Linux network
filesystems, apart from afs, so that the core can be rewritten.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/9p/Kconfig | 2 +-
fs/ceph/Kconfig | 2 +-
fs/cifs/Kconfig | 2 +-
fs/fscache/Kconfig | 4 ++++
fs/nfs/Kconfig | 2 +-
5 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/fs/9p/Kconfig b/fs/9p/Kconfig
index 09fd4a185fd2..66ca72422961 100644
--- a/fs/9p/Kconfig
+++ b/fs/9p/Kconfig
@@ -13,7 +13,7 @@ config 9P_FS
if 9P_FS
config 9P_FSCACHE
bool "Enable 9P client caching support"
- depends on 9P_FS=m && FSCACHE || 9P_FS=y && FSCACHE=y
+ depends on 9P_FS=m && FSCACHE_OLD || 9P_FS=y && FSCACHE_OLD=y
help
Choose Y here to enable persistent, read-only local
caching support for 9p clients using FS-Cache
diff --git a/fs/ceph/Kconfig b/fs/ceph/Kconfig
index cf235f6eacf9..9aee341392a2 100644
--- a/fs/ceph/Kconfig
+++ b/fs/ceph/Kconfig
@@ -20,7 +20,7 @@ config CEPH_FS
if CEPH_FS
config CEPH_FSCACHE
bool "Enable Ceph client caching support"
- depends on CEPH_FS=m && FSCACHE || CEPH_FS=y && FSCACHE=y
+ depends on CEPH_FS=m && FSCACHE_OLD || CEPH_FS=y && FSCACHE_OLD=y
help
Choose Y here to enable persistent, read-only local
caching support for Ceph clients using FS-Cache
diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
index 604f65f4b6c5..441f24bff49c 100644
--- a/fs/cifs/Kconfig
+++ b/fs/cifs/Kconfig
@@ -206,7 +206,7 @@ config CIFS_SMB_DIRECT
config CIFS_FSCACHE
bool "Provide CIFS client caching support"
- depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
+ depends on CIFS=m && FSCACHE_OLD || CIFS=y && FSCACHE_OLD=y
help
Makes CIFS FS-Cache capable. Say Y here if you want your CIFS data
to be cached locally on disk through the general filesystem cache
diff --git a/fs/fscache/Kconfig b/fs/fscache/Kconfig
index 506c5e643f0d..f95a5aa03b6b 100644
--- a/fs/fscache/Kconfig
+++ b/fs/fscache/Kconfig
@@ -60,3 +60,7 @@ config FSCACHE_OBJECT_LIST
help
Maintain a global list of active fscache objects that can be
retrieved through /proc/fs/fscache/objects for debugging purposes
+
+config FSCACHE_OLD
+ bool
+ default n
diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
index 88e1763e02f3..8909ef506073 100644
--- a/fs/nfs/Kconfig
+++ b/fs/nfs/Kconfig
@@ -170,7 +170,7 @@ config ROOT_NFS
config NFS_FSCACHE
bool "Provide NFS client caching support"
- depends on NFS_FS=m && FSCACHE || NFS_FS=y && FSCACHE=y
+ depends on NFS_FS=m && FSCACHE_OLD || NFS_FS=y && FSCACHE_OLD=y
help
Say Y here if you want NFS data to be cached locally on disc through
the general filesystem cache manager
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 10/61] fscache: Remove the old I/O API
@ 2020-05-04 17:08 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the old fscache I/O API. There's no point trying to transform it as
the new one will bear no similarities to the old one.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/Makefile | 1
fs/cachefiles/interface.c | 15 -
fs/cachefiles/internal.h | 38 -
fs/cachefiles/main.c | 1
fs/cachefiles/rdwr.c | 974 -----------------------------------
fs/fscache/cache.c | 6
fs/fscache/cookie.c | 10
fs/fscache/internal.h | 23 -
fs/fscache/object.c | 2
fs/fscache/page.c | 1068 --------------------------------------
fs/fscache/stats.c | 6
include/linux/fscache-cache.h | 132 -----
include/linux/fscache-obsolete.h | 13
include/linux/fscache.h | 349 ------------
14 files changed, 15 insertions(+), 2623 deletions(-)
delete mode 100644 fs/cachefiles/rdwr.c
create mode 100644 include/linux/fscache-obsolete.h
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index 891dedda5905..3455d3646547 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -10,7 +10,6 @@ cachefiles-y := \
key.o \
main.o \
namei.o \
- rdwr.o \
security.o \
xattr.o
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 4cea5fbf695e..04d92ad402a4 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -540,14 +540,6 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
_leave("");
}
-/*
- * dissociate a cache from all the pages it was backing
- */
-static void cachefiles_dissociate_pages(struct fscache_cache *cache)
-{
- _enter("");
-}
-
const struct fscache_cache_ops cachefiles_cache_ops = {
.name = "cachefiles",
.alloc_object = cachefiles_alloc_object,
@@ -560,12 +552,5 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
.attr_changed = cachefiles_attr_changed,
- .read_or_alloc_page = cachefiles_read_or_alloc_page,
- .read_or_alloc_pages = cachefiles_read_or_alloc_pages,
- .allocate_page = cachefiles_allocate_page,
- .allocate_pages = cachefiles_allocate_pages,
- .write_page = cachefiles_write_page,
- .uncache_page = cachefiles_uncache_page,
- .dissociate_pages = cachefiles_dissociate_pages,
.check_consistency = cachefiles_check_consistency,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index cf9bd6401c2d..aca73c8403ab 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -43,7 +43,6 @@ struct cachefiles_object {
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
uint8_t new; /* T if object new */
- spinlock_t work_lock;
struct rb_node active_node; /* link in active tree (dentry is key) */
};
@@ -89,28 +88,6 @@ struct cachefiles_cache {
char *tag; /* cache binding tag */
};
-/*
- * backing file read tracking
- */
-struct cachefiles_one_read {
- wait_queue_entry_t monitor; /* link into monitored waitqueue */
- struct page *back_page; /* backing file page we're waiting for */
- struct page *netfs_page; /* netfs page we're going to fill */
- struct fscache_retrieval *op; /* retrieval op covering this */
- struct list_head op_link; /* link in op's todo list */
-};
-
-/*
- * backing file write tracking
- */
-struct cachefiles_one_write {
- struct page *netfs_page; /* netfs page to copy */
- struct cachefiles_object *object;
- struct list_head obj_link; /* link in object's lists */
- fscache_rw_complete_t end_io_func;
- void *context;
-};
-
/*
* auxiliary data xattr buffer
*/
@@ -202,21 +179,6 @@ void cachefiles_hist(atomic_t histogram[], unsigned long start_jif)
#define cachefiles_hist(hist, start_jif) do {} while (0)
#endif
-/*
- * rdwr.c
- */
-extern int cachefiles_read_or_alloc_page(struct fscache_retrieval *,
- struct page *, gfp_t);
-extern int cachefiles_read_or_alloc_pages(struct fscache_retrieval *,
- struct list_head *, unsigned *,
- gfp_t);
-extern int cachefiles_allocate_page(struct fscache_retrieval *, struct page *,
- gfp_t);
-extern int cachefiles_allocate_pages(struct fscache_retrieval *,
- struct list_head *, unsigned *, gfp_t);
-extern int cachefiles_write_page(struct fscache_storage *, struct page *);
-extern void cachefiles_uncache_page(struct fscache_object *, struct page *);
-
/*
* security.c
*/
diff --git a/fs/cachefiles/main.c b/fs/cachefiles/main.c
index ddf0cd58d60c..3f0101a74809 100644
--- a/fs/cachefiles/main.c
+++ b/fs/cachefiles/main.c
@@ -42,7 +42,6 @@ static void cachefiles_object_init_once(void *_object)
struct cachefiles_object *object = _object;
memset(object, 0, sizeof(*object));
- spin_lock_init(&object->work_lock);
}
/*
diff --git a/fs/cachefiles/rdwr.c b/fs/cachefiles/rdwr.c
deleted file mode 100644
index 1dc97f2d6201..000000000000
--- a/fs/cachefiles/rdwr.c
+++ /dev/null
@@ -1,974 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Storage object read/write
- *
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- */
-
-#include <linux/mount.h>
-#include <linux/slab.h>
-#include <linux/file.h>
-#include <linux/swap.h>
-#include "internal.h"
-
-/*
- * detect wake up events generated by the unlocking of pages in which we're
- * interested
- * - we use this to detect read completion of backing pages
- * - the caller holds the waitqueue lock
- */
-static int cachefiles_read_waiter(wait_queue_entry_t *wait, unsigned mode,
- int sync, void *_key)
-{
- struct cachefiles_one_read *monitor =
- container_of(wait, struct cachefiles_one_read, monitor);
- struct cachefiles_object *object;
- struct fscache_retrieval *op = monitor->op;
- struct wait_bit_key *key = _key;
- struct page *page = wait->private;
-
- ASSERT(key);
-
- _enter("{%lu},%u,%d,{%p,%u}",
- monitor->netfs_page->index, mode, sync,
- key->flags, key->bit_nr);
-
- if (key->flags != &page->flags ||
- key->bit_nr != PG_locked)
- return 0;
-
- _debug("--- monitor %p %lx ---", page, page->flags);
-
- if (!PageUptodate(page) && !PageError(page)) {
- /* unlocked, not uptodate and not erronous? */
- _debug("page probably truncated");
- }
-
- /* remove from the waitqueue */
- list_del(&wait->entry);
-
- /* move onto the action list and queue for FS-Cache thread pool */
- ASSERT(op);
-
- /* We need to temporarily bump the usage count as we don't own a ref
- * here otherwise cachefiles_read_copier() may free the op between the
- * monitor being enqueued on the op->to_do list and the op getting
- * enqueued on the work queue.
- */
- fscache_get_retrieval(op);
-
- object = container_of(op->op.object, struct cachefiles_object, fscache);
- spin_lock(&object->work_lock);
- list_add_tail(&monitor->op_link, &op->to_do);
- spin_unlock(&object->work_lock);
-
- fscache_enqueue_retrieval(op);
- fscache_put_retrieval(op);
- return 0;
-}
-
-/*
- * handle a probably truncated page
- * - check to see if the page is still relevant and reissue the read if
- * possible
- * - return -EIO on error, -ENODATA if the page is gone, -EINPROGRESS if we
- * must wait again and 0 if successful
- */
-static int cachefiles_read_reissue(struct cachefiles_object *object,
- struct cachefiles_one_read *monitor)
-{
- struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
- struct page *backpage = monitor->back_page, *backpage2;
- int ret;
-
- _enter("{ino=%lx},{%lx,%lx}",
- d_backing_inode(object->backer)->i_ino,
- backpage->index, backpage->flags);
-
- /* skip if the page was truncated away completely */
- if (backpage->mapping != bmapping) {
- _leave(" = -ENODATA [mapping]");
- return -ENODATA;
- }
-
- backpage2 = find_get_page(bmapping, backpage->index);
- if (!backpage2) {
- _leave(" = -ENODATA [gone]");
- return -ENODATA;
- }
-
- if (backpage != backpage2) {
- put_page(backpage2);
- _leave(" = -ENODATA [different]");
- return -ENODATA;
- }
-
- /* the page is still there and we already have a ref on it, so we don't
- * need a second */
- put_page(backpage2);
-
- INIT_LIST_HEAD(&monitor->op_link);
- add_page_wait_queue(backpage, &monitor->monitor);
-
- if (trylock_page(backpage)) {
- ret = -EIO;
- if (PageError(backpage))
- goto unlock_discard;
- ret = 0;
- if (PageUptodate(backpage))
- goto unlock_discard;
-
- _debug("reissue read");
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto unlock_discard;
- }
-
- /* but the page may have been read before the monitor was installed, so
- * the monitor may miss the event - so we have to ensure that we do get
- * one in such a case */
- if (trylock_page(backpage)) {
- _debug("jumpstart %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
-
- /* it'll reappear on the todo list */
- _leave(" = -EINPROGRESS");
- return -EINPROGRESS;
-
-unlock_discard:
- unlock_page(backpage);
- spin_lock_irq(&object->work_lock);
- list_del(&monitor->op_link);
- spin_unlock_irq(&object->work_lock);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * copy data from backing pages to netfs pages to complete a read operation
- * - driven by FS-Cache's thread pool
- */
-static void cachefiles_read_copier(struct fscache_operation *_op)
-{
- struct cachefiles_one_read *monitor;
- struct cachefiles_object *object;
- struct fscache_retrieval *op;
- int error, max;
-
- op = container_of(_op, struct fscache_retrieval, op);
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
-
- _enter("{ino=%lu}", d_backing_inode(object->backer)->i_ino);
-
- max = 8;
- spin_lock_irq(&object->work_lock);
-
- while (!list_empty(&op->to_do)) {
- monitor = list_entry(op->to_do.next,
- struct cachefiles_one_read, op_link);
- list_del(&monitor->op_link);
-
- spin_unlock_irq(&object->work_lock);
-
- _debug("- copy {%lu}", monitor->back_page->index);
-
- recheck:
- if (test_bit(FSCACHE_COOKIE_INVALIDATING,
- &object->fscache.cookie->flags)) {
- error = -ESTALE;
- } else if (PageUptodate(monitor->back_page)) {
- copy_highpage(monitor->netfs_page, monitor->back_page);
- fscache_mark_page_cached(monitor->op,
- monitor->netfs_page);
- error = 0;
- } else if (!PageError(monitor->back_page)) {
- /* the page has probably been truncated */
- error = cachefiles_read_reissue(object, monitor);
- if (error == -EINPROGRESS)
- goto next;
- goto recheck;
- } else {
- cachefiles_io_error_obj(
- object,
- "Readpage failed on backing file %lx",
- (unsigned long) monitor->back_page->flags);
- error = -EIO;
- }
-
- put_page(monitor->back_page);
-
- fscache_end_io(op, monitor->netfs_page, error);
- put_page(monitor->netfs_page);
- fscache_retrieval_complete(op, 1);
- fscache_put_retrieval(op);
- kfree(monitor);
-
- next:
- /* let the thread pool have some air occasionally */
- max--;
- if (max < 0 || need_resched()) {
- if (!list_empty(&op->to_do))
- fscache_enqueue_retrieval(op);
- _leave(" [maxed out]");
- return;
- }
-
- spin_lock_irq(&object->work_lock);
- }
-
- spin_unlock_irq(&object->work_lock);
- _leave("");
-}
-
-/*
- * read the corresponding page to the given set from the backing file
- * - an uncertain page is simply discarded, to be tried again another time
- */
-static int cachefiles_read_backing_file_one(struct cachefiles_object *object,
- struct fscache_retrieval *op,
- struct page *netpage)
-{
- struct cachefiles_one_read *monitor;
- struct address_space *bmapping;
- struct page *newpage, *backpage;
- int ret;
-
- _enter("");
-
- _debug("read back %p{%lu,%d}",
- netpage, netpage->index, page_count(netpage));
-
- monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
- if (!monitor)
- goto nomem;
-
- monitor->netfs_page = netpage;
- monitor->op = fscache_get_retrieval(op);
-
- init_waitqueue_func_entry(&monitor->monitor, cachefiles_read_waiter);
-
- /* attempt to get hold of the backing page */
- bmapping = d_backing_inode(object->backer)->i_mapping;
- newpage = NULL;
-
- for (;;) {
- backpage = find_get_page(bmapping, netpage->index);
- if (backpage)
- goto backing_page_already_present;
-
- if (!newpage) {
- newpage = __page_cache_alloc(cachefiles_gfp);
- if (!newpage)
- goto nomem_monitor;
- }
-
- ret = add_to_page_cache_lru(newpage, bmapping,
- netpage->index, cachefiles_gfp);
- if (ret == 0)
- goto installed_new_backing_page;
- if (ret != -EEXIST)
- goto nomem_page;
- }
-
- /* we've installed a new backing page, so now we need to start
- * it reading */
-installed_new_backing_page:
- _debug("- new %p", newpage);
-
- backpage = newpage;
- newpage = NULL;
-
-read_backing_page:
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto read_error;
-
- /* set the monitor to transfer the data across */
-monitor_backing_page:
- _debug("- monitor add");
-
- /* install the monitor */
- get_page(monitor->netfs_page);
- get_page(backpage);
- monitor->back_page = backpage;
- monitor->monitor.private = backpage;
- add_page_wait_queue(backpage, &monitor->monitor);
- monitor = NULL;
-
- /* but the page may have been read before the monitor was installed, so
- * the monitor may miss the event - so we have to ensure that we do get
- * one in such a case */
- if (trylock_page(backpage)) {
- _debug("jumpstart %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
- goto success;
-
- /* if the backing page is already present, it can be in one of
- * three states: read in progress, read failed or read okay */
-backing_page_already_present:
- _debug("- present");
-
- if (newpage) {
- put_page(newpage);
- newpage = NULL;
- }
-
- if (PageError(backpage))
- goto io_error;
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate;
-
- if (!trylock_page(backpage))
- goto monitor_backing_page;
- _debug("read %p {%lx}", backpage, backpage->flags);
- goto read_backing_page;
-
- /* the backing page is already up to date, attach the netfs
- * page to the pagecache and LRU and copy the data across */
-backing_page_already_uptodate:
- _debug("- uptodate");
-
- fscache_mark_page_cached(op, netpage);
-
- copy_highpage(netpage, backpage);
- fscache_end_io(op, netpage, 0);
- fscache_retrieval_complete(op, 1);
-
-success:
- _debug("success");
- ret = 0;
-
-out:
- if (backpage)
- put_page(backpage);
- if (monitor) {
- fscache_put_retrieval(monitor->op);
- kfree(monitor);
- }
- _leave(" = %d", ret);
- return ret;
-
-read_error:
- _debug("read error %d", ret);
- if (ret == -ENOMEM) {
- fscache_retrieval_complete(op, 1);
- goto out;
- }
-io_error:
- cachefiles_io_error_obj(object, "Page read error on backing file");
- fscache_retrieval_complete(op, 1);
- ret = -ENOBUFS;
- goto out;
-
-nomem_page:
- put_page(newpage);
-nomem_monitor:
- fscache_put_retrieval(monitor->op);
- kfree(monitor);
-nomem:
- fscache_retrieval_complete(op, 1);
- _leave(" = -ENOMEM");
- return -ENOMEM;
-}
-
-/*
- * read a page from the cache or allocate a block in which to store it
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if no buffers can be made available
- * - returns -ENOBUFS if page is beyond EOF
- * - if the page is backed by a block in the cache:
- * - a read will be started which will call the callback on completion
- * - 0 will be returned
- * - else if the page is unbacked:
- * - the metadata will be retained
- * - -ENODATA will be returned
- */
-int cachefiles_read_or_alloc_page(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct inode *inode;
- sector_t block;
- unsigned shift;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("{%p},{%lx},,,", object, page->index);
-
- if (!object->backer)
- goto enobufs;
-
- inode = d_backing_inode(object->backer);
- ASSERT(S_ISREG(inode->i_mode));
- ASSERT(inode->i_mapping->a_ops->readpages);
-
- /* calculate the shift required to use bmap */
- shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
-
- op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
- op->op.flags |= FSCACHE_OP_ASYNC;
- op->op.processor = cachefiles_read_copier;
-
- /* we assume the absence or presence of the first block is a good
- * enough indication for the page as a whole
- * - TODO: don't use bmap() for this as it is _not_ actually good
- * enough for this as it doesn't indicate errors, but it's all we've
- * got for the moment
- */
- block = page->index;
- block <<= shift;
-
- ret = bmap(inode, &block);
- ASSERT(ret < 0);
-
- _debug("%llx -> %llx",
- (unsigned long long) (page->index << shift),
- (unsigned long long) block);
-
- if (block) {
- /* submit the apparently valid page to the backing fs to be
- * read from disk */
- ret = cachefiles_read_backing_file_one(object, op, page);
- } else if (cachefiles_has_space(cache, 0, 1) == 0) {
- /* there's space in the cache we can use */
- fscache_mark_page_cached(op, page);
- fscache_retrieval_complete(op, 1);
- ret = -ENODATA;
- } else {
- goto enobufs;
- }
-
- _leave(" = %d", ret);
- return ret;
-
-enobufs:
- fscache_retrieval_complete(op, 1);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-
-/*
- * read the corresponding pages to the given set from the backing file
- * - any uncertain pages are simply discarded, to be tried again another time
- */
-static int cachefiles_read_backing_file(struct cachefiles_object *object,
- struct fscache_retrieval *op,
- struct list_head *list)
-{
- struct cachefiles_one_read *monitor = NULL;
- struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
- struct page *newpage = NULL, *netpage, *_n, *backpage = NULL;
- int ret = 0;
-
- _enter("");
-
- list_for_each_entry_safe(netpage, _n, list, lru) {
- list_del(&netpage->lru);
-
- _debug("read back %p{%lu,%d}",
- netpage, netpage->index, page_count(netpage));
-
- if (!monitor) {
- monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
- if (!monitor)
- goto nomem;
-
- monitor->op = fscache_get_retrieval(op);
- init_waitqueue_func_entry(&monitor->monitor,
- cachefiles_read_waiter);
- }
-
- for (;;) {
- backpage = find_get_page(bmapping, netpage->index);
- if (backpage)
- goto backing_page_already_present;
-
- if (!newpage) {
- newpage = __page_cache_alloc(cachefiles_gfp);
- if (!newpage)
- goto nomem;
- }
-
- ret = add_to_page_cache_lru(newpage, bmapping,
- netpage->index,
- cachefiles_gfp);
- if (ret == 0)
- goto installed_new_backing_page;
- if (ret != -EEXIST)
- goto nomem;
- }
-
- /* we've installed a new backing page, so now we need
- * to start it reading */
- installed_new_backing_page:
- _debug("- new %p", newpage);
-
- backpage = newpage;
- newpage = NULL;
-
- reread_backing_page:
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto read_error;
-
- /* add the netfs page to the pagecache and LRU, and set the
- * monitor to transfer the data across */
- monitor_backing_page:
- _debug("- monitor add");
-
- ret = add_to_page_cache_lru(netpage, op->mapping,
- netpage->index, cachefiles_gfp);
- if (ret < 0) {
- if (ret == -EEXIST) {
- put_page(backpage);
- backpage = NULL;
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
- goto nomem;
- }
-
- /* install a monitor */
- get_page(netpage);
- monitor->netfs_page = netpage;
-
- get_page(backpage);
- monitor->back_page = backpage;
- monitor->monitor.private = backpage;
- add_page_wait_queue(backpage, &monitor->monitor);
- monitor = NULL;
-
- /* but the page may have been read before the monitor was
- * installed, so the monitor may miss the event - so we have to
- * ensure that we do get one in such a case */
- if (trylock_page(backpage)) {
- _debug("2unlock %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
-
- put_page(backpage);
- backpage = NULL;
-
- put_page(netpage);
- netpage = NULL;
- continue;
-
- /* if the backing page is already present, it can be in one of
- * three states: read in progress, read failed or read okay */
- backing_page_already_present:
- _debug("- present %p", backpage);
-
- if (PageError(backpage))
- goto io_error;
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate;
-
- _debug("- not ready %p{%lx}", backpage, backpage->flags);
-
- if (!trylock_page(backpage))
- goto monitor_backing_page;
-
- if (PageError(backpage)) {
- _debug("error %lx", backpage->flags);
- unlock_page(backpage);
- goto io_error;
- }
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate_unlock;
-
- /* we've locked a page that's neither up to date nor erroneous,
- * so we need to attempt to read it again */
- goto reread_backing_page;
-
- /* the backing page is already up to date, attach the netfs
- * page to the pagecache and LRU and copy the data across */
- backing_page_already_uptodate_unlock:
- _debug("uptodate %lx", backpage->flags);
- unlock_page(backpage);
- backing_page_already_uptodate:
- _debug("- uptodate");
-
- ret = add_to_page_cache_lru(netpage, op->mapping,
- netpage->index, cachefiles_gfp);
- if (ret < 0) {
- if (ret == -EEXIST) {
- put_page(backpage);
- backpage = NULL;
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
- goto nomem;
- }
-
- copy_highpage(netpage, backpage);
-
- put_page(backpage);
- backpage = NULL;
-
- fscache_mark_page_cached(op, netpage);
-
- /* the netpage is unlocked and marked up to date here */
- fscache_end_io(op, netpage, 0);
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
-
- netpage = NULL;
-
- _debug("out");
-
-out:
- /* tidy up */
- if (newpage)
- put_page(newpage);
- if (netpage)
- put_page(netpage);
- if (backpage)
- put_page(backpage);
- if (monitor) {
- fscache_put_retrieval(op);
- kfree(monitor);
- }
-
- list_for_each_entry_safe(netpage, _n, list, lru) {
- list_del(&netpage->lru);
- put_page(netpage);
- fscache_retrieval_complete(op, 1);
- }
-
- _leave(" = %d", ret);
- return ret;
-
-nomem:
- _debug("nomem");
- ret = -ENOMEM;
- goto record_page_complete;
-
-read_error:
- _debug("read error %d", ret);
- if (ret == -ENOMEM)
- goto record_page_complete;
-io_error:
- cachefiles_io_error_obj(object, "Page read error on backing file");
- ret = -ENOBUFS;
-record_page_complete:
- fscache_retrieval_complete(op, 1);
- goto out;
-}
-
-/*
- * read a list of pages from the cache or allocate blocks in which to store
- * them
- */
-int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct list_head backpages;
- struct pagevec pagevec;
- struct inode *inode;
- struct page *page, *_n;
- unsigned shift, nrbackpages;
- int ret, ret2, space;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("{OBJ%x,%d},,%d,,",
- object->fscache.debug_id, atomic_read(&op->op.usage),
- *nr_pages);
-
- if (!object->backer)
- goto all_enobufs;
-
- space = 1;
- if (cachefiles_has_space(cache, 0, *nr_pages) < 0)
- space = 0;
-
- inode = d_backing_inode(object->backer);
- ASSERT(S_ISREG(inode->i_mode));
- ASSERT(inode->i_mapping->a_ops->readpages);
-
- /* calculate the shift required to use bmap */
- shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
-
- pagevec_init(&pagevec);
-
- op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
- op->op.flags |= FSCACHE_OP_ASYNC;
- op->op.processor = cachefiles_read_copier;
-
- INIT_LIST_HEAD(&backpages);
- nrbackpages = 0;
-
- ret = space ? -ENODATA : -ENOBUFS;
- list_for_each_entry_safe(page, _n, pages, lru) {
- sector_t block;
-
- /* we assume the absence or presence of the first block is a
- * good enough indication for the page as a whole
- * - TODO: don't use bmap() for this as it is _not_ actually
- * good enough for this as it doesn't indicate errors, but
- * it's all we've got for the moment
- */
- block = page->index;
- block <<= shift;
-
- ret = bmap(inode, &block);
- ASSERT(!ret);
-
- _debug("%llx -> %llx",
- (unsigned long long) (page->index << shift),
- (unsigned long long) block);
-
- if (block) {
- /* we have data - add it to the list to give to the
- * backing fs */
- list_move(&page->lru, &backpages);
- (*nr_pages)--;
- nrbackpages++;
- } else if (space && pagevec_add(&pagevec, page) == 0) {
- fscache_mark_pages_cached(op, &pagevec);
- fscache_retrieval_complete(op, 1);
- ret = -ENODATA;
- } else {
- fscache_retrieval_complete(op, 1);
- }
- }
-
- if (pagevec_count(&pagevec) > 0)
- fscache_mark_pages_cached(op, &pagevec);
-
- if (list_empty(pages))
- ret = 0;
-
- /* submit the apparently valid pages to the backing fs to be read from
- * disk */
- if (nrbackpages > 0) {
- ret2 = cachefiles_read_backing_file(object, op, &backpages);
- if (ret2 == -ENOMEM || ret2 == -EINTR)
- ret = ret2;
- }
-
- _leave(" = %d [nr=%u%s]",
- ret, *nr_pages, list_empty(pages) ? " empty" : "");
- return ret;
-
-all_enobufs:
- fscache_retrieval_complete(op, *nr_pages);
- return -ENOBUFS;
-}
-
-/*
- * allocate a block in the cache in which to store a page
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if no buffers can be made available
- * - returns -ENOBUFS if page is beyond EOF
- * - otherwise:
- * - the metadata will be retained
- * - 0 will be returned
- */
-int cachefiles_allocate_page(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("%p,{%lx},", object, page->index);
-
- ret = cachefiles_has_space(cache, 0, 1);
- if (ret == 0)
- fscache_mark_page_cached(op, page);
- else
- ret = -ENOBUFS;
-
- fscache_retrieval_complete(op, 1);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * allocate blocks in the cache in which to store a set of pages
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if some buffers couldn't be made available
- * - returns -ENOBUFS if some pages are beyond EOF
- * - otherwise:
- * - -ENODATA will be returned
- * - metadata will be retained for any page marked
- */
-int cachefiles_allocate_pages(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct pagevec pagevec;
- struct page *page;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("%p,,,%d,", object, *nr_pages);
-
- ret = cachefiles_has_space(cache, 0, *nr_pages);
- if (ret == 0) {
- pagevec_init(&pagevec);
-
- list_for_each_entry(page, pages, lru) {
- if (pagevec_add(&pagevec, page) == 0)
- fscache_mark_pages_cached(op, &pagevec);
- }
-
- if (pagevec_count(&pagevec) > 0)
- fscache_mark_pages_cached(op, &pagevec);
- ret = -ENODATA;
- } else {
- ret = -ENOBUFS;
- }
-
- fscache_retrieval_complete(op, *nr_pages);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * request a page be stored in the cache
- * - cache withdrawal is prevented by the caller
- * - this request may be ignored if there's no cache block available, in which
- * case -ENOBUFS will be returned
- * - if the op is in progress, 0 will be returned
- */
-int cachefiles_write_page(struct fscache_storage *op, struct page *page)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct file *file;
- struct path path;
- loff_t pos, eof;
- size_t len;
- void *data;
- int ret = -ENOBUFS;
-
- ASSERT(op != NULL);
- ASSERT(page != NULL);
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
-
- _enter("%p,%p{%lx},,,", object, page, page->index);
-
- if (!object->backer) {
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
- }
-
- ASSERT(d_is_reg(object->backer));
-
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- pos = (loff_t)page->index << PAGE_SHIFT;
-
- /* We mustn't write more data than we have, so we have to beware of a
- * partial page at EOF.
- */
- eof = object->fscache.store_limit_l;
- if (pos >= eof)
- goto error;
-
- /* write the page to the backing filesystem and let it store it in its
- * own time */
- path.mnt = cache->mnt;
- path.dentry = object->backer;
- file = dentry_open(&path, O_RDWR | O_LARGEFILE, cache->cache_cred);
- if (IS_ERR(file)) {
- ret = PTR_ERR(file);
- goto error_2;
- }
-
- len = PAGE_SIZE;
- if (eof & ~PAGE_MASK) {
- if (eof - pos < PAGE_SIZE) {
- _debug("cut short %llx to %llx",
- pos, eof);
- len = eof - pos;
- ASSERTCMP(pos + len, ==, eof);
- }
- }
-
- data = kmap(page);
- ret = __kernel_write(file, data, len, &pos);
- kunmap(page);
- fput(file);
- if (ret != len)
- goto error_eio;
-
- _leave(" = 0");
- return 0;
-
-error_eio:
- ret = -EIO;
-error_2:
- if (ret == -EIO)
- cachefiles_io_error_obj(object,
- "Write page to backing file failed");
-error:
- _leave(" = -ENOBUFS [%d]", ret);
- return -ENOBUFS;
-}
-
-/*
- * detach a backing block from a page
- * - cache withdrawal is prevented by the caller
- */
-void cachefiles_uncache_page(struct fscache_object *_object, struct page *page)
- __releases(&object->fscache.cookie->lock)
-{
- struct cachefiles_object *object;
-
- object = container_of(_object, struct cachefiles_object, fscache);
-
- _enter("%p,{%lu}", object, page->index);
-
- spin_unlock(&object->fscache.cookie->lock);
-}
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index f78793f3d21e..7ec80456a700 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -382,12 +382,6 @@ void fscache_withdraw_cache(struct fscache_cache *cache)
cache->ops->sync_cache(cache);
fscache_stat_d(&fscache_n_cop_sync_cache);
- /* dissociate all the netfs pages backed by this cache from the block
- * mappings in the cache */
- fscache_stat(&fscache_n_cop_dissociate_pages);
- cache->ops->dissociate_pages(cache);
- fscache_stat_d(&fscache_n_cop_dissociate_pages);
-
/* we now have to destroy all the active objects pertaining to this
* cache - which we do by passing them off to thread pool to be
* disposed of */
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 477c5fb349ee..fa002ef7766d 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -191,13 +191,8 @@ struct fscache_cookie *fscache_alloc_cookie(
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
cookie->type = def->type;
spin_lock_init(&cookie->lock);
- spin_lock_init(&cookie->stores_lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
- /* radix tree insertion won't use the preallocation pool unless it's
- * told it may not wait */
- INIT_RADIX_TREE(&cookie->stores, GFP_NOFS & ~__GFP_DIRECT_RECLAIM);
-
write_lock(&fscache_cookies_lock);
list_add_tail(&cookie->proc_link, &fscache_cookies);
write_unlock(&fscache_cookies_lock);
@@ -786,10 +781,6 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
!atomic_read(&cookie->n_active));
}
- /* Make sure any pending writes are cancelled. */
- if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
- fscache_invalidate_writes(cookie);
-
/* Reset the cookie state if it wasn't relinquished */
if (!test_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags)) {
atomic_inc(&cookie->n_active);
@@ -838,7 +829,6 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
/* Clear pointers back to the netfs */
cookie->netfs_data = NULL;
cookie->def = NULL;
- BUG_ON(!radix_tree_empty(&cookie->stores));
if (cookie->parent) {
ASSERTCMP(atomic_read(&cookie->parent->usage), >, 0);
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 4b535c2dae4a..83dfbe0e3964 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -142,7 +142,6 @@ extern int fscache_wait_for_operation_activation(struct fscache_object *,
struct fscache_operation *,
atomic_t *,
atomic_t *);
-extern void fscache_invalidate_writes(struct fscache_cookie *);
/*
* proc.c
@@ -272,7 +271,6 @@ extern atomic_t fscache_n_cop_allocate_page;
extern atomic_t fscache_n_cop_allocate_pages;
extern atomic_t fscache_n_cop_write_page;
extern atomic_t fscache_n_cop_uncache_page;
-extern atomic_t fscache_n_cop_dissociate_pages;
extern atomic_t fscache_n_cache_no_space_reject;
extern atomic_t fscache_n_cache_stale_objects;
@@ -325,27 +323,6 @@ static inline void fscache_cookie_get(struct fscache_cookie *cookie,
trace_fscache_cookie(cookie, where, usage);
}
-/*
- * get an extra reference to a netfs retrieval context
- */
-static inline
-void *fscache_get_context(struct fscache_cookie *cookie, void *context)
-{
- if (cookie->def->get_context)
- cookie->def->get_context(cookie->netfs_data, context);
- return context;
-}
-
-/*
- * release a reference to a netfs retrieval context
- */
-static inline
-void fscache_put_context(struct fscache_cookie *cookie, void *context)
-{
- if (cookie->def->put_context)
- cookie->def->put_context(cookie->netfs_data, context);
-}
-
/*
* Update the auxiliary data on a cookie.
*/
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index cfeba839a0f2..6e4ade2f1e6f 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -973,14 +973,12 @@ static const struct fscache_state *_fscache_invalidate_object(struct fscache_obj
* retire the object instead.
*/
if (!fscache_use_cookie(object)) {
- ASSERT(radix_tree_empty(&object->cookie->stores));
set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
_leave(" [no cookie]");
return transit_to(KILL_OBJECT);
}
/* Reject any new read/write ops and abort any that are pending. */
- fscache_invalidate_writes(cookie);
clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
fscache_cancel_all_ops(object);
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index 26af6fdf1538..1beffb071205 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -13,174 +13,6 @@
#include <linux/slab.h>
#include "internal.h"
-/*
- * check to see if a page is being written to the cache
- */
-bool __fscache_check_page_write(struct fscache_cookie *cookie, struct page *page)
-{
- void *val;
-
- rcu_read_lock();
- val = radix_tree_lookup(&cookie->stores, page->index);
- rcu_read_unlock();
- trace_fscache_check_page(cookie, page, val, 0);
-
- return val != NULL;
-}
-EXPORT_SYMBOL(__fscache_check_page_write);
-
-/*
- * wait for a page to finish being written to the cache
- */
-void __fscache_wait_on_page_write(struct fscache_cookie *cookie, struct page *page)
-{
- wait_queue_head_t *wq = bit_waitqueue(&cookie->flags, 0);
-
- trace_fscache_page(cookie, page, fscache_page_write_wait);
-
- wait_event(*wq, !__fscache_check_page_write(cookie, page));
-}
-EXPORT_SYMBOL(__fscache_wait_on_page_write);
-
-/*
- * wait for a page to finish being written to the cache. Put a timeout here
- * since we might be called recursively via parent fs.
- */
-static
-bool release_page_wait_timeout(struct fscache_cookie *cookie, struct page *page)
-{
- wait_queue_head_t *wq = bit_waitqueue(&cookie->flags, 0);
-
- return wait_event_timeout(*wq, !__fscache_check_page_write(cookie, page),
- HZ);
-}
-
-/*
- * decide whether a page can be released, possibly by cancelling a store to it
- * - we're allowed to sleep if __GFP_DIRECT_RECLAIM is flagged
- */
-bool __fscache_maybe_release_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- struct page *xpage;
- void *val;
-
- _enter("%p,%p,%x", cookie, page, gfp);
-
- trace_fscache_page(cookie, page, fscache_page_maybe_release);
-
-try_again:
- rcu_read_lock();
- val = radix_tree_lookup(&cookie->stores, page->index);
- if (!val) {
- rcu_read_unlock();
- fscache_stat(&fscache_n_store_vmscan_not_storing);
- __fscache_uncache_page(cookie, page);
- return true;
- }
-
- /* see if the page is actually undergoing storage - if so we can't get
- * rid of it till the cache has finished with it */
- if (radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG)) {
- rcu_read_unlock();
- goto page_busy;
- }
-
- /* the page is pending storage, so we attempt to cancel the store and
- * discard the store request so that the page can be reclaimed */
- spin_lock(&cookie->stores_lock);
- rcu_read_unlock();
-
- if (radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG)) {
- /* the page started to undergo storage whilst we were looking,
- * so now we can only wait or return */
- spin_unlock(&cookie->stores_lock);
- goto page_busy;
- }
-
- xpage = radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- spin_unlock(&cookie->stores_lock);
-
- if (xpage) {
- fscache_stat(&fscache_n_store_vmscan_cancelled);
- fscache_stat(&fscache_n_store_radix_deletes);
- ASSERTCMP(xpage, ==, page);
- } else {
- fscache_stat(&fscache_n_store_vmscan_gone);
- }
-
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
- if (xpage)
- put_page(xpage);
- __fscache_uncache_page(cookie, page);
- return true;
-
-page_busy:
- /* We will wait here if we're allowed to, but that could deadlock the
- * allocator as the work threads writing to the cache may all end up
- * sleeping on memory allocation, so we may need to impose a timeout
- * too. */
- if (!(gfp & __GFP_DIRECT_RECLAIM) || !(gfp & __GFP_FS)) {
- fscache_stat(&fscache_n_store_vmscan_busy);
- return false;
- }
-
- fscache_stat(&fscache_n_store_vmscan_wait);
- if (!release_page_wait_timeout(cookie, page))
- _debug("fscache writeout timeout page: %p{%lx}",
- page, page->index);
-
- gfp &= ~__GFP_DIRECT_RECLAIM;
- goto try_again;
-}
-EXPORT_SYMBOL(__fscache_maybe_release_page);
-
-/*
- * note that a page has finished being written to the cache
- */
-static void fscache_end_page_write(struct fscache_object *object,
- struct page *page)
-{
- struct fscache_cookie *cookie;
- struct page *xpage = NULL, *val;
-
- spin_lock(&object->lock);
- cookie = object->cookie;
- if (cookie) {
- /* delete the page from the tree if it is now no longer
- * pending */
- spin_lock(&cookie->stores_lock);
- radix_tree_tag_clear(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_clear_store);
- if (!radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG)) {
- fscache_stat(&fscache_n_store_radix_deletes);
- xpage = radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- trace_fscache_page(cookie, page, fscache_page_write_end);
-
- val = radix_tree_lookup(&cookie->stores, page->index);
- trace_fscache_check_page(cookie, page, val, 1);
- } else {
- trace_fscache_page(cookie, page, fscache_page_write_end_pend);
- }
- spin_unlock(&cookie->stores_lock);
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
- } else {
- trace_fscache_page(cookie, page, fscache_page_write_end_noc);
- }
- spin_unlock(&object->lock);
- if (xpage)
- put_page(xpage);
-}
-
/*
* actually apply the changed attributes to a cache object
*/
@@ -265,76 +97,6 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
}
EXPORT_SYMBOL(__fscache_attr_changed);
-/*
- * Handle cancellation of a pending retrieval op
- */
-static void fscache_do_cancel_retrieval(struct fscache_operation *_op)
-{
- struct fscache_retrieval *op =
- container_of(_op, struct fscache_retrieval, op);
-
- atomic_set(&op->n_pages, 0);
-}
-
-/*
- * release a retrieval op reference
- */
-static void fscache_release_retrieval_op(struct fscache_operation *_op)
-{
- struct fscache_retrieval *op =
- container_of(_op, struct fscache_retrieval, op);
-
- _enter("{OP%x}", op->op.debug_id);
-
- ASSERTIFCMP(op->op.state != FSCACHE_OP_ST_INITIALISED,
- atomic_read(&op->n_pages), ==, 0);
-
- fscache_hist(fscache_retrieval_histogram, op->start_time);
- if (op->context)
- fscache_put_context(op->cookie, op->context);
-
- _leave("");
-}
-
-/*
- * allocate a retrieval op
- */
-static struct fscache_retrieval *fscache_alloc_retrieval(
- struct fscache_cookie *cookie,
- struct address_space *mapping,
- fscache_rw_complete_t end_io_func,
- void *context)
-{
- struct fscache_retrieval *op;
-
- /* allocate a retrieval operation and attempt to submit it */
- op = kzalloc(sizeof(*op), GFP_NOIO);
- if (!op) {
- fscache_stat(&fscache_n_retrievals_nomem);
- return NULL;
- }
-
- fscache_operation_init(cookie, &op->op, NULL,
- fscache_do_cancel_retrieval,
- fscache_release_retrieval_op);
- op->op.flags = FSCACHE_OP_MYTHREAD |
- (1UL << FSCACHE_OP_WAITING) |
- (1UL << FSCACHE_OP_UNUSE_COOKIE);
- op->cookie = cookie;
- op->mapping = mapping;
- op->end_io_func = end_io_func;
- op->context = context;
- op->start_time = jiffies;
- INIT_LIST_HEAD(&op->to_do);
-
- /* Pin the netfs read context in case we need to do the actual netfs
- * read because we've encountered a cache read failure.
- */
- if (context)
- fscache_get_context(op->cookie, context);
- return op;
-}
-
/*
* wait for a deferred lookup to complete
*/
@@ -416,833 +178,3 @@ int fscache_wait_for_operation_activation(struct fscache_object *object,
}
return 0;
}
-
-/*
- * read a page from the cache or allocate a block in which to store it
- * - we return:
- * -ENOMEM - out of memory, nothing done
- * -ERESTARTSYS - interrupted
- * -ENOBUFS - no backing object available in which to cache the block
- * -ENODATA - no data available in the backing object for this block
- * 0 - dispatched a read - it'll call end_io_func() when finished
- */
-int __fscache_read_or_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%p,,,", cookie, page);
-
- fscache_stat(&fscache_n_retrievals);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, page->mapping,
- end_io_func, context);
- if (!op) {
- _leave(" = -ENOMEM");
- return -ENOMEM;
- }
- atomic_set(&op->n_pages, 1);
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_retr_one);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- ASSERT(test_bit(FSCACHE_OBJECT_IS_LOOKED_UP, &object->flags));
-
- __fscache_use_cookie(cookie);
- atomic_inc(&object->n_reads);
- __set_bit(FSCACHE_OP_DEC_READ_CNT, &op->op.flags);
-
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_retrieval_ops);
-
- /* we wait for the operation to become active, and then process it
- * *here*, in this thread, and not in the thread pool */
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_retrieval_op_waits),
- __fscache_stat(&fscache_n_retrievals_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- if (test_bit(FSCACHE_COOKIE_NO_DATA_YET, &object->cookie->flags)) {
- fscache_stat(&fscache_n_cop_allocate_page);
- ret = object->cache->ops->allocate_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_page);
- if (ret == 0)
- ret = -ENODATA;
- } else {
- fscache_stat(&fscache_n_cop_read_or_alloc_page);
- ret = object->cache->ops->read_or_alloc_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_read_or_alloc_page);
- }
-
-error:
- if (ret == -ENOMEM)
- fscache_stat(&fscache_n_retrievals_nomem);
- else if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_retrievals_intr);
- else if (ret == -ENODATA)
- fscache_stat(&fscache_n_retrievals_nodata);
- else if (ret < 0)
- fscache_stat(&fscache_n_retrievals_nobufs);
- else
- fscache_stat(&fscache_n_retrievals_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- atomic_dec(&object->n_reads);
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- fscache_put_retrieval(op);
-nobufs:
- fscache_stat(&fscache_n_retrievals_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_read_or_alloc_page);
-
-/*
- * read a list of page from the cache or allocate a block in which to store
- * them
- * - we return:
- * -ENOMEM - out of memory, some pages may be being read
- * -ERESTARTSYS - interrupted, some pages may be being read
- * -ENOBUFS - no backing object or space available in which to cache any
- * pages not being read
- * -ENODATA - no data available in the backing object for some or all of
- * the pages
- * 0 - dispatched a read on all pages
- *
- * end_io_func() will be called for each page read from the cache as it is
- * finishes being read
- *
- * any pages for which a read is dispatched will be removed from pages and
- * nr_pages
- */
-int __fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
- struct address_space *mapping,
- struct list_head *pages,
- unsigned *nr_pages,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,,%d,,,", cookie, *nr_pages);
-
- fscache_stat(&fscache_n_retrievals);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(*nr_pages, >, 0);
- ASSERT(!list_empty(pages));
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, mapping, end_io_func, context);
- if (!op)
- return -ENOMEM;
- atomic_set(&op->n_pages, *nr_pages);
- trace_fscache_page_op(cookie, NULL, &op->op, fscache_page_op_retr_multi);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- __fscache_use_cookie(cookie);
- atomic_inc(&object->n_reads);
- __set_bit(FSCACHE_OP_DEC_READ_CNT, &op->op.flags);
-
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_retrieval_ops);
-
- /* we wait for the operation to become active, and then process it
- * *here*, in this thread, and not in the thread pool */
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_retrieval_op_waits),
- __fscache_stat(&fscache_n_retrievals_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- if (test_bit(FSCACHE_COOKIE_NO_DATA_YET, &object->cookie->flags)) {
- fscache_stat(&fscache_n_cop_allocate_pages);
- ret = object->cache->ops->allocate_pages(
- op, pages, nr_pages, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_pages);
- } else {
- fscache_stat(&fscache_n_cop_read_or_alloc_pages);
- ret = object->cache->ops->read_or_alloc_pages(
- op, pages, nr_pages, gfp);
- fscache_stat_d(&fscache_n_cop_read_or_alloc_pages);
- }
-
-error:
- if (ret == -ENOMEM)
- fscache_stat(&fscache_n_retrievals_nomem);
- else if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_retrievals_intr);
- else if (ret == -ENODATA)
- fscache_stat(&fscache_n_retrievals_nodata);
- else if (ret < 0)
- fscache_stat(&fscache_n_retrievals_nobufs);
- else
- fscache_stat(&fscache_n_retrievals_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- atomic_dec(&object->n_reads);
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- fscache_put_retrieval(op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
-nobufs:
- fscache_stat(&fscache_n_retrievals_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_read_or_alloc_pages);
-
-/*
- * allocate a block in the cache on which to store a page
- * - we return:
- * -ENOMEM - out of memory, nothing done
- * -ERESTARTSYS - interrupted
- * -ENOBUFS - no backing object available in which to cache the block
- * 0 - block allocated
- */
-int __fscache_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%p,,,", cookie, page);
-
- fscache_stat(&fscache_n_allocs);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, page->mapping, NULL, NULL);
- if (!op)
- return -ENOMEM;
- atomic_set(&op->n_pages, 1);
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_alloc_one);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_alloc_ops);
-
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_alloc_op_waits),
- __fscache_stat(&fscache_n_allocs_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- fscache_stat(&fscache_n_cop_allocate_page);
- ret = object->cache->ops->allocate_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_page);
-
-error:
- if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_allocs_intr);
- else if (ret < 0)
- fscache_stat(&fscache_n_allocs_nobufs);
- else
- fscache_stat(&fscache_n_allocs_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- fscache_put_retrieval(op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
-nobufs:
- fscache_stat(&fscache_n_allocs_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_alloc_page);
-
-/*
- * Unmark pages allocate in the readahead code path (via:
- * fscache_readpages_or_alloc) after delegating to the base filesystem
- */
-void __fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages)
-{
- struct page *page;
-
- list_for_each_entry(page, pages, lru) {
- if (PageFsCache(page))
- __fscache_uncache_page(cookie, page);
- }
-}
-EXPORT_SYMBOL(__fscache_readpages_cancel);
-
-/*
- * release a write op reference
- */
-static void fscache_release_write_op(struct fscache_operation *_op)
-{
- _enter("{OP%x}", _op->debug_id);
-}
-
-/*
- * perform the background storage of a page into the cache
- */
-static void fscache_write_op(struct fscache_operation *_op)
-{
- struct fscache_storage *op =
- container_of(_op, struct fscache_storage, op);
- struct fscache_object *object = op->op.object;
- struct fscache_cookie *cookie;
- struct page *page;
- unsigned n;
- void *results[1];
- int ret;
-
- _enter("{OP%x,%d}", op->op.debug_id, atomic_read(&op->op.usage));
-
-again:
- spin_lock(&object->lock);
- cookie = object->cookie;
-
- if (!fscache_object_is_active(object)) {
- /* If we get here, then the on-disk cache object likely no
- * longer exists, so we should just cancel this write
- * operation.
- */
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, true);
- _leave(" [inactive]");
- return;
- }
-
- if (!cookie) {
- /* If we get here, then the cookie belonging to the object was
- * detached, probably by the cookie being withdrawn due to
- * memory pressure, which means that the pages we might write
- * to the cache from no longer exist - therefore, we can just
- * cancel this write operation.
- */
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, true);
- _leave(" [cancel] op{f=%lx s=%u} obj{s=%s f=%lx}",
- _op->flags, _op->state, object->state->short_name,
- object->flags);
- return;
- }
-
- spin_lock(&cookie->stores_lock);
-
- fscache_stat(&fscache_n_store_calls);
-
- /* find a page to store */
- results[0] = NULL;
- page = NULL;
- n = radix_tree_gang_lookup_tag(&cookie->stores, results, 0, 1,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_gang_lookup(cookie, &op->op, results, n, op->store_limit);
- if (n != 1)
- goto superseded;
- page = results[0];
- _debug("gang %d [%lx]", n, page->index);
-
- radix_tree_tag_set(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG);
- radix_tree_tag_clear(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_pend2store);
-
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-
- if (page->index >= op->store_limit)
- goto discard_page;
-
- fscache_stat(&fscache_n_store_pages);
- fscache_stat(&fscache_n_cop_write_page);
- ret = object->cache->ops->write_page(op, page);
- fscache_stat_d(&fscache_n_cop_write_page);
- trace_fscache_wrote_page(cookie, page, &op->op, ret);
- fscache_end_page_write(object, page);
- if (ret < 0) {
- fscache_abort_object(object);
- fscache_op_complete(&op->op, true);
- } else {
- fscache_enqueue_operation(&op->op);
- }
-
- _leave("");
- return;
-
-discard_page:
- fscache_stat(&fscache_n_store_pages_over_limit);
- trace_fscache_wrote_page(cookie, page, &op->op, -ENOBUFS);
- fscache_end_page_write(object, page);
- goto again;
-
-superseded:
- /* this writer is going away and there aren't any more things to
- * write */
- _debug("cease");
- spin_unlock(&cookie->stores_lock);
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, false);
- _leave("");
-}
-
-/*
- * Clear the pages pending writing for invalidation
- */
-void fscache_invalidate_writes(struct fscache_cookie *cookie)
-{
- struct page *page;
- void *results[16];
- int n, i;
-
- _enter("");
-
- for (;;) {
- spin_lock(&cookie->stores_lock);
- n = radix_tree_gang_lookup_tag(&cookie->stores, results, 0,
- ARRAY_SIZE(results),
- FSCACHE_COOKIE_PENDING_TAG);
- if (n == 0) {
- spin_unlock(&cookie->stores_lock);
- break;
- }
-
- for (i = n - 1; i >= 0; i--) {
- page = results[i];
- radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- trace_fscache_page(cookie, page, fscache_page_inval);
- }
-
- spin_unlock(&cookie->stores_lock);
-
- for (i = n - 1; i >= 0; i--)
- put_page(results[i]);
- }
-
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
-
- _leave("");
-}
-
-/*
- * request a page be stored in the cache
- * - returns:
- * -ENOMEM - out of memory, nothing done
- * -ENOBUFS - no backing object available in which to cache the page
- * 0 - dispatched a write - it'll call end_io_func() when finished
- *
- * if the cookie still has a backing object at this point, that object can be
- * in one of a few states with respect to storage processing:
- *
- * (1) negative lookup, object not yet created (FSCACHE_COOKIE_CREATING is
- * set)
- *
- * (a) no writes yet
- *
- * (b) writes deferred till post-creation (mark page for writing and
- * return immediately)
- *
- * (2) negative lookup, object created, initial fill being made from netfs
- *
- * (a) fill point not yet reached this page (mark page for writing and
- * return)
- *
- * (b) fill point passed this page (queue op to store this page)
- *
- * (3) object extant (queue op to store this page)
- *
- * any other state is invalid
- */
-int __fscache_write_page(struct fscache_cookie *cookie,
- struct page *page,
- loff_t object_size,
- gfp_t gfp)
-{
- struct fscache_storage *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%x,", cookie, (u32) page->flags);
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERT(PageFsCache(page));
-
- fscache_stat(&fscache_n_stores);
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- op = kzalloc(sizeof(*op), GFP_NOIO | __GFP_NOMEMALLOC | __GFP_NORETRY);
- if (!op)
- goto nomem;
-
- fscache_operation_init(cookie, &op->op, fscache_write_op, NULL,
- fscache_release_write_op);
- op->op.flags = FSCACHE_OP_ASYNC |
- (1 << FSCACHE_OP_WAITING) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
-
- ret = radix_tree_maybe_preload(gfp & ~__GFP_HIGHMEM);
- if (ret < 0)
- goto nomem_free;
-
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_write_one);
-
- ret = -ENOBUFS;
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
- if (test_bit(FSCACHE_IOERROR, &object->cache->flags))
- goto nobufs;
-
- trace_fscache_page(cookie, page, fscache_page_write);
-
- /* add the page to the pending-storage radix tree on the backing
- * object */
- spin_lock(&object->lock);
-
- if (object->store_limit_l != object_size)
- fscache_set_store_limit(object, object_size);
-
- spin_lock(&cookie->stores_lock);
-
- _debug("store limit %llx", (unsigned long long) object->store_limit);
-
- ret = radix_tree_insert(&cookie->stores, page->index, page);
- if (ret < 0) {
- if (ret == -EEXIST)
- goto already_queued;
- _debug("insert failed %d", ret);
- goto nobufs_unlock_obj;
- }
-
- trace_fscache_page(cookie, page, fscache_page_radix_insert);
- radix_tree_tag_set(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_set_pend);
- get_page(page);
-
- /* we only want one writer at a time, but we do need to queue new
- * writers after exclusive ops */
- if (test_and_set_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags))
- goto already_pending;
-
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-
- op->op.debug_id = atomic_inc_return(&fscache_op_debug_id);
- op->store_limit = object->store_limit;
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, &op->op) < 0)
- goto submit_failed;
-
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_stat(&fscache_n_store_ops);
- fscache_stat(&fscache_n_stores_ok);
-
- /* the work queue now carries its own ref on the object */
- fscache_put_operation(&op->op);
- _leave(" = 0");
- return 0;
-
-already_queued:
- fscache_stat(&fscache_n_stores_again);
-already_pending:
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_put_operation(&op->op);
- fscache_stat(&fscache_n_stores_ok);
- _leave(" = 0");
- return 0;
-
-submit_failed:
- spin_lock(&cookie->stores_lock);
- radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- spin_unlock(&cookie->stores_lock);
- wake_cookie = __fscache_unuse_cookie(cookie);
- put_page(page);
- ret = -ENOBUFS;
- goto nobufs;
-
-nobufs_unlock_obj:
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-nobufs:
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_put_operation(&op->op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- fscache_stat(&fscache_n_stores_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-
-nomem_free:
- fscache_put_operation(&op->op);
-nomem:
- fscache_stat(&fscache_n_stores_oom);
- _leave(" = -ENOMEM");
- return -ENOMEM;
-}
-EXPORT_SYMBOL(__fscache_write_page);
-
-/*
- * remove a page from the cache
- */
-void __fscache_uncache_page(struct fscache_cookie *cookie, struct page *page)
-{
- struct fscache_object *object;
-
- _enter(",%p", page);
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- fscache_stat(&fscache_n_uncaches);
-
- /* cache withdrawal may beat us to it */
- if (!PageFsCache(page))
- goto done;
-
- trace_fscache_page(cookie, page, fscache_page_uncache);
-
- /* get the object */
- spin_lock(&cookie->lock);
-
- if (hlist_empty(&cookie->backing_objects)) {
- ClearPageFsCache(page);
- goto done_unlock;
- }
-
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- /* there might now be stuff on disk we could read */
- clear_bit(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
-
- /* only invoke the cache backend if we managed to mark the page
- * uncached here; this deals with synchronisation vs withdrawal */
- if (TestClearPageFsCache(page) &&
- object->cache->ops->uncache_page) {
- /* the cache backend releases the cookie lock */
- fscache_stat(&fscache_n_cop_uncache_page);
- object->cache->ops->uncache_page(object, page);
- fscache_stat_d(&fscache_n_cop_uncache_page);
- goto done;
- }
-
-done_unlock:
- spin_unlock(&cookie->lock);
-done:
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_uncache_page);
-
-/**
- * fscache_mark_page_cached - Mark a page as being cached
- * @op: The retrieval op pages are being marked for
- * @page: The page to be marked
- *
- * Mark a netfs page as being cached. After this is called, the netfs
- * must call fscache_uncache_page() to remove the mark.
- */
-void fscache_mark_page_cached(struct fscache_retrieval *op, struct page *page)
-{
- struct fscache_cookie *cookie = op->op.object->cookie;
-
-#ifdef CONFIG_FSCACHE_STATS
- atomic_inc(&fscache_n_marks);
-#endif
-
- trace_fscache_page(cookie, page, fscache_page_cached);
-
- _debug("- mark %p{%lx}", page, page->index);
- if (TestSetPageFsCache(page)) {
- static bool once_only;
- if (!once_only) {
- once_only = true;
- pr_warn("Cookie type %s marked page %lx multiple times\n",
- cookie->def->name, page->index);
- }
- }
-
- if (cookie->def->mark_page_cached)
- cookie->def->mark_page_cached(cookie->netfs_data,
- op->mapping, page);
-}
-EXPORT_SYMBOL(fscache_mark_page_cached);
-
-/**
- * fscache_mark_pages_cached - Mark pages as being cached
- * @op: The retrieval op pages are being marked for
- * @pagevec: The pages to be marked
- *
- * Mark a bunch of netfs pages as being cached. After this is called,
- * the netfs must call fscache_uncache_page() to remove the mark.
- */
-void fscache_mark_pages_cached(struct fscache_retrieval *op,
- struct pagevec *pagevec)
-{
- unsigned long loop;
-
- for (loop = 0; loop < pagevec->nr; loop++)
- fscache_mark_page_cached(op, pagevec->pages[loop]);
-
- pagevec_reinit(pagevec);
-}
-EXPORT_SYMBOL(fscache_mark_pages_cached);
-
-/*
- * Uncache all the pages in an inode that are marked PG_fscache, assuming them
- * to be associated with the given cookie.
- */
-void __fscache_uncache_all_inode_pages(struct fscache_cookie *cookie,
- struct inode *inode)
-{
- struct address_space *mapping = inode->i_mapping;
- struct pagevec pvec;
- pgoff_t next;
- int i;
-
- _enter("%p,%p", cookie, inode);
-
- if (!mapping || mapping->nrpages == 0) {
- _leave(" [no pages]");
- return;
- }
-
- pagevec_init(&pvec);
- next = 0;
- do {
- if (!pagevec_lookup(&pvec, mapping, &next))
- break;
- for (i = 0; i < pagevec_count(&pvec); i++) {
- struct page *page = pvec.pages[i];
- if (PageFsCache(page)) {
- __fscache_wait_on_page_write(cookie, page);
- __fscache_uncache_page(cookie, page);
- }
- }
- pagevec_release(&pvec);
- cond_resched();
- } while (next);
-
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_uncache_all_inode_pages);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index a5aa93ece8c5..281022871e70 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -124,7 +124,6 @@ atomic_t fscache_n_cop_allocate_page;
atomic_t fscache_n_cop_allocate_pages;
atomic_t fscache_n_cop_write_page;
atomic_t fscache_n_cop_uncache_page;
-atomic_t fscache_n_cop_dissociate_pages;
atomic_t fscache_n_cache_no_space_reject;
atomic_t fscache_n_cache_stale_objects;
@@ -265,14 +264,13 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
atomic_read(&fscache_n_cop_sync_cache));
- seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d dsp=%d\n",
+ seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d\n",
atomic_read(&fscache_n_cop_read_or_alloc_page),
atomic_read(&fscache_n_cop_read_or_alloc_pages),
atomic_read(&fscache_n_cop_allocate_page),
atomic_read(&fscache_n_cop_allocate_pages),
atomic_read(&fscache_n_cop_write_page),
- atomic_read(&fscache_n_cop_uncache_page),
- atomic_read(&fscache_n_cop_dissociate_pages));
+ atomic_read(&fscache_n_cop_uncache_page));
seq_printf(m, "CacheEv: nsp=%d stl=%d rtr=%d cul=%d\n",
atomic_read(&fscache_n_cache_no_space_reject),
atomic_read(&fscache_n_cache_stale_objects),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index d5ba431b5d63..2db4ffbf4523 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -137,88 +137,6 @@ extern void fscache_operation_init(struct fscache_cookie *,
fscache_operation_cancel_t,
fscache_operation_release_t);
-/*
- * data read operation
- */
-struct fscache_retrieval {
- struct fscache_operation op;
- struct fscache_cookie *cookie; /* The netfs cookie */
- struct address_space *mapping; /* netfs pages */
- fscache_rw_complete_t end_io_func; /* function to call on I/O completion */
- void *context; /* netfs read context (pinned) */
- struct list_head to_do; /* list of things to be done by the backend */
- unsigned long start_time; /* time at which retrieval started */
- atomic_t n_pages; /* number of pages to be retrieved */
-};
-
-typedef int (*fscache_page_retrieval_func_t)(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp);
-
-typedef int (*fscache_pages_retrieval_func_t)(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp);
-
-/**
- * fscache_get_retrieval - Get an extra reference on a retrieval operation
- * @op: The retrieval operation to get a reference on
- *
- * Get an extra reference on a retrieval operation.
- */
-static inline
-struct fscache_retrieval *fscache_get_retrieval(struct fscache_retrieval *op)
-{
- atomic_inc(&op->op.usage);
- return op;
-}
-
-/**
- * fscache_enqueue_retrieval - Enqueue a retrieval operation for processing
- * @op: The retrieval operation affected
- *
- * Enqueue a retrieval operation for processing by the FS-Cache thread pool.
- */
-static inline void fscache_enqueue_retrieval(struct fscache_retrieval *op)
-{
- fscache_enqueue_operation(&op->op);
-}
-
-/**
- * fscache_retrieval_complete - Record (partial) completion of a retrieval
- * @op: The retrieval operation affected
- * @n_pages: The number of pages to account for
- */
-static inline void fscache_retrieval_complete(struct fscache_retrieval *op,
- int n_pages)
-{
- if (atomic_sub_return_relaxed(n_pages, &op->n_pages) <= 0)
- fscache_op_complete(&op->op, false);
-}
-
-/**
- * fscache_put_retrieval - Drop a reference to a retrieval operation
- * @op: The retrieval operation affected
- *
- * Drop a reference to a retrieval operation.
- */
-static inline void fscache_put_retrieval(struct fscache_retrieval *op)
-{
- fscache_put_operation(&op->op);
-}
-
-/*
- * cached page storage work item
- * - used to do three things:
- * - batch writes to the cache
- * - do cache writes asynchronously
- * - defer writes until cache object lookup completion
- */
-struct fscache_storage {
- struct fscache_operation op;
- pgoff_t store_limit; /* don't write more than this */
-};
-
/*
* cache operations
*/
@@ -275,35 +193,6 @@ struct fscache_cache_ops {
/* reserve space for an object's data and associated metadata */
int (*reserve_space)(struct fscache_object *object, loff_t i_size);
-
- /* request a backing block for a page be read or allocated in the
- * cache */
- fscache_page_retrieval_func_t read_or_alloc_page;
-
- /* request backing blocks for a list of pages be read or allocated in
- * the cache */
- fscache_pages_retrieval_func_t read_or_alloc_pages;
-
- /* request a backing block for a page be allocated in the cache so that
- * it can be written directly */
- fscache_page_retrieval_func_t allocate_page;
-
- /* request backing blocks for pages be allocated in the cache so that
- * they can be written directly */
- fscache_pages_retrieval_func_t allocate_pages;
-
- /* write a page to its backing block in the cache */
- int (*write_page)(struct fscache_storage *op, struct page *page);
-
- /* detach backing block from a page (optional)
- * - must release the cookie lock before returning
- * - may sleep
- */
- void (*uncache_page)(struct fscache_object *object,
- struct page *page);
-
- /* dissociate a cache from all the pages it was backing */
- void (*dissociate_pages)(struct fscache_cache *cache);
};
extern struct fscache_cookie fscache_fsdef_index;
@@ -466,21 +355,6 @@ void fscache_set_store_limit(struct fscache_object *object, loff_t i_size)
object->store_limit++;
}
-/**
- * fscache_end_io - End a retrieval operation on a page
- * @op: The FS-Cache operation covering the retrieval
- * @page: The page that was to be fetched
- * @error: The error code (0 if successful)
- *
- * Note the end of an operation to retrieve a page, as covered by a particular
- * operation record.
- */
-static inline void fscache_end_io(struct fscache_retrieval *op,
- struct page *page, int error)
-{
- op->end_io_func(page, op->context, error);
-}
-
static inline void __fscache_use_cookie(struct fscache_cookie *cookie)
{
atomic_inc(&cookie->n_active);
@@ -538,12 +412,6 @@ extern void fscache_withdraw_cache(struct fscache_cache *cache);
extern void fscache_io_error(struct fscache_cache *cache);
-extern void fscache_mark_page_cached(struct fscache_retrieval *op,
- struct page *page);
-
-extern void fscache_mark_pages_cached(struct fscache_retrieval *op,
- struct pagevec *pagevec);
-
extern bool fscache_object_sleep_till_congested(signed long *timeoutp);
extern enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
diff --git a/include/linux/fscache-obsolete.h b/include/linux/fscache-obsolete.h
new file mode 100644
index 000000000000..8d6d3a3b0d0a
--- /dev/null
+++ b/include/linux/fscache-obsolete.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* Obsolete fscache bits
+ *
+ * Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_FSCACHE_OBSOLETE_H
+#define _LINUX_FSCACHE_OBSOLETE_H
+
+
+
+#endif /* _LINUX_FSCACHE_OBSOLETE_H */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 38ec80217899..029a2ddd9d93 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -47,10 +47,6 @@ struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
-typedef void (*fscache_rw_complete_t)(struct page *page,
- void *context,
- int error);
-
/* result of index entry consultation */
enum fscache_checkaux {
FSCACHE_CHECKAUX_OKAY, /* entry okay as is */
@@ -89,27 +85,6 @@ struct fscache_cookie_def {
const void *data,
uint16_t datalen,
loff_t object_size);
-
- /* get an extra reference on a read context
- * - this function can be absent if the completion function doesn't
- * require a context
- */
- void (*get_context)(void *cookie_netfs_data, void *context);
-
- /* release an extra reference on a read context
- * - this function can be absent if the completion function doesn't
- * require a context
- */
- void (*put_context)(void *cookie_netfs_data, void *context);
-
- /* indicate page that now have cache metadata retained
- * - this function should mark the specified page as now being cached
- * - the page will have been marked with PG_fscache before this is
- * called, so this is optional
- */
- void (*mark_page_cached)(void *cookie_netfs_data,
- struct address_space *mapping,
- struct page *page);
};
/*
@@ -136,16 +111,12 @@ struct fscache_cookie {
atomic_t n_active; /* number of active users of netfs ptrs */
unsigned int debug_id;
spinlock_t lock;
- spinlock_t stores_lock; /* lock on page store tree */
struct hlist_head backing_objects; /* object(s) backing this file/index */
const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
void *netfs_data; /* back pointer to netfs */
- struct radix_tree_root stores; /* pages to be stored on this cookie */
-#define FSCACHE_COOKIE_PENDING_TAG 0 /* pages tag: pending write to cache */
-#define FSCACHE_COOKIE_STORING_TAG 1 /* pages tag: writing to cache */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
@@ -202,29 +173,6 @@ extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
extern int __fscache_attr_changed(struct fscache_cookie *);
extern void __fscache_invalidate(struct fscache_cookie *);
extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
-extern int __fscache_read_or_alloc_page(struct fscache_cookie *,
- struct page *,
- fscache_rw_complete_t,
- void *,
- gfp_t);
-extern int __fscache_read_or_alloc_pages(struct fscache_cookie *,
- struct address_space *,
- struct list_head *,
- unsigned *,
- fscache_rw_complete_t,
- void *,
- gfp_t);
-extern int __fscache_alloc_page(struct fscache_cookie *, struct page *, gfp_t);
-extern int __fscache_write_page(struct fscache_cookie *, struct page *, loff_t, gfp_t);
-extern void __fscache_uncache_page(struct fscache_cookie *, struct page *);
-extern bool __fscache_check_page_write(struct fscache_cookie *, struct page *);
-extern void __fscache_wait_on_page_write(struct fscache_cookie *, struct page *);
-extern bool __fscache_maybe_release_page(struct fscache_cookie *, struct page *,
- gfp_t);
-extern void __fscache_uncache_all_inode_pages(struct fscache_cookie *,
- struct inode *);
-extern void __fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages);
extern void __fscache_disable_cookie(struct fscache_cookie *, const void *, bool);
extern void __fscache_enable_cookie(struct fscache_cookie *, const void *, loff_t,
bool (*)(void *), void *);
@@ -491,303 +439,6 @@ void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
__fscache_wait_on_invalidate(cookie);
}
-/**
- * fscache_reserve_space - Reserve data space for a cached object
- * @cookie: The cookie representing the cache object
- * @i_size: The amount of space to be reserved
- *
- * Reserve an amount of space in the cache for the cache object attached to a
- * cookie so that a write to that object within the space can always be
- * honoured.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_reserve_space(struct fscache_cookie *cookie, loff_t size)
-{
- return -ENOBUFS;
-}
-
-/**
- * fscache_read_or_alloc_page - Read a page from the cache or allocate a block
- * in which to store it
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to fill if possible
- * @end_io_func: The callback to invoke when and if the page is filled
- * @context: An arbitrary piece of data to pass on to end_io_func()
- * @gfp: The conditions under which memory allocation should be made
- *
- * Read a page from the cache, or if that's not possible make a potential
- * one-block reservation in the cache into which the page may be stored once
- * fetched from the server.
- *
- * If the page is not backed by the cache object, or if it there's some reason
- * it can't be, -ENOBUFS will be returned and nothing more will be done for
- * that page.
- *
- * Else, if that page is backed by the cache, a read will be initiated directly
- * to the netfs's page and 0 will be returned by this function. The
- * end_io_func() callback will be invoked when the operation terminates on a
- * completion or failure. Note that the callback may be invoked before the
- * return.
- *
- * Else, if the page is unbacked, -ENODATA is returned and a block may have
- * been allocated in the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_read_or_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_read_or_alloc_page(cookie, page, end_io_func,
- context, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_read_or_alloc_pages - Read pages from the cache and/or allocate
- * blocks in which to store them
- * @cookie: The cookie representing the cache object
- * @mapping: The netfs inode mapping to which the pages will be attached
- * @pages: A list of potential netfs pages to be filled
- * @nr_pages: Number of pages to be read and/or allocated
- * @end_io_func: The callback to invoke when and if each page is filled
- * @context: An arbitrary piece of data to pass on to end_io_func()
- * @gfp: The conditions under which memory allocation should be made
- *
- * Read a set of pages from the cache, or if that's not possible, attempt to
- * make a potential one-block reservation for each page in the cache into which
- * that page may be stored once fetched from the server.
- *
- * If some pages are not backed by the cache object, or if it there's some
- * reason they can't be, -ENOBUFS will be returned and nothing more will be
- * done for that pages.
- *
- * Else, if some of the pages are backed by the cache, a read will be initiated
- * directly to the netfs's page and 0 will be returned by this function. The
- * end_io_func() callback will be invoked when the operation terminates on a
- * completion or failure. Note that the callback may be invoked before the
- * return.
- *
- * Else, if a page is unbacked, -ENODATA is returned and a block may have
- * been allocated in the cache.
- *
- * Because the function may want to return all of -ENOBUFS, -ENODATA and 0 in
- * regard to different pages, the return values are prioritised in that order.
- * Any pages submitted for reading are removed from the pages list.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
- struct address_space *mapping,
- struct list_head *pages,
- unsigned *nr_pages,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_read_or_alloc_pages(cookie, mapping, pages,
- nr_pages, end_io_func,
- context, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_alloc_page - Allocate a block in which to store a page
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to allocate a page for
- * @gfp: The conditions under which memory allocation should be made
- *
- * Request Allocation a block in the cache in which to store a netfs page
- * without retrieving any contents from the cache.
- *
- * If the page is not backed by a file then -ENOBUFS will be returned and
- * nothing more will be done, and no reservation will be made.
- *
- * Else, a block will be allocated if one wasn't already, and 0 will be
- * returned
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_alloc_page(cookie, page, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_readpages_cancel - Cancel read/alloc on pages
- * @cookie: The cookie representing the inode's cache object.
- * @pages: The netfs pages that we canceled write on in readpages()
- *
- * Uncache/unreserve the pages reserved earlier in readpages() via
- * fscache_readpages_or_alloc() and similar. In most successful caches in
- * readpages() this doesn't do anything. In cases when the underlying netfs's
- * readahead failed we need to clean up the pagelist (unmark and uncache).
- *
- * This function may sleep as it may have to clean up disk state.
- */
-static inline
-void fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_readpages_cancel(cookie, pages);
-}
-
-/**
- * fscache_write_page - Request storage of a page in the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to store
- * @object_size: Updated size of object
- * @gfp: The conditions under which memory allocation should be made
- *
- * Request the contents of the netfs page be written into the cache. This
- * request may be ignored if no cache block is currently allocated, in which
- * case it will return -ENOBUFS.
- *
- * If a cache block was already allocated, a write will be initiated and 0 will
- * be returned. The PG_fscache_write page bit is set immediately and will then
- * be cleared at the completion of the write to indicate the success or failure
- * of the operation. Note that the completion may happen before the return.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_write_page(struct fscache_cookie *cookie,
- struct page *page,
- loff_t object_size,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_write_page(cookie, page, object_size, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_uncache_page - Indicate that caching is no longer required on a page
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that was being cached.
- *
- * Tell the cache that we no longer want a page to be cached and that it should
- * remove any knowledge of the netfs page it may have.
- *
- * Note that this cannot cancel any outstanding I/O operations between this
- * page and the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_uncache_page(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_uncache_page(cookie, page);
-}
-
-/**
- * fscache_check_page_write - Ask if a page is being writing to the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- *
- * Ask the cache if a page is being written to the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-bool fscache_check_page_write(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- return __fscache_check_page_write(cookie, page);
- return false;
-}
-
-/**
- * fscache_wait_on_page_write - Wait for a page to complete writing to the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- *
- * Ask the cache to wake us up when a page is no longer being written to the
- * cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_wait_on_page_write(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_wait_on_page_write(cookie, page);
-}
-
-/**
- * fscache_maybe_release_page - Consider releasing a page, cancelling a store
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- * @gfp: The gfp flags passed to releasepage()
- *
- * Consider releasing a page for the vmscan algorithm, on behalf of the netfs's
- * releasepage() call. A storage request on the page may cancelled if it is
- * not currently being processed.
- *
- * The function returns true if the page no longer has a storage request on it,
- * and false if a storage request is left in place. If true is returned, the
- * page will have been passed to fscache_uncache_page(). If false is returned
- * the page cannot be freed yet.
- */
-static inline
-bool fscache_maybe_release_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && PageFsCache(page))
- return __fscache_maybe_release_page(cookie, page, gfp);
- return true;
-}
-
-/**
- * fscache_uncache_all_inode_pages - Uncache all an inode's pages
- * @cookie: The cookie representing the inode's cache object.
- * @inode: The inode to uncache pages from.
- *
- * Uncache all the pages in an inode that are marked PG_fscache, assuming them
- * to be associated with the given cookie.
- *
- * This function may sleep. It will wait for pages that are being written out
- * and will wait whilst the PG_fscache mark is removed by the cache.
- */
-static inline
-void fscache_uncache_all_inode_pages(struct fscache_cookie *cookie,
- struct inode *inode)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_uncache_all_inode_pages(cookie, inode);
-}
-
/**
* fscache_disable_cookie - Disable a cookie
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 10/61] fscache: Remove the old I/O API
@ 2020-05-04 17:08 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:08 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove the old fscache I/O API. There's no point trying to transform it as
the new one will bear no similarities to the old one.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/Makefile | 1
fs/cachefiles/interface.c | 15 -
fs/cachefiles/internal.h | 38 -
fs/cachefiles/main.c | 1
fs/cachefiles/rdwr.c | 974 -----------------------------------
fs/fscache/cache.c | 6
fs/fscache/cookie.c | 10
fs/fscache/internal.h | 23 -
fs/fscache/object.c | 2
fs/fscache/page.c | 1068 --------------------------------------
fs/fscache/stats.c | 6
include/linux/fscache-cache.h | 132 -----
include/linux/fscache-obsolete.h | 13
include/linux/fscache.h | 349 ------------
14 files changed, 15 insertions(+), 2623 deletions(-)
delete mode 100644 fs/cachefiles/rdwr.c
create mode 100644 include/linux/fscache-obsolete.h
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index 891dedda5905..3455d3646547 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -10,7 +10,6 @@ cachefiles-y := \
key.o \
main.o \
namei.o \
- rdwr.o \
security.o \
xattr.o
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 4cea5fbf695e..04d92ad402a4 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -540,14 +540,6 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
_leave("");
}
-/*
- * dissociate a cache from all the pages it was backing
- */
-static void cachefiles_dissociate_pages(struct fscache_cache *cache)
-{
- _enter("");
-}
-
const struct fscache_cache_ops cachefiles_cache_ops = {
.name = "cachefiles",
.alloc_object = cachefiles_alloc_object,
@@ -560,12 +552,5 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
.attr_changed = cachefiles_attr_changed,
- .read_or_alloc_page = cachefiles_read_or_alloc_page,
- .read_or_alloc_pages = cachefiles_read_or_alloc_pages,
- .allocate_page = cachefiles_allocate_page,
- .allocate_pages = cachefiles_allocate_pages,
- .write_page = cachefiles_write_page,
- .uncache_page = cachefiles_uncache_page,
- .dissociate_pages = cachefiles_dissociate_pages,
.check_consistency = cachefiles_check_consistency,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index cf9bd6401c2d..aca73c8403ab 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -43,7 +43,6 @@ struct cachefiles_object {
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
uint8_t new; /* T if object new */
- spinlock_t work_lock;
struct rb_node active_node; /* link in active tree (dentry is key) */
};
@@ -89,28 +88,6 @@ struct cachefiles_cache {
char *tag; /* cache binding tag */
};
-/*
- * backing file read tracking
- */
-struct cachefiles_one_read {
- wait_queue_entry_t monitor; /* link into monitored waitqueue */
- struct page *back_page; /* backing file page we're waiting for */
- struct page *netfs_page; /* netfs page we're going to fill */
- struct fscache_retrieval *op; /* retrieval op covering this */
- struct list_head op_link; /* link in op's todo list */
-};
-
-/*
- * backing file write tracking
- */
-struct cachefiles_one_write {
- struct page *netfs_page; /* netfs page to copy */
- struct cachefiles_object *object;
- struct list_head obj_link; /* link in object's lists */
- fscache_rw_complete_t end_io_func;
- void *context;
-};
-
/*
* auxiliary data xattr buffer
*/
@@ -202,21 +179,6 @@ void cachefiles_hist(atomic_t histogram[], unsigned long start_jif)
#define cachefiles_hist(hist, start_jif) do {} while (0)
#endif
-/*
- * rdwr.c
- */
-extern int cachefiles_read_or_alloc_page(struct fscache_retrieval *,
- struct page *, gfp_t);
-extern int cachefiles_read_or_alloc_pages(struct fscache_retrieval *,
- struct list_head *, unsigned *,
- gfp_t);
-extern int cachefiles_allocate_page(struct fscache_retrieval *, struct page *,
- gfp_t);
-extern int cachefiles_allocate_pages(struct fscache_retrieval *,
- struct list_head *, unsigned *, gfp_t);
-extern int cachefiles_write_page(struct fscache_storage *, struct page *);
-extern void cachefiles_uncache_page(struct fscache_object *, struct page *);
-
/*
* security.c
*/
diff --git a/fs/cachefiles/main.c b/fs/cachefiles/main.c
index ddf0cd58d60c..3f0101a74809 100644
--- a/fs/cachefiles/main.c
+++ b/fs/cachefiles/main.c
@@ -42,7 +42,6 @@ static void cachefiles_object_init_once(void *_object)
struct cachefiles_object *object = _object;
memset(object, 0, sizeof(*object));
- spin_lock_init(&object->work_lock);
}
/*
diff --git a/fs/cachefiles/rdwr.c b/fs/cachefiles/rdwr.c
deleted file mode 100644
index 1dc97f2d6201..000000000000
--- a/fs/cachefiles/rdwr.c
+++ /dev/null
@@ -1,974 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Storage object read/write
- *
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
- */
-
-#include <linux/mount.h>
-#include <linux/slab.h>
-#include <linux/file.h>
-#include <linux/swap.h>
-#include "internal.h"
-
-/*
- * detect wake up events generated by the unlocking of pages in which we're
- * interested
- * - we use this to detect read completion of backing pages
- * - the caller holds the waitqueue lock
- */
-static int cachefiles_read_waiter(wait_queue_entry_t *wait, unsigned mode,
- int sync, void *_key)
-{
- struct cachefiles_one_read *monitor =
- container_of(wait, struct cachefiles_one_read, monitor);
- struct cachefiles_object *object;
- struct fscache_retrieval *op = monitor->op;
- struct wait_bit_key *key = _key;
- struct page *page = wait->private;
-
- ASSERT(key);
-
- _enter("{%lu},%u,%d,{%p,%u}",
- monitor->netfs_page->index, mode, sync,
- key->flags, key->bit_nr);
-
- if (key->flags != &page->flags ||
- key->bit_nr != PG_locked)
- return 0;
-
- _debug("--- monitor %p %lx ---", page, page->flags);
-
- if (!PageUptodate(page) && !PageError(page)) {
- /* unlocked, not uptodate and not erronous? */
- _debug("page probably truncated");
- }
-
- /* remove from the waitqueue */
- list_del(&wait->entry);
-
- /* move onto the action list and queue for FS-Cache thread pool */
- ASSERT(op);
-
- /* We need to temporarily bump the usage count as we don't own a ref
- * here otherwise cachefiles_read_copier() may free the op between the
- * monitor being enqueued on the op->to_do list and the op getting
- * enqueued on the work queue.
- */
- fscache_get_retrieval(op);
-
- object = container_of(op->op.object, struct cachefiles_object, fscache);
- spin_lock(&object->work_lock);
- list_add_tail(&monitor->op_link, &op->to_do);
- spin_unlock(&object->work_lock);
-
- fscache_enqueue_retrieval(op);
- fscache_put_retrieval(op);
- return 0;
-}
-
-/*
- * handle a probably truncated page
- * - check to see if the page is still relevant and reissue the read if
- * possible
- * - return -EIO on error, -ENODATA if the page is gone, -EINPROGRESS if we
- * must wait again and 0 if successful
- */
-static int cachefiles_read_reissue(struct cachefiles_object *object,
- struct cachefiles_one_read *monitor)
-{
- struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
- struct page *backpage = monitor->back_page, *backpage2;
- int ret;
-
- _enter("{ino=%lx},{%lx,%lx}",
- d_backing_inode(object->backer)->i_ino,
- backpage->index, backpage->flags);
-
- /* skip if the page was truncated away completely */
- if (backpage->mapping != bmapping) {
- _leave(" = -ENODATA [mapping]");
- return -ENODATA;
- }
-
- backpage2 = find_get_page(bmapping, backpage->index);
- if (!backpage2) {
- _leave(" = -ENODATA [gone]");
- return -ENODATA;
- }
-
- if (backpage != backpage2) {
- put_page(backpage2);
- _leave(" = -ENODATA [different]");
- return -ENODATA;
- }
-
- /* the page is still there and we already have a ref on it, so we don't
- * need a second */
- put_page(backpage2);
-
- INIT_LIST_HEAD(&monitor->op_link);
- add_page_wait_queue(backpage, &monitor->monitor);
-
- if (trylock_page(backpage)) {
- ret = -EIO;
- if (PageError(backpage))
- goto unlock_discard;
- ret = 0;
- if (PageUptodate(backpage))
- goto unlock_discard;
-
- _debug("reissue read");
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto unlock_discard;
- }
-
- /* but the page may have been read before the monitor was installed, so
- * the monitor may miss the event - so we have to ensure that we do get
- * one in such a case */
- if (trylock_page(backpage)) {
- _debug("jumpstart %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
-
- /* it'll reappear on the todo list */
- _leave(" = -EINPROGRESS");
- return -EINPROGRESS;
-
-unlock_discard:
- unlock_page(backpage);
- spin_lock_irq(&object->work_lock);
- list_del(&monitor->op_link);
- spin_unlock_irq(&object->work_lock);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * copy data from backing pages to netfs pages to complete a read operation
- * - driven by FS-Cache's thread pool
- */
-static void cachefiles_read_copier(struct fscache_operation *_op)
-{
- struct cachefiles_one_read *monitor;
- struct cachefiles_object *object;
- struct fscache_retrieval *op;
- int error, max;
-
- op = container_of(_op, struct fscache_retrieval, op);
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
-
- _enter("{ino=%lu}", d_backing_inode(object->backer)->i_ino);
-
- max = 8;
- spin_lock_irq(&object->work_lock);
-
- while (!list_empty(&op->to_do)) {
- monitor = list_entry(op->to_do.next,
- struct cachefiles_one_read, op_link);
- list_del(&monitor->op_link);
-
- spin_unlock_irq(&object->work_lock);
-
- _debug("- copy {%lu}", monitor->back_page->index);
-
- recheck:
- if (test_bit(FSCACHE_COOKIE_INVALIDATING,
- &object->fscache.cookie->flags)) {
- error = -ESTALE;
- } else if (PageUptodate(monitor->back_page)) {
- copy_highpage(monitor->netfs_page, monitor->back_page);
- fscache_mark_page_cached(monitor->op,
- monitor->netfs_page);
- error = 0;
- } else if (!PageError(monitor->back_page)) {
- /* the page has probably been truncated */
- error = cachefiles_read_reissue(object, monitor);
- if (error == -EINPROGRESS)
- goto next;
- goto recheck;
- } else {
- cachefiles_io_error_obj(
- object,
- "Readpage failed on backing file %lx",
- (unsigned long) monitor->back_page->flags);
- error = -EIO;
- }
-
- put_page(monitor->back_page);
-
- fscache_end_io(op, monitor->netfs_page, error);
- put_page(monitor->netfs_page);
- fscache_retrieval_complete(op, 1);
- fscache_put_retrieval(op);
- kfree(monitor);
-
- next:
- /* let the thread pool have some air occasionally */
- max--;
- if (max < 0 || need_resched()) {
- if (!list_empty(&op->to_do))
- fscache_enqueue_retrieval(op);
- _leave(" [maxed out]");
- return;
- }
-
- spin_lock_irq(&object->work_lock);
- }
-
- spin_unlock_irq(&object->work_lock);
- _leave("");
-}
-
-/*
- * read the corresponding page to the given set from the backing file
- * - an uncertain page is simply discarded, to be tried again another time
- */
-static int cachefiles_read_backing_file_one(struct cachefiles_object *object,
- struct fscache_retrieval *op,
- struct page *netpage)
-{
- struct cachefiles_one_read *monitor;
- struct address_space *bmapping;
- struct page *newpage, *backpage;
- int ret;
-
- _enter("");
-
- _debug("read back %p{%lu,%d}",
- netpage, netpage->index, page_count(netpage));
-
- monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
- if (!monitor)
- goto nomem;
-
- monitor->netfs_page = netpage;
- monitor->op = fscache_get_retrieval(op);
-
- init_waitqueue_func_entry(&monitor->monitor, cachefiles_read_waiter);
-
- /* attempt to get hold of the backing page */
- bmapping = d_backing_inode(object->backer)->i_mapping;
- newpage = NULL;
-
- for (;;) {
- backpage = find_get_page(bmapping, netpage->index);
- if (backpage)
- goto backing_page_already_present;
-
- if (!newpage) {
- newpage = __page_cache_alloc(cachefiles_gfp);
- if (!newpage)
- goto nomem_monitor;
- }
-
- ret = add_to_page_cache_lru(newpage, bmapping,
- netpage->index, cachefiles_gfp);
- if (ret == 0)
- goto installed_new_backing_page;
- if (ret != -EEXIST)
- goto nomem_page;
- }
-
- /* we've installed a new backing page, so now we need to start
- * it reading */
-installed_new_backing_page:
- _debug("- new %p", newpage);
-
- backpage = newpage;
- newpage = NULL;
-
-read_backing_page:
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto read_error;
-
- /* set the monitor to transfer the data across */
-monitor_backing_page:
- _debug("- monitor add");
-
- /* install the monitor */
- get_page(monitor->netfs_page);
- get_page(backpage);
- monitor->back_page = backpage;
- monitor->monitor.private = backpage;
- add_page_wait_queue(backpage, &monitor->monitor);
- monitor = NULL;
-
- /* but the page may have been read before the monitor was installed, so
- * the monitor may miss the event - so we have to ensure that we do get
- * one in such a case */
- if (trylock_page(backpage)) {
- _debug("jumpstart %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
- goto success;
-
- /* if the backing page is already present, it can be in one of
- * three states: read in progress, read failed or read okay */
-backing_page_already_present:
- _debug("- present");
-
- if (newpage) {
- put_page(newpage);
- newpage = NULL;
- }
-
- if (PageError(backpage))
- goto io_error;
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate;
-
- if (!trylock_page(backpage))
- goto monitor_backing_page;
- _debug("read %p {%lx}", backpage, backpage->flags);
- goto read_backing_page;
-
- /* the backing page is already up to date, attach the netfs
- * page to the pagecache and LRU and copy the data across */
-backing_page_already_uptodate:
- _debug("- uptodate");
-
- fscache_mark_page_cached(op, netpage);
-
- copy_highpage(netpage, backpage);
- fscache_end_io(op, netpage, 0);
- fscache_retrieval_complete(op, 1);
-
-success:
- _debug("success");
- ret = 0;
-
-out:
- if (backpage)
- put_page(backpage);
- if (monitor) {
- fscache_put_retrieval(monitor->op);
- kfree(monitor);
- }
- _leave(" = %d", ret);
- return ret;
-
-read_error:
- _debug("read error %d", ret);
- if (ret == -ENOMEM) {
- fscache_retrieval_complete(op, 1);
- goto out;
- }
-io_error:
- cachefiles_io_error_obj(object, "Page read error on backing file");
- fscache_retrieval_complete(op, 1);
- ret = -ENOBUFS;
- goto out;
-
-nomem_page:
- put_page(newpage);
-nomem_monitor:
- fscache_put_retrieval(monitor->op);
- kfree(monitor);
-nomem:
- fscache_retrieval_complete(op, 1);
- _leave(" = -ENOMEM");
- return -ENOMEM;
-}
-
-/*
- * read a page from the cache or allocate a block in which to store it
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if no buffers can be made available
- * - returns -ENOBUFS if page is beyond EOF
- * - if the page is backed by a block in the cache:
- * - a read will be started which will call the callback on completion
- * - 0 will be returned
- * - else if the page is unbacked:
- * - the metadata will be retained
- * - -ENODATA will be returned
- */
-int cachefiles_read_or_alloc_page(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct inode *inode;
- sector_t block;
- unsigned shift;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("{%p},{%lx},,,", object, page->index);
-
- if (!object->backer)
- goto enobufs;
-
- inode = d_backing_inode(object->backer);
- ASSERT(S_ISREG(inode->i_mode));
- ASSERT(inode->i_mapping->a_ops->readpages);
-
- /* calculate the shift required to use bmap */
- shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
-
- op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
- op->op.flags |= FSCACHE_OP_ASYNC;
- op->op.processor = cachefiles_read_copier;
-
- /* we assume the absence or presence of the first block is a good
- * enough indication for the page as a whole
- * - TODO: don't use bmap() for this as it is _not_ actually good
- * enough for this as it doesn't indicate errors, but it's all we've
- * got for the moment
- */
- block = page->index;
- block <<= shift;
-
- ret = bmap(inode, &block);
- ASSERT(ret < 0);
-
- _debug("%llx -> %llx",
- (unsigned long long) (page->index << shift),
- (unsigned long long) block);
-
- if (block) {
- /* submit the apparently valid page to the backing fs to be
- * read from disk */
- ret = cachefiles_read_backing_file_one(object, op, page);
- } else if (cachefiles_has_space(cache, 0, 1) == 0) {
- /* there's space in the cache we can use */
- fscache_mark_page_cached(op, page);
- fscache_retrieval_complete(op, 1);
- ret = -ENODATA;
- } else {
- goto enobufs;
- }
-
- _leave(" = %d", ret);
- return ret;
-
-enobufs:
- fscache_retrieval_complete(op, 1);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-
-/*
- * read the corresponding pages to the given set from the backing file
- * - any uncertain pages are simply discarded, to be tried again another time
- */
-static int cachefiles_read_backing_file(struct cachefiles_object *object,
- struct fscache_retrieval *op,
- struct list_head *list)
-{
- struct cachefiles_one_read *monitor = NULL;
- struct address_space *bmapping = d_backing_inode(object->backer)->i_mapping;
- struct page *newpage = NULL, *netpage, *_n, *backpage = NULL;
- int ret = 0;
-
- _enter("");
-
- list_for_each_entry_safe(netpage, _n, list, lru) {
- list_del(&netpage->lru);
-
- _debug("read back %p{%lu,%d}",
- netpage, netpage->index, page_count(netpage));
-
- if (!monitor) {
- monitor = kzalloc(sizeof(*monitor), cachefiles_gfp);
- if (!monitor)
- goto nomem;
-
- monitor->op = fscache_get_retrieval(op);
- init_waitqueue_func_entry(&monitor->monitor,
- cachefiles_read_waiter);
- }
-
- for (;;) {
- backpage = find_get_page(bmapping, netpage->index);
- if (backpage)
- goto backing_page_already_present;
-
- if (!newpage) {
- newpage = __page_cache_alloc(cachefiles_gfp);
- if (!newpage)
- goto nomem;
- }
-
- ret = add_to_page_cache_lru(newpage, bmapping,
- netpage->index,
- cachefiles_gfp);
- if (ret == 0)
- goto installed_new_backing_page;
- if (ret != -EEXIST)
- goto nomem;
- }
-
- /* we've installed a new backing page, so now we need
- * to start it reading */
- installed_new_backing_page:
- _debug("- new %p", newpage);
-
- backpage = newpage;
- newpage = NULL;
-
- reread_backing_page:
- ret = bmapping->a_ops->readpage(NULL, backpage);
- if (ret < 0)
- goto read_error;
-
- /* add the netfs page to the pagecache and LRU, and set the
- * monitor to transfer the data across */
- monitor_backing_page:
- _debug("- monitor add");
-
- ret = add_to_page_cache_lru(netpage, op->mapping,
- netpage->index, cachefiles_gfp);
- if (ret < 0) {
- if (ret == -EEXIST) {
- put_page(backpage);
- backpage = NULL;
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
- goto nomem;
- }
-
- /* install a monitor */
- get_page(netpage);
- monitor->netfs_page = netpage;
-
- get_page(backpage);
- monitor->back_page = backpage;
- monitor->monitor.private = backpage;
- add_page_wait_queue(backpage, &monitor->monitor);
- monitor = NULL;
-
- /* but the page may have been read before the monitor was
- * installed, so the monitor may miss the event - so we have to
- * ensure that we do get one in such a case */
- if (trylock_page(backpage)) {
- _debug("2unlock %p {%lx}", backpage, backpage->flags);
- unlock_page(backpage);
- }
-
- put_page(backpage);
- backpage = NULL;
-
- put_page(netpage);
- netpage = NULL;
- continue;
-
- /* if the backing page is already present, it can be in one of
- * three states: read in progress, read failed or read okay */
- backing_page_already_present:
- _debug("- present %p", backpage);
-
- if (PageError(backpage))
- goto io_error;
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate;
-
- _debug("- not ready %p{%lx}", backpage, backpage->flags);
-
- if (!trylock_page(backpage))
- goto monitor_backing_page;
-
- if (PageError(backpage)) {
- _debug("error %lx", backpage->flags);
- unlock_page(backpage);
- goto io_error;
- }
-
- if (PageUptodate(backpage))
- goto backing_page_already_uptodate_unlock;
-
- /* we've locked a page that's neither up to date nor erroneous,
- * so we need to attempt to read it again */
- goto reread_backing_page;
-
- /* the backing page is already up to date, attach the netfs
- * page to the pagecache and LRU and copy the data across */
- backing_page_already_uptodate_unlock:
- _debug("uptodate %lx", backpage->flags);
- unlock_page(backpage);
- backing_page_already_uptodate:
- _debug("- uptodate");
-
- ret = add_to_page_cache_lru(netpage, op->mapping,
- netpage->index, cachefiles_gfp);
- if (ret < 0) {
- if (ret == -EEXIST) {
- put_page(backpage);
- backpage = NULL;
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
- goto nomem;
- }
-
- copy_highpage(netpage, backpage);
-
- put_page(backpage);
- backpage = NULL;
-
- fscache_mark_page_cached(op, netpage);
-
- /* the netpage is unlocked and marked up to date here */
- fscache_end_io(op, netpage, 0);
- put_page(netpage);
- netpage = NULL;
- fscache_retrieval_complete(op, 1);
- continue;
- }
-
- netpage = NULL;
-
- _debug("out");
-
-out:
- /* tidy up */
- if (newpage)
- put_page(newpage);
- if (netpage)
- put_page(netpage);
- if (backpage)
- put_page(backpage);
- if (monitor) {
- fscache_put_retrieval(op);
- kfree(monitor);
- }
-
- list_for_each_entry_safe(netpage, _n, list, lru) {
- list_del(&netpage->lru);
- put_page(netpage);
- fscache_retrieval_complete(op, 1);
- }
-
- _leave(" = %d", ret);
- return ret;
-
-nomem:
- _debug("nomem");
- ret = -ENOMEM;
- goto record_page_complete;
-
-read_error:
- _debug("read error %d", ret);
- if (ret == -ENOMEM)
- goto record_page_complete;
-io_error:
- cachefiles_io_error_obj(object, "Page read error on backing file");
- ret = -ENOBUFS;
-record_page_complete:
- fscache_retrieval_complete(op, 1);
- goto out;
-}
-
-/*
- * read a list of pages from the cache or allocate blocks in which to store
- * them
- */
-int cachefiles_read_or_alloc_pages(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct list_head backpages;
- struct pagevec pagevec;
- struct inode *inode;
- struct page *page, *_n;
- unsigned shift, nrbackpages;
- int ret, ret2, space;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("{OBJ%x,%d},,%d,,",
- object->fscache.debug_id, atomic_read(&op->op.usage),
- *nr_pages);
-
- if (!object->backer)
- goto all_enobufs;
-
- space = 1;
- if (cachefiles_has_space(cache, 0, *nr_pages) < 0)
- space = 0;
-
- inode = d_backing_inode(object->backer);
- ASSERT(S_ISREG(inode->i_mode));
- ASSERT(inode->i_mapping->a_ops->readpages);
-
- /* calculate the shift required to use bmap */
- shift = PAGE_SHIFT - inode->i_sb->s_blocksize_bits;
-
- pagevec_init(&pagevec);
-
- op->op.flags &= FSCACHE_OP_KEEP_FLAGS;
- op->op.flags |= FSCACHE_OP_ASYNC;
- op->op.processor = cachefiles_read_copier;
-
- INIT_LIST_HEAD(&backpages);
- nrbackpages = 0;
-
- ret = space ? -ENODATA : -ENOBUFS;
- list_for_each_entry_safe(page, _n, pages, lru) {
- sector_t block;
-
- /* we assume the absence or presence of the first block is a
- * good enough indication for the page as a whole
- * - TODO: don't use bmap() for this as it is _not_ actually
- * good enough for this as it doesn't indicate errors, but
- * it's all we've got for the moment
- */
- block = page->index;
- block <<= shift;
-
- ret = bmap(inode, &block);
- ASSERT(!ret);
-
- _debug("%llx -> %llx",
- (unsigned long long) (page->index << shift),
- (unsigned long long) block);
-
- if (block) {
- /* we have data - add it to the list to give to the
- * backing fs */
- list_move(&page->lru, &backpages);
- (*nr_pages)--;
- nrbackpages++;
- } else if (space && pagevec_add(&pagevec, page) == 0) {
- fscache_mark_pages_cached(op, &pagevec);
- fscache_retrieval_complete(op, 1);
- ret = -ENODATA;
- } else {
- fscache_retrieval_complete(op, 1);
- }
- }
-
- if (pagevec_count(&pagevec) > 0)
- fscache_mark_pages_cached(op, &pagevec);
-
- if (list_empty(pages))
- ret = 0;
-
- /* submit the apparently valid pages to the backing fs to be read from
- * disk */
- if (nrbackpages > 0) {
- ret2 = cachefiles_read_backing_file(object, op, &backpages);
- if (ret2 == -ENOMEM || ret2 == -EINTR)
- ret = ret2;
- }
-
- _leave(" = %d [nr=%u%s]",
- ret, *nr_pages, list_empty(pages) ? " empty" : "");
- return ret;
-
-all_enobufs:
- fscache_retrieval_complete(op, *nr_pages);
- return -ENOBUFS;
-}
-
-/*
- * allocate a block in the cache in which to store a page
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if no buffers can be made available
- * - returns -ENOBUFS if page is beyond EOF
- * - otherwise:
- * - the metadata will be retained
- * - 0 will be returned
- */
-int cachefiles_allocate_page(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("%p,{%lx},", object, page->index);
-
- ret = cachefiles_has_space(cache, 0, 1);
- if (ret == 0)
- fscache_mark_page_cached(op, page);
- else
- ret = -ENOBUFS;
-
- fscache_retrieval_complete(op, 1);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * allocate blocks in the cache in which to store a set of pages
- * - cache withdrawal is prevented by the caller
- * - returns -EINTR if interrupted
- * - returns -ENOMEM if ran out of memory
- * - returns -ENOBUFS if some buffers couldn't be made available
- * - returns -ENOBUFS if some pages are beyond EOF
- * - otherwise:
- * - -ENODATA will be returned
- * - metadata will be retained for any page marked
- */
-int cachefiles_allocate_pages(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct pagevec pagevec;
- struct page *page;
- int ret;
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- _enter("%p,,,%d,", object, *nr_pages);
-
- ret = cachefiles_has_space(cache, 0, *nr_pages);
- if (ret == 0) {
- pagevec_init(&pagevec);
-
- list_for_each_entry(page, pages, lru) {
- if (pagevec_add(&pagevec, page) == 0)
- fscache_mark_pages_cached(op, &pagevec);
- }
-
- if (pagevec_count(&pagevec) > 0)
- fscache_mark_pages_cached(op, &pagevec);
- ret = -ENODATA;
- } else {
- ret = -ENOBUFS;
- }
-
- fscache_retrieval_complete(op, *nr_pages);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * request a page be stored in the cache
- * - cache withdrawal is prevented by the caller
- * - this request may be ignored if there's no cache block available, in which
- * case -ENOBUFS will be returned
- * - if the op is in progress, 0 will be returned
- */
-int cachefiles_write_page(struct fscache_storage *op, struct page *page)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- struct file *file;
- struct path path;
- loff_t pos, eof;
- size_t len;
- void *data;
- int ret = -ENOBUFS;
-
- ASSERT(op != NULL);
- ASSERT(page != NULL);
-
- object = container_of(op->op.object,
- struct cachefiles_object, fscache);
-
- _enter("%p,%p{%lx},,,", object, page, page->index);
-
- if (!object->backer) {
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
- }
-
- ASSERT(d_is_reg(object->backer));
-
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- pos = (loff_t)page->index << PAGE_SHIFT;
-
- /* We mustn't write more data than we have, so we have to beware of a
- * partial page at EOF.
- */
- eof = object->fscache.store_limit_l;
- if (pos >= eof)
- goto error;
-
- /* write the page to the backing filesystem and let it store it in its
- * own time */
- path.mnt = cache->mnt;
- path.dentry = object->backer;
- file = dentry_open(&path, O_RDWR | O_LARGEFILE, cache->cache_cred);
- if (IS_ERR(file)) {
- ret = PTR_ERR(file);
- goto error_2;
- }
-
- len = PAGE_SIZE;
- if (eof & ~PAGE_MASK) {
- if (eof - pos < PAGE_SIZE) {
- _debug("cut short %llx to %llx",
- pos, eof);
- len = eof - pos;
- ASSERTCMP(pos + len, ==, eof);
- }
- }
-
- data = kmap(page);
- ret = __kernel_write(file, data, len, &pos);
- kunmap(page);
- fput(file);
- if (ret != len)
- goto error_eio;
-
- _leave(" = 0");
- return 0;
-
-error_eio:
- ret = -EIO;
-error_2:
- if (ret == -EIO)
- cachefiles_io_error_obj(object,
- "Write page to backing file failed");
-error:
- _leave(" = -ENOBUFS [%d]", ret);
- return -ENOBUFS;
-}
-
-/*
- * detach a backing block from a page
- * - cache withdrawal is prevented by the caller
- */
-void cachefiles_uncache_page(struct fscache_object *_object, struct page *page)
- __releases(&object->fscache.cookie->lock)
-{
- struct cachefiles_object *object;
-
- object = container_of(_object, struct cachefiles_object, fscache);
-
- _enter("%p,{%lu}", object, page->index);
-
- spin_unlock(&object->fscache.cookie->lock);
-}
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index f78793f3d21e..7ec80456a700 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -382,12 +382,6 @@ void fscache_withdraw_cache(struct fscache_cache *cache)
cache->ops->sync_cache(cache);
fscache_stat_d(&fscache_n_cop_sync_cache);
- /* dissociate all the netfs pages backed by this cache from the block
- * mappings in the cache */
- fscache_stat(&fscache_n_cop_dissociate_pages);
- cache->ops->dissociate_pages(cache);
- fscache_stat_d(&fscache_n_cop_dissociate_pages);
-
/* we now have to destroy all the active objects pertaining to this
* cache - which we do by passing them off to thread pool to be
* disposed of */
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 477c5fb349ee..fa002ef7766d 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -191,13 +191,8 @@ struct fscache_cookie *fscache_alloc_cookie(
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
cookie->type = def->type;
spin_lock_init(&cookie->lock);
- spin_lock_init(&cookie->stores_lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
- /* radix tree insertion won't use the preallocation pool unless it's
- * told it may not wait */
- INIT_RADIX_TREE(&cookie->stores, GFP_NOFS & ~__GFP_DIRECT_RECLAIM);
-
write_lock(&fscache_cookies_lock);
list_add_tail(&cookie->proc_link, &fscache_cookies);
write_unlock(&fscache_cookies_lock);
@@ -786,10 +781,6 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
!atomic_read(&cookie->n_active));
}
- /* Make sure any pending writes are cancelled. */
- if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
- fscache_invalidate_writes(cookie);
-
/* Reset the cookie state if it wasn't relinquished */
if (!test_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags)) {
atomic_inc(&cookie->n_active);
@@ -838,7 +829,6 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
/* Clear pointers back to the netfs */
cookie->netfs_data = NULL;
cookie->def = NULL;
- BUG_ON(!radix_tree_empty(&cookie->stores));
if (cookie->parent) {
ASSERTCMP(atomic_read(&cookie->parent->usage), >, 0);
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 4b535c2dae4a..83dfbe0e3964 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -142,7 +142,6 @@ extern int fscache_wait_for_operation_activation(struct fscache_object *,
struct fscache_operation *,
atomic_t *,
atomic_t *);
-extern void fscache_invalidate_writes(struct fscache_cookie *);
/*
* proc.c
@@ -272,7 +271,6 @@ extern atomic_t fscache_n_cop_allocate_page;
extern atomic_t fscache_n_cop_allocate_pages;
extern atomic_t fscache_n_cop_write_page;
extern atomic_t fscache_n_cop_uncache_page;
-extern atomic_t fscache_n_cop_dissociate_pages;
extern atomic_t fscache_n_cache_no_space_reject;
extern atomic_t fscache_n_cache_stale_objects;
@@ -325,27 +323,6 @@ static inline void fscache_cookie_get(struct fscache_cookie *cookie,
trace_fscache_cookie(cookie, where, usage);
}
-/*
- * get an extra reference to a netfs retrieval context
- */
-static inline
-void *fscache_get_context(struct fscache_cookie *cookie, void *context)
-{
- if (cookie->def->get_context)
- cookie->def->get_context(cookie->netfs_data, context);
- return context;
-}
-
-/*
- * release a reference to a netfs retrieval context
- */
-static inline
-void fscache_put_context(struct fscache_cookie *cookie, void *context)
-{
- if (cookie->def->put_context)
- cookie->def->put_context(cookie->netfs_data, context);
-}
-
/*
* Update the auxiliary data on a cookie.
*/
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index cfeba839a0f2..6e4ade2f1e6f 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -973,14 +973,12 @@ static const struct fscache_state *_fscache_invalidate_object(struct fscache_obj
* retire the object instead.
*/
if (!fscache_use_cookie(object)) {
- ASSERT(radix_tree_empty(&object->cookie->stores));
set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
_leave(" [no cookie]");
return transit_to(KILL_OBJECT);
}
/* Reject any new read/write ops and abort any that are pending. */
- fscache_invalidate_writes(cookie);
clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
fscache_cancel_all_ops(object);
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index 26af6fdf1538..1beffb071205 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -13,174 +13,6 @@
#include <linux/slab.h>
#include "internal.h"
-/*
- * check to see if a page is being written to the cache
- */
-bool __fscache_check_page_write(struct fscache_cookie *cookie, struct page *page)
-{
- void *val;
-
- rcu_read_lock();
- val = radix_tree_lookup(&cookie->stores, page->index);
- rcu_read_unlock();
- trace_fscache_check_page(cookie, page, val, 0);
-
- return val != NULL;
-}
-EXPORT_SYMBOL(__fscache_check_page_write);
-
-/*
- * wait for a page to finish being written to the cache
- */
-void __fscache_wait_on_page_write(struct fscache_cookie *cookie, struct page *page)
-{
- wait_queue_head_t *wq = bit_waitqueue(&cookie->flags, 0);
-
- trace_fscache_page(cookie, page, fscache_page_write_wait);
-
- wait_event(*wq, !__fscache_check_page_write(cookie, page));
-}
-EXPORT_SYMBOL(__fscache_wait_on_page_write);
-
-/*
- * wait for a page to finish being written to the cache. Put a timeout here
- * since we might be called recursively via parent fs.
- */
-static
-bool release_page_wait_timeout(struct fscache_cookie *cookie, struct page *page)
-{
- wait_queue_head_t *wq = bit_waitqueue(&cookie->flags, 0);
-
- return wait_event_timeout(*wq, !__fscache_check_page_write(cookie, page),
- HZ);
-}
-
-/*
- * decide whether a page can be released, possibly by cancelling a store to it
- * - we're allowed to sleep if __GFP_DIRECT_RECLAIM is flagged
- */
-bool __fscache_maybe_release_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- struct page *xpage;
- void *val;
-
- _enter("%p,%p,%x", cookie, page, gfp);
-
- trace_fscache_page(cookie, page, fscache_page_maybe_release);
-
-try_again:
- rcu_read_lock();
- val = radix_tree_lookup(&cookie->stores, page->index);
- if (!val) {
- rcu_read_unlock();
- fscache_stat(&fscache_n_store_vmscan_not_storing);
- __fscache_uncache_page(cookie, page);
- return true;
- }
-
- /* see if the page is actually undergoing storage - if so we can't get
- * rid of it till the cache has finished with it */
- if (radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG)) {
- rcu_read_unlock();
- goto page_busy;
- }
-
- /* the page is pending storage, so we attempt to cancel the store and
- * discard the store request so that the page can be reclaimed */
- spin_lock(&cookie->stores_lock);
- rcu_read_unlock();
-
- if (radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG)) {
- /* the page started to undergo storage whilst we were looking,
- * so now we can only wait or return */
- spin_unlock(&cookie->stores_lock);
- goto page_busy;
- }
-
- xpage = radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- spin_unlock(&cookie->stores_lock);
-
- if (xpage) {
- fscache_stat(&fscache_n_store_vmscan_cancelled);
- fscache_stat(&fscache_n_store_radix_deletes);
- ASSERTCMP(xpage, ==, page);
- } else {
- fscache_stat(&fscache_n_store_vmscan_gone);
- }
-
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
- if (xpage)
- put_page(xpage);
- __fscache_uncache_page(cookie, page);
- return true;
-
-page_busy:
- /* We will wait here if we're allowed to, but that could deadlock the
- * allocator as the work threads writing to the cache may all end up
- * sleeping on memory allocation, so we may need to impose a timeout
- * too. */
- if (!(gfp & __GFP_DIRECT_RECLAIM) || !(gfp & __GFP_FS)) {
- fscache_stat(&fscache_n_store_vmscan_busy);
- return false;
- }
-
- fscache_stat(&fscache_n_store_vmscan_wait);
- if (!release_page_wait_timeout(cookie, page))
- _debug("fscache writeout timeout page: %p{%lx}",
- page, page->index);
-
- gfp &= ~__GFP_DIRECT_RECLAIM;
- goto try_again;
-}
-EXPORT_SYMBOL(__fscache_maybe_release_page);
-
-/*
- * note that a page has finished being written to the cache
- */
-static void fscache_end_page_write(struct fscache_object *object,
- struct page *page)
-{
- struct fscache_cookie *cookie;
- struct page *xpage = NULL, *val;
-
- spin_lock(&object->lock);
- cookie = object->cookie;
- if (cookie) {
- /* delete the page from the tree if it is now no longer
- * pending */
- spin_lock(&cookie->stores_lock);
- radix_tree_tag_clear(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_clear_store);
- if (!radix_tree_tag_get(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG)) {
- fscache_stat(&fscache_n_store_radix_deletes);
- xpage = radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- trace_fscache_page(cookie, page, fscache_page_write_end);
-
- val = radix_tree_lookup(&cookie->stores, page->index);
- trace_fscache_check_page(cookie, page, val, 1);
- } else {
- trace_fscache_page(cookie, page, fscache_page_write_end_pend);
- }
- spin_unlock(&cookie->stores_lock);
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
- } else {
- trace_fscache_page(cookie, page, fscache_page_write_end_noc);
- }
- spin_unlock(&object->lock);
- if (xpage)
- put_page(xpage);
-}
-
/*
* actually apply the changed attributes to a cache object
*/
@@ -265,76 +97,6 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
}
EXPORT_SYMBOL(__fscache_attr_changed);
-/*
- * Handle cancellation of a pending retrieval op
- */
-static void fscache_do_cancel_retrieval(struct fscache_operation *_op)
-{
- struct fscache_retrieval *op =
- container_of(_op, struct fscache_retrieval, op);
-
- atomic_set(&op->n_pages, 0);
-}
-
-/*
- * release a retrieval op reference
- */
-static void fscache_release_retrieval_op(struct fscache_operation *_op)
-{
- struct fscache_retrieval *op =
- container_of(_op, struct fscache_retrieval, op);
-
- _enter("{OP%x}", op->op.debug_id);
-
- ASSERTIFCMP(op->op.state != FSCACHE_OP_ST_INITIALISED,
- atomic_read(&op->n_pages), ==, 0);
-
- fscache_hist(fscache_retrieval_histogram, op->start_time);
- if (op->context)
- fscache_put_context(op->cookie, op->context);
-
- _leave("");
-}
-
-/*
- * allocate a retrieval op
- */
-static struct fscache_retrieval *fscache_alloc_retrieval(
- struct fscache_cookie *cookie,
- struct address_space *mapping,
- fscache_rw_complete_t end_io_func,
- void *context)
-{
- struct fscache_retrieval *op;
-
- /* allocate a retrieval operation and attempt to submit it */
- op = kzalloc(sizeof(*op), GFP_NOIO);
- if (!op) {
- fscache_stat(&fscache_n_retrievals_nomem);
- return NULL;
- }
-
- fscache_operation_init(cookie, &op->op, NULL,
- fscache_do_cancel_retrieval,
- fscache_release_retrieval_op);
- op->op.flags = FSCACHE_OP_MYTHREAD |
- (1UL << FSCACHE_OP_WAITING) |
- (1UL << FSCACHE_OP_UNUSE_COOKIE);
- op->cookie = cookie;
- op->mapping = mapping;
- op->end_io_func = end_io_func;
- op->context = context;
- op->start_time = jiffies;
- INIT_LIST_HEAD(&op->to_do);
-
- /* Pin the netfs read context in case we need to do the actual netfs
- * read because we've encountered a cache read failure.
- */
- if (context)
- fscache_get_context(op->cookie, context);
- return op;
-}
-
/*
* wait for a deferred lookup to complete
*/
@@ -416,833 +178,3 @@ int fscache_wait_for_operation_activation(struct fscache_object *object,
}
return 0;
}
-
-/*
- * read a page from the cache or allocate a block in which to store it
- * - we return:
- * -ENOMEM - out of memory, nothing done
- * -ERESTARTSYS - interrupted
- * -ENOBUFS - no backing object available in which to cache the block
- * -ENODATA - no data available in the backing object for this block
- * 0 - dispatched a read - it'll call end_io_func() when finished
- */
-int __fscache_read_or_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%p,,,", cookie, page);
-
- fscache_stat(&fscache_n_retrievals);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, page->mapping,
- end_io_func, context);
- if (!op) {
- _leave(" = -ENOMEM");
- return -ENOMEM;
- }
- atomic_set(&op->n_pages, 1);
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_retr_one);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- ASSERT(test_bit(FSCACHE_OBJECT_IS_LOOKED_UP, &object->flags));
-
- __fscache_use_cookie(cookie);
- atomic_inc(&object->n_reads);
- __set_bit(FSCACHE_OP_DEC_READ_CNT, &op->op.flags);
-
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_retrieval_ops);
-
- /* we wait for the operation to become active, and then process it
- * *here*, in this thread, and not in the thread pool */
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_retrieval_op_waits),
- __fscache_stat(&fscache_n_retrievals_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- if (test_bit(FSCACHE_COOKIE_NO_DATA_YET, &object->cookie->flags)) {
- fscache_stat(&fscache_n_cop_allocate_page);
- ret = object->cache->ops->allocate_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_page);
- if (ret == 0)
- ret = -ENODATA;
- } else {
- fscache_stat(&fscache_n_cop_read_or_alloc_page);
- ret = object->cache->ops->read_or_alloc_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_read_or_alloc_page);
- }
-
-error:
- if (ret == -ENOMEM)
- fscache_stat(&fscache_n_retrievals_nomem);
- else if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_retrievals_intr);
- else if (ret == -ENODATA)
- fscache_stat(&fscache_n_retrievals_nodata);
- else if (ret < 0)
- fscache_stat(&fscache_n_retrievals_nobufs);
- else
- fscache_stat(&fscache_n_retrievals_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- atomic_dec(&object->n_reads);
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- fscache_put_retrieval(op);
-nobufs:
- fscache_stat(&fscache_n_retrievals_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_read_or_alloc_page);
-
-/*
- * read a list of page from the cache or allocate a block in which to store
- * them
- * - we return:
- * -ENOMEM - out of memory, some pages may be being read
- * -ERESTARTSYS - interrupted, some pages may be being read
- * -ENOBUFS - no backing object or space available in which to cache any
- * pages not being read
- * -ENODATA - no data available in the backing object for some or all of
- * the pages
- * 0 - dispatched a read on all pages
- *
- * end_io_func() will be called for each page read from the cache as it is
- * finishes being read
- *
- * any pages for which a read is dispatched will be removed from pages and
- * nr_pages
- */
-int __fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
- struct address_space *mapping,
- struct list_head *pages,
- unsigned *nr_pages,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,,%d,,,", cookie, *nr_pages);
-
- fscache_stat(&fscache_n_retrievals);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(*nr_pages, >, 0);
- ASSERT(!list_empty(pages));
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, mapping, end_io_func, context);
- if (!op)
- return -ENOMEM;
- atomic_set(&op->n_pages, *nr_pages);
- trace_fscache_page_op(cookie, NULL, &op->op, fscache_page_op_retr_multi);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- __fscache_use_cookie(cookie);
- atomic_inc(&object->n_reads);
- __set_bit(FSCACHE_OP_DEC_READ_CNT, &op->op.flags);
-
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_retrieval_ops);
-
- /* we wait for the operation to become active, and then process it
- * *here*, in this thread, and not in the thread pool */
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_retrieval_op_waits),
- __fscache_stat(&fscache_n_retrievals_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- if (test_bit(FSCACHE_COOKIE_NO_DATA_YET, &object->cookie->flags)) {
- fscache_stat(&fscache_n_cop_allocate_pages);
- ret = object->cache->ops->allocate_pages(
- op, pages, nr_pages, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_pages);
- } else {
- fscache_stat(&fscache_n_cop_read_or_alloc_pages);
- ret = object->cache->ops->read_or_alloc_pages(
- op, pages, nr_pages, gfp);
- fscache_stat_d(&fscache_n_cop_read_or_alloc_pages);
- }
-
-error:
- if (ret == -ENOMEM)
- fscache_stat(&fscache_n_retrievals_nomem);
- else if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_retrievals_intr);
- else if (ret == -ENODATA)
- fscache_stat(&fscache_n_retrievals_nodata);
- else if (ret < 0)
- fscache_stat(&fscache_n_retrievals_nobufs);
- else
- fscache_stat(&fscache_n_retrievals_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- atomic_dec(&object->n_reads);
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- fscache_put_retrieval(op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
-nobufs:
- fscache_stat(&fscache_n_retrievals_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_read_or_alloc_pages);
-
-/*
- * allocate a block in the cache on which to store a page
- * - we return:
- * -ENOMEM - out of memory, nothing done
- * -ERESTARTSYS - interrupted
- * -ENOBUFS - no backing object available in which to cache the block
- * 0 - block allocated
- */
-int __fscache_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- struct fscache_retrieval *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%p,,,", cookie, page);
-
- fscache_stat(&fscache_n_allocs);
-
- if (hlist_empty(&cookie->backing_objects))
- goto nobufs;
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- op = fscache_alloc_retrieval(cookie, page->mapping, NULL, NULL);
- if (!op)
- return -ENOMEM;
- atomic_set(&op->n_pages, 1);
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_alloc_one);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs_unlock;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, &op->op) < 0)
- goto nobufs_unlock_dec;
- spin_unlock(&cookie->lock);
-
- fscache_stat(&fscache_n_alloc_ops);
-
- ret = fscache_wait_for_operation_activation(
- object, &op->op,
- __fscache_stat(&fscache_n_alloc_op_waits),
- __fscache_stat(&fscache_n_allocs_object_dead));
- if (ret < 0)
- goto error;
-
- /* ask the cache to honour the operation */
- fscache_stat(&fscache_n_cop_allocate_page);
- ret = object->cache->ops->allocate_page(op, page, gfp);
- fscache_stat_d(&fscache_n_cop_allocate_page);
-
-error:
- if (ret == -ERESTARTSYS)
- fscache_stat(&fscache_n_allocs_intr);
- else if (ret < 0)
- fscache_stat(&fscache_n_allocs_nobufs);
- else
- fscache_stat(&fscache_n_allocs_ok);
-
- fscache_put_retrieval(op);
- _leave(" = %d", ret);
- return ret;
-
-nobufs_unlock_dec:
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs_unlock:
- spin_unlock(&cookie->lock);
- fscache_put_retrieval(op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
-nobufs:
- fscache_stat(&fscache_n_allocs_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_alloc_page);
-
-/*
- * Unmark pages allocate in the readahead code path (via:
- * fscache_readpages_or_alloc) after delegating to the base filesystem
- */
-void __fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages)
-{
- struct page *page;
-
- list_for_each_entry(page, pages, lru) {
- if (PageFsCache(page))
- __fscache_uncache_page(cookie, page);
- }
-}
-EXPORT_SYMBOL(__fscache_readpages_cancel);
-
-/*
- * release a write op reference
- */
-static void fscache_release_write_op(struct fscache_operation *_op)
-{
- _enter("{OP%x}", _op->debug_id);
-}
-
-/*
- * perform the background storage of a page into the cache
- */
-static void fscache_write_op(struct fscache_operation *_op)
-{
- struct fscache_storage *op =
- container_of(_op, struct fscache_storage, op);
- struct fscache_object *object = op->op.object;
- struct fscache_cookie *cookie;
- struct page *page;
- unsigned n;
- void *results[1];
- int ret;
-
- _enter("{OP%x,%d}", op->op.debug_id, atomic_read(&op->op.usage));
-
-again:
- spin_lock(&object->lock);
- cookie = object->cookie;
-
- if (!fscache_object_is_active(object)) {
- /* If we get here, then the on-disk cache object likely no
- * longer exists, so we should just cancel this write
- * operation.
- */
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, true);
- _leave(" [inactive]");
- return;
- }
-
- if (!cookie) {
- /* If we get here, then the cookie belonging to the object was
- * detached, probably by the cookie being withdrawn due to
- * memory pressure, which means that the pages we might write
- * to the cache from no longer exist - therefore, we can just
- * cancel this write operation.
- */
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, true);
- _leave(" [cancel] op{f=%lx s=%u} obj{s=%s f=%lx}",
- _op->flags, _op->state, object->state->short_name,
- object->flags);
- return;
- }
-
- spin_lock(&cookie->stores_lock);
-
- fscache_stat(&fscache_n_store_calls);
-
- /* find a page to store */
- results[0] = NULL;
- page = NULL;
- n = radix_tree_gang_lookup_tag(&cookie->stores, results, 0, 1,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_gang_lookup(cookie, &op->op, results, n, op->store_limit);
- if (n != 1)
- goto superseded;
- page = results[0];
- _debug("gang %d [%lx]", n, page->index);
-
- radix_tree_tag_set(&cookie->stores, page->index,
- FSCACHE_COOKIE_STORING_TAG);
- radix_tree_tag_clear(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_pend2store);
-
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-
- if (page->index >= op->store_limit)
- goto discard_page;
-
- fscache_stat(&fscache_n_store_pages);
- fscache_stat(&fscache_n_cop_write_page);
- ret = object->cache->ops->write_page(op, page);
- fscache_stat_d(&fscache_n_cop_write_page);
- trace_fscache_wrote_page(cookie, page, &op->op, ret);
- fscache_end_page_write(object, page);
- if (ret < 0) {
- fscache_abort_object(object);
- fscache_op_complete(&op->op, true);
- } else {
- fscache_enqueue_operation(&op->op);
- }
-
- _leave("");
- return;
-
-discard_page:
- fscache_stat(&fscache_n_store_pages_over_limit);
- trace_fscache_wrote_page(cookie, page, &op->op, -ENOBUFS);
- fscache_end_page_write(object, page);
- goto again;
-
-superseded:
- /* this writer is going away and there aren't any more things to
- * write */
- _debug("cease");
- spin_unlock(&cookie->stores_lock);
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- spin_unlock(&object->lock);
- fscache_op_complete(&op->op, false);
- _leave("");
-}
-
-/*
- * Clear the pages pending writing for invalidation
- */
-void fscache_invalidate_writes(struct fscache_cookie *cookie)
-{
- struct page *page;
- void *results[16];
- int n, i;
-
- _enter("");
-
- for (;;) {
- spin_lock(&cookie->stores_lock);
- n = radix_tree_gang_lookup_tag(&cookie->stores, results, 0,
- ARRAY_SIZE(results),
- FSCACHE_COOKIE_PENDING_TAG);
- if (n == 0) {
- spin_unlock(&cookie->stores_lock);
- break;
- }
-
- for (i = n - 1; i >= 0; i--) {
- page = results[i];
- radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- trace_fscache_page(cookie, page, fscache_page_inval);
- }
-
- spin_unlock(&cookie->stores_lock);
-
- for (i = n - 1; i >= 0; i--)
- put_page(results[i]);
- }
-
- wake_up_bit(&cookie->flags, 0);
- trace_fscache_wake_cookie(cookie);
-
- _leave("");
-}
-
-/*
- * request a page be stored in the cache
- * - returns:
- * -ENOMEM - out of memory, nothing done
- * -ENOBUFS - no backing object available in which to cache the page
- * 0 - dispatched a write - it'll call end_io_func() when finished
- *
- * if the cookie still has a backing object at this point, that object can be
- * in one of a few states with respect to storage processing:
- *
- * (1) negative lookup, object not yet created (FSCACHE_COOKIE_CREATING is
- * set)
- *
- * (a) no writes yet
- *
- * (b) writes deferred till post-creation (mark page for writing and
- * return immediately)
- *
- * (2) negative lookup, object created, initial fill being made from netfs
- *
- * (a) fill point not yet reached this page (mark page for writing and
- * return)
- *
- * (b) fill point passed this page (queue op to store this page)
- *
- * (3) object extant (queue op to store this page)
- *
- * any other state is invalid
- */
-int __fscache_write_page(struct fscache_cookie *cookie,
- struct page *page,
- loff_t object_size,
- gfp_t gfp)
-{
- struct fscache_storage *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,%x,", cookie, (u32) page->flags);
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERT(PageFsCache(page));
-
- fscache_stat(&fscache_n_stores);
-
- if (test_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags)) {
- _leave(" = -ENOBUFS [invalidating]");
- return -ENOBUFS;
- }
-
- op = kzalloc(sizeof(*op), GFP_NOIO | __GFP_NOMEMALLOC | __GFP_NORETRY);
- if (!op)
- goto nomem;
-
- fscache_operation_init(cookie, &op->op, fscache_write_op, NULL,
- fscache_release_write_op);
- op->op.flags = FSCACHE_OP_ASYNC |
- (1 << FSCACHE_OP_WAITING) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
-
- ret = radix_tree_maybe_preload(gfp & ~__GFP_HIGHMEM);
- if (ret < 0)
- goto nomem_free;
-
- trace_fscache_page_op(cookie, page, &op->op, fscache_page_op_write_one);
-
- ret = -ENOBUFS;
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
- if (test_bit(FSCACHE_IOERROR, &object->cache->flags))
- goto nobufs;
-
- trace_fscache_page(cookie, page, fscache_page_write);
-
- /* add the page to the pending-storage radix tree on the backing
- * object */
- spin_lock(&object->lock);
-
- if (object->store_limit_l != object_size)
- fscache_set_store_limit(object, object_size);
-
- spin_lock(&cookie->stores_lock);
-
- _debug("store limit %llx", (unsigned long long) object->store_limit);
-
- ret = radix_tree_insert(&cookie->stores, page->index, page);
- if (ret < 0) {
- if (ret == -EEXIST)
- goto already_queued;
- _debug("insert failed %d", ret);
- goto nobufs_unlock_obj;
- }
-
- trace_fscache_page(cookie, page, fscache_page_radix_insert);
- radix_tree_tag_set(&cookie->stores, page->index,
- FSCACHE_COOKIE_PENDING_TAG);
- trace_fscache_page(cookie, page, fscache_page_radix_set_pend);
- get_page(page);
-
- /* we only want one writer at a time, but we do need to queue new
- * writers after exclusive ops */
- if (test_and_set_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags))
- goto already_pending;
-
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-
- op->op.debug_id = atomic_inc_return(&fscache_op_debug_id);
- op->store_limit = object->store_limit;
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, &op->op) < 0)
- goto submit_failed;
-
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_stat(&fscache_n_store_ops);
- fscache_stat(&fscache_n_stores_ok);
-
- /* the work queue now carries its own ref on the object */
- fscache_put_operation(&op->op);
- _leave(" = 0");
- return 0;
-
-already_queued:
- fscache_stat(&fscache_n_stores_again);
-already_pending:
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_put_operation(&op->op);
- fscache_stat(&fscache_n_stores_ok);
- _leave(" = 0");
- return 0;
-
-submit_failed:
- spin_lock(&cookie->stores_lock);
- radix_tree_delete(&cookie->stores, page->index);
- trace_fscache_page(cookie, page, fscache_page_radix_delete);
- spin_unlock(&cookie->stores_lock);
- wake_cookie = __fscache_unuse_cookie(cookie);
- put_page(page);
- ret = -ENOBUFS;
- goto nobufs;
-
-nobufs_unlock_obj:
- spin_unlock(&cookie->stores_lock);
- spin_unlock(&object->lock);
-nobufs:
- spin_unlock(&cookie->lock);
- radix_tree_preload_end();
- fscache_put_operation(&op->op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- fscache_stat(&fscache_n_stores_nobufs);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
-
-nomem_free:
- fscache_put_operation(&op->op);
-nomem:
- fscache_stat(&fscache_n_stores_oom);
- _leave(" = -ENOMEM");
- return -ENOMEM;
-}
-EXPORT_SYMBOL(__fscache_write_page);
-
-/*
- * remove a page from the cache
- */
-void __fscache_uncache_page(struct fscache_cookie *cookie, struct page *page)
-{
- struct fscache_object *object;
-
- _enter(",%p", page);
-
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
- ASSERTCMP(page, !=, NULL);
-
- fscache_stat(&fscache_n_uncaches);
-
- /* cache withdrawal may beat us to it */
- if (!PageFsCache(page))
- goto done;
-
- trace_fscache_page(cookie, page, fscache_page_uncache);
-
- /* get the object */
- spin_lock(&cookie->lock);
-
- if (hlist_empty(&cookie->backing_objects)) {
- ClearPageFsCache(page);
- goto done_unlock;
- }
-
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- /* there might now be stuff on disk we could read */
- clear_bit(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
-
- /* only invoke the cache backend if we managed to mark the page
- * uncached here; this deals with synchronisation vs withdrawal */
- if (TestClearPageFsCache(page) &&
- object->cache->ops->uncache_page) {
- /* the cache backend releases the cookie lock */
- fscache_stat(&fscache_n_cop_uncache_page);
- object->cache->ops->uncache_page(object, page);
- fscache_stat_d(&fscache_n_cop_uncache_page);
- goto done;
- }
-
-done_unlock:
- spin_unlock(&cookie->lock);
-done:
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_uncache_page);
-
-/**
- * fscache_mark_page_cached - Mark a page as being cached
- * @op: The retrieval op pages are being marked for
- * @page: The page to be marked
- *
- * Mark a netfs page as being cached. After this is called, the netfs
- * must call fscache_uncache_page() to remove the mark.
- */
-void fscache_mark_page_cached(struct fscache_retrieval *op, struct page *page)
-{
- struct fscache_cookie *cookie = op->op.object->cookie;
-
-#ifdef CONFIG_FSCACHE_STATS
- atomic_inc(&fscache_n_marks);
-#endif
-
- trace_fscache_page(cookie, page, fscache_page_cached);
-
- _debug("- mark %p{%lx}", page, page->index);
- if (TestSetPageFsCache(page)) {
- static bool once_only;
- if (!once_only) {
- once_only = true;
- pr_warn("Cookie type %s marked page %lx multiple times\n",
- cookie->def->name, page->index);
- }
- }
-
- if (cookie->def->mark_page_cached)
- cookie->def->mark_page_cached(cookie->netfs_data,
- op->mapping, page);
-}
-EXPORT_SYMBOL(fscache_mark_page_cached);
-
-/**
- * fscache_mark_pages_cached - Mark pages as being cached
- * @op: The retrieval op pages are being marked for
- * @pagevec: The pages to be marked
- *
- * Mark a bunch of netfs pages as being cached. After this is called,
- * the netfs must call fscache_uncache_page() to remove the mark.
- */
-void fscache_mark_pages_cached(struct fscache_retrieval *op,
- struct pagevec *pagevec)
-{
- unsigned long loop;
-
- for (loop = 0; loop < pagevec->nr; loop++)
- fscache_mark_page_cached(op, pagevec->pages[loop]);
-
- pagevec_reinit(pagevec);
-}
-EXPORT_SYMBOL(fscache_mark_pages_cached);
-
-/*
- * Uncache all the pages in an inode that are marked PG_fscache, assuming them
- * to be associated with the given cookie.
- */
-void __fscache_uncache_all_inode_pages(struct fscache_cookie *cookie,
- struct inode *inode)
-{
- struct address_space *mapping = inode->i_mapping;
- struct pagevec pvec;
- pgoff_t next;
- int i;
-
- _enter("%p,%p", cookie, inode);
-
- if (!mapping || mapping->nrpages == 0) {
- _leave(" [no pages]");
- return;
- }
-
- pagevec_init(&pvec);
- next = 0;
- do {
- if (!pagevec_lookup(&pvec, mapping, &next))
- break;
- for (i = 0; i < pagevec_count(&pvec); i++) {
- struct page *page = pvec.pages[i];
- if (PageFsCache(page)) {
- __fscache_wait_on_page_write(cookie, page);
- __fscache_uncache_page(cookie, page);
- }
- }
- pagevec_release(&pvec);
- cond_resched();
- } while (next);
-
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_uncache_all_inode_pages);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index a5aa93ece8c5..281022871e70 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -124,7 +124,6 @@ atomic_t fscache_n_cop_allocate_page;
atomic_t fscache_n_cop_allocate_pages;
atomic_t fscache_n_cop_write_page;
atomic_t fscache_n_cop_uncache_page;
-atomic_t fscache_n_cop_dissociate_pages;
atomic_t fscache_n_cache_no_space_reject;
atomic_t fscache_n_cache_stale_objects;
@@ -265,14 +264,13 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
atomic_read(&fscache_n_cop_sync_cache));
- seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d dsp=%d\n",
+ seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d\n",
atomic_read(&fscache_n_cop_read_or_alloc_page),
atomic_read(&fscache_n_cop_read_or_alloc_pages),
atomic_read(&fscache_n_cop_allocate_page),
atomic_read(&fscache_n_cop_allocate_pages),
atomic_read(&fscache_n_cop_write_page),
- atomic_read(&fscache_n_cop_uncache_page),
- atomic_read(&fscache_n_cop_dissociate_pages));
+ atomic_read(&fscache_n_cop_uncache_page));
seq_printf(m, "CacheEv: nsp=%d stl=%d rtr=%d cul=%d\n",
atomic_read(&fscache_n_cache_no_space_reject),
atomic_read(&fscache_n_cache_stale_objects),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index d5ba431b5d63..2db4ffbf4523 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -137,88 +137,6 @@ extern void fscache_operation_init(struct fscache_cookie *,
fscache_operation_cancel_t,
fscache_operation_release_t);
-/*
- * data read operation
- */
-struct fscache_retrieval {
- struct fscache_operation op;
- struct fscache_cookie *cookie; /* The netfs cookie */
- struct address_space *mapping; /* netfs pages */
- fscache_rw_complete_t end_io_func; /* function to call on I/O completion */
- void *context; /* netfs read context (pinned) */
- struct list_head to_do; /* list of things to be done by the backend */
- unsigned long start_time; /* time at which retrieval started */
- atomic_t n_pages; /* number of pages to be retrieved */
-};
-
-typedef int (*fscache_page_retrieval_func_t)(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp);
-
-typedef int (*fscache_pages_retrieval_func_t)(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp);
-
-/**
- * fscache_get_retrieval - Get an extra reference on a retrieval operation
- * @op: The retrieval operation to get a reference on
- *
- * Get an extra reference on a retrieval operation.
- */
-static inline
-struct fscache_retrieval *fscache_get_retrieval(struct fscache_retrieval *op)
-{
- atomic_inc(&op->op.usage);
- return op;
-}
-
-/**
- * fscache_enqueue_retrieval - Enqueue a retrieval operation for processing
- * @op: The retrieval operation affected
- *
- * Enqueue a retrieval operation for processing by the FS-Cache thread pool.
- */
-static inline void fscache_enqueue_retrieval(struct fscache_retrieval *op)
-{
- fscache_enqueue_operation(&op->op);
-}
-
-/**
- * fscache_retrieval_complete - Record (partial) completion of a retrieval
- * @op: The retrieval operation affected
- * @n_pages: The number of pages to account for
- */
-static inline void fscache_retrieval_complete(struct fscache_retrieval *op,
- int n_pages)
-{
- if (atomic_sub_return_relaxed(n_pages, &op->n_pages) <= 0)
- fscache_op_complete(&op->op, false);
-}
-
-/**
- * fscache_put_retrieval - Drop a reference to a retrieval operation
- * @op: The retrieval operation affected
- *
- * Drop a reference to a retrieval operation.
- */
-static inline void fscache_put_retrieval(struct fscache_retrieval *op)
-{
- fscache_put_operation(&op->op);
-}
-
-/*
- * cached page storage work item
- * - used to do three things:
- * - batch writes to the cache
- * - do cache writes asynchronously
- * - defer writes until cache object lookup completion
- */
-struct fscache_storage {
- struct fscache_operation op;
- pgoff_t store_limit; /* don't write more than this */
-};
-
/*
* cache operations
*/
@@ -275,35 +193,6 @@ struct fscache_cache_ops {
/* reserve space for an object's data and associated metadata */
int (*reserve_space)(struct fscache_object *object, loff_t i_size);
-
- /* request a backing block for a page be read or allocated in the
- * cache */
- fscache_page_retrieval_func_t read_or_alloc_page;
-
- /* request backing blocks for a list of pages be read or allocated in
- * the cache */
- fscache_pages_retrieval_func_t read_or_alloc_pages;
-
- /* request a backing block for a page be allocated in the cache so that
- * it can be written directly */
- fscache_page_retrieval_func_t allocate_page;
-
- /* request backing blocks for pages be allocated in the cache so that
- * they can be written directly */
- fscache_pages_retrieval_func_t allocate_pages;
-
- /* write a page to its backing block in the cache */
- int (*write_page)(struct fscache_storage *op, struct page *page);
-
- /* detach backing block from a page (optional)
- * - must release the cookie lock before returning
- * - may sleep
- */
- void (*uncache_page)(struct fscache_object *object,
- struct page *page);
-
- /* dissociate a cache from all the pages it was backing */
- void (*dissociate_pages)(struct fscache_cache *cache);
};
extern struct fscache_cookie fscache_fsdef_index;
@@ -466,21 +355,6 @@ void fscache_set_store_limit(struct fscache_object *object, loff_t i_size)
object->store_limit++;
}
-/**
- * fscache_end_io - End a retrieval operation on a page
- * @op: The FS-Cache operation covering the retrieval
- * @page: The page that was to be fetched
- * @error: The error code (0 if successful)
- *
- * Note the end of an operation to retrieve a page, as covered by a particular
- * operation record.
- */
-static inline void fscache_end_io(struct fscache_retrieval *op,
- struct page *page, int error)
-{
- op->end_io_func(page, op->context, error);
-}
-
static inline void __fscache_use_cookie(struct fscache_cookie *cookie)
{
atomic_inc(&cookie->n_active);
@@ -538,12 +412,6 @@ extern void fscache_withdraw_cache(struct fscache_cache *cache);
extern void fscache_io_error(struct fscache_cache *cache);
-extern void fscache_mark_page_cached(struct fscache_retrieval *op,
- struct page *page);
-
-extern void fscache_mark_pages_cached(struct fscache_retrieval *op,
- struct pagevec *pagevec);
-
extern bool fscache_object_sleep_till_congested(signed long *timeoutp);
extern enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
diff --git a/include/linux/fscache-obsolete.h b/include/linux/fscache-obsolete.h
new file mode 100644
index 000000000000..8d6d3a3b0d0a
--- /dev/null
+++ b/include/linux/fscache-obsolete.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* Obsolete fscache bits
+ *
+ * Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
+ */
+
+#ifndef _LINUX_FSCACHE_OBSOLETE_H
+#define _LINUX_FSCACHE_OBSOLETE_H
+
+
+
+#endif /* _LINUX_FSCACHE_OBSOLETE_H */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 38ec80217899..029a2ddd9d93 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -47,10 +47,6 @@ struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
-typedef void (*fscache_rw_complete_t)(struct page *page,
- void *context,
- int error);
-
/* result of index entry consultation */
enum fscache_checkaux {
FSCACHE_CHECKAUX_OKAY, /* entry okay as is */
@@ -89,27 +85,6 @@ struct fscache_cookie_def {
const void *data,
uint16_t datalen,
loff_t object_size);
-
- /* get an extra reference on a read context
- * - this function can be absent if the completion function doesn't
- * require a context
- */
- void (*get_context)(void *cookie_netfs_data, void *context);
-
- /* release an extra reference on a read context
- * - this function can be absent if the completion function doesn't
- * require a context
- */
- void (*put_context)(void *cookie_netfs_data, void *context);
-
- /* indicate page that now have cache metadata retained
- * - this function should mark the specified page as now being cached
- * - the page will have been marked with PG_fscache before this is
- * called, so this is optional
- */
- void (*mark_page_cached)(void *cookie_netfs_data,
- struct address_space *mapping,
- struct page *page);
};
/*
@@ -136,16 +111,12 @@ struct fscache_cookie {
atomic_t n_active; /* number of active users of netfs ptrs */
unsigned int debug_id;
spinlock_t lock;
- spinlock_t stores_lock; /* lock on page store tree */
struct hlist_head backing_objects; /* object(s) backing this file/index */
const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
void *netfs_data; /* back pointer to netfs */
- struct radix_tree_root stores; /* pages to be stored on this cookie */
-#define FSCACHE_COOKIE_PENDING_TAG 0 /* pages tag: pending write to cache */
-#define FSCACHE_COOKIE_STORING_TAG 1 /* pages tag: writing to cache */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
@@ -202,29 +173,6 @@ extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
extern int __fscache_attr_changed(struct fscache_cookie *);
extern void __fscache_invalidate(struct fscache_cookie *);
extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
-extern int __fscache_read_or_alloc_page(struct fscache_cookie *,
- struct page *,
- fscache_rw_complete_t,
- void *,
- gfp_t);
-extern int __fscache_read_or_alloc_pages(struct fscache_cookie *,
- struct address_space *,
- struct list_head *,
- unsigned *,
- fscache_rw_complete_t,
- void *,
- gfp_t);
-extern int __fscache_alloc_page(struct fscache_cookie *, struct page *, gfp_t);
-extern int __fscache_write_page(struct fscache_cookie *, struct page *, loff_t, gfp_t);
-extern void __fscache_uncache_page(struct fscache_cookie *, struct page *);
-extern bool __fscache_check_page_write(struct fscache_cookie *, struct page *);
-extern void __fscache_wait_on_page_write(struct fscache_cookie *, struct page *);
-extern bool __fscache_maybe_release_page(struct fscache_cookie *, struct page *,
- gfp_t);
-extern void __fscache_uncache_all_inode_pages(struct fscache_cookie *,
- struct inode *);
-extern void __fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages);
extern void __fscache_disable_cookie(struct fscache_cookie *, const void *, bool);
extern void __fscache_enable_cookie(struct fscache_cookie *, const void *, loff_t,
bool (*)(void *), void *);
@@ -491,303 +439,6 @@ void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
__fscache_wait_on_invalidate(cookie);
}
-/**
- * fscache_reserve_space - Reserve data space for a cached object
- * @cookie: The cookie representing the cache object
- * @i_size: The amount of space to be reserved
- *
- * Reserve an amount of space in the cache for the cache object attached to a
- * cookie so that a write to that object within the space can always be
- * honoured.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_reserve_space(struct fscache_cookie *cookie, loff_t size)
-{
- return -ENOBUFS;
-}
-
-/**
- * fscache_read_or_alloc_page - Read a page from the cache or allocate a block
- * in which to store it
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to fill if possible
- * @end_io_func: The callback to invoke when and if the page is filled
- * @context: An arbitrary piece of data to pass on to end_io_func()
- * @gfp: The conditions under which memory allocation should be made
- *
- * Read a page from the cache, or if that's not possible make a potential
- * one-block reservation in the cache into which the page may be stored once
- * fetched from the server.
- *
- * If the page is not backed by the cache object, or if it there's some reason
- * it can't be, -ENOBUFS will be returned and nothing more will be done for
- * that page.
- *
- * Else, if that page is backed by the cache, a read will be initiated directly
- * to the netfs's page and 0 will be returned by this function. The
- * end_io_func() callback will be invoked when the operation terminates on a
- * completion or failure. Note that the callback may be invoked before the
- * return.
- *
- * Else, if the page is unbacked, -ENODATA is returned and a block may have
- * been allocated in the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_read_or_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_read_or_alloc_page(cookie, page, end_io_func,
- context, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_read_or_alloc_pages - Read pages from the cache and/or allocate
- * blocks in which to store them
- * @cookie: The cookie representing the cache object
- * @mapping: The netfs inode mapping to which the pages will be attached
- * @pages: A list of potential netfs pages to be filled
- * @nr_pages: Number of pages to be read and/or allocated
- * @end_io_func: The callback to invoke when and if each page is filled
- * @context: An arbitrary piece of data to pass on to end_io_func()
- * @gfp: The conditions under which memory allocation should be made
- *
- * Read a set of pages from the cache, or if that's not possible, attempt to
- * make a potential one-block reservation for each page in the cache into which
- * that page may be stored once fetched from the server.
- *
- * If some pages are not backed by the cache object, or if it there's some
- * reason they can't be, -ENOBUFS will be returned and nothing more will be
- * done for that pages.
- *
- * Else, if some of the pages are backed by the cache, a read will be initiated
- * directly to the netfs's page and 0 will be returned by this function. The
- * end_io_func() callback will be invoked when the operation terminates on a
- * completion or failure. Note that the callback may be invoked before the
- * return.
- *
- * Else, if a page is unbacked, -ENODATA is returned and a block may have
- * been allocated in the cache.
- *
- * Because the function may want to return all of -ENOBUFS, -ENODATA and 0 in
- * regard to different pages, the return values are prioritised in that order.
- * Any pages submitted for reading are removed from the pages list.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
- struct address_space *mapping,
- struct list_head *pages,
- unsigned *nr_pages,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_read_or_alloc_pages(cookie, mapping, pages,
- nr_pages, end_io_func,
- context, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_alloc_page - Allocate a block in which to store a page
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to allocate a page for
- * @gfp: The conditions under which memory allocation should be made
- *
- * Request Allocation a block in the cache in which to store a netfs page
- * without retrieving any contents from the cache.
- *
- * If the page is not backed by a file then -ENOBUFS will be returned and
- * nothing more will be done, and no reservation will be made.
- *
- * Else, a block will be allocated if one wasn't already, and 0 will be
- * returned
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_alloc_page(cookie, page, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_readpages_cancel - Cancel read/alloc on pages
- * @cookie: The cookie representing the inode's cache object.
- * @pages: The netfs pages that we canceled write on in readpages()
- *
- * Uncache/unreserve the pages reserved earlier in readpages() via
- * fscache_readpages_or_alloc() and similar. In most successful caches in
- * readpages() this doesn't do anything. In cases when the underlying netfs's
- * readahead failed we need to clean up the pagelist (unmark and uncache).
- *
- * This function may sleep as it may have to clean up disk state.
- */
-static inline
-void fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_readpages_cancel(cookie, pages);
-}
-
-/**
- * fscache_write_page - Request storage of a page in the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page to store
- * @object_size: Updated size of object
- * @gfp: The conditions under which memory allocation should be made
- *
- * Request the contents of the netfs page be written into the cache. This
- * request may be ignored if no cache block is currently allocated, in which
- * case it will return -ENOBUFS.
- *
- * If a cache block was already allocated, a write will be initiated and 0 will
- * be returned. The PG_fscache_write page bit is set immediately and will then
- * be cleared at the completion of the write to indicate the success or failure
- * of the operation. Note that the completion may happen before the return.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_write_page(struct fscache_cookie *cookie,
- struct page *page,
- loff_t object_size,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_write_page(cookie, page, object_size, gfp);
- else
- return -ENOBUFS;
-}
-
-/**
- * fscache_uncache_page - Indicate that caching is no longer required on a page
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that was being cached.
- *
- * Tell the cache that we no longer want a page to be cached and that it should
- * remove any knowledge of the netfs page it may have.
- *
- * Note that this cannot cancel any outstanding I/O operations between this
- * page and the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_uncache_page(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_uncache_page(cookie, page);
-}
-
-/**
- * fscache_check_page_write - Ask if a page is being writing to the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- *
- * Ask the cache if a page is being written to the cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-bool fscache_check_page_write(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- return __fscache_check_page_write(cookie, page);
- return false;
-}
-
-/**
- * fscache_wait_on_page_write - Wait for a page to complete writing to the cache
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- *
- * Ask the cache to wake us up when a page is no longer being written to the
- * cache.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_wait_on_page_write(struct fscache_cookie *cookie,
- struct page *page)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_wait_on_page_write(cookie, page);
-}
-
-/**
- * fscache_maybe_release_page - Consider releasing a page, cancelling a store
- * @cookie: The cookie representing the cache object
- * @page: The netfs page that is being cached.
- * @gfp: The gfp flags passed to releasepage()
- *
- * Consider releasing a page for the vmscan algorithm, on behalf of the netfs's
- * releasepage() call. A storage request on the page may cancelled if it is
- * not currently being processed.
- *
- * The function returns true if the page no longer has a storage request on it,
- * and false if a storage request is left in place. If true is returned, the
- * page will have been passed to fscache_uncache_page(). If false is returned
- * the page cannot be freed yet.
- */
-static inline
-bool fscache_maybe_release_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp)
-{
- if (fscache_cookie_valid(cookie) && PageFsCache(page))
- return __fscache_maybe_release_page(cookie, page, gfp);
- return true;
-}
-
-/**
- * fscache_uncache_all_inode_pages - Uncache all an inode's pages
- * @cookie: The cookie representing the inode's cache object.
- * @inode: The inode to uncache pages from.
- *
- * Uncache all the pages in an inode that are marked PG_fscache, assuming them
- * to be associated with the given cookie.
- *
- * This function may sleep. It will wait for pages that are being written out
- * and will wait whilst the PG_fscache mark is removed by the cache.
- */
-static inline
-void fscache_uncache_all_inode_pages(struct fscache_cookie *cookie,
- struct inode *inode)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_uncache_all_inode_pages(cookie, inode);
-}
-
/**
* fscache_disable_cookie - Disable a cookie
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 11/61] fscache: Remove the netfs data from the cookie
2020-05-04 17:07 ` David Howells
` (10 preceding siblings ...)
(?)
@ 2020-05-04 17:09 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the netfs data pointer from the cookie so that we don't refer back
to the netfs state and don't need accessors to manage this. Keep the
information we do need (of which there's not actually a lot) in the cookie
which we can keep hold of if the netfs state goes away.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/cache.c | 39 --------
fs/afs/cell.c | 3 -
fs/afs/inode.c | 3 -
fs/afs/volume.c | 4 -
fs/cachefiles/interface.c | 104 +++------------------
fs/cachefiles/internal.h | 20 +---
fs/cachefiles/namei.c | 10 +-
fs/cachefiles/xattr.c | 204 +++++++----------------------------------
fs/fscache/cache.c | 20 +---
fs/fscache/cookie.c | 37 ++++---
fs/fscache/fsdef.c | 37 -------
fs/fscache/internal.h | 29 ++++--
fs/fscache/netfs.c | 3 -
fs/fscache/object-list.c | 9 +-
fs/fscache/object.c | 48 ----------
fs/nfs/fscache-index.c | 4 -
include/linux/fscache-cache.h | 30 +++++-
include/linux/fscache.h | 39 +-------
18 files changed, 143 insertions(+), 500 deletions(-)
diff --git a/fs/afs/cache.c b/fs/afs/cache.c
index 037af93e3aba..9b2de3014dbf 100644
--- a/fs/afs/cache.c
+++ b/fs/afs/cache.c
@@ -8,11 +8,6 @@
#include <linux/sched.h>
#include "internal.h"
-static enum fscache_checkaux afs_vnode_cache_check_aux(void *cookie_netfs_data,
- const void *buffer,
- uint16_t buflen,
- loff_t object_size);
-
struct fscache_netfs afs_cache_netfs = {
.name = "afs",
.version = 2,
@@ -31,38 +26,4 @@ struct fscache_cookie_def afs_volume_cache_index_def = {
struct fscache_cookie_def afs_vnode_cache_index_def = {
.name = "AFS.vnode",
.type = FSCACHE_COOKIE_TYPE_DATAFILE,
- .check_aux = afs_vnode_cache_check_aux,
};
-
-/*
- * check that the auxiliary data indicates that the entry is still valid
- */
-static enum fscache_checkaux afs_vnode_cache_check_aux(void *cookie_netfs_data,
- const void *buffer,
- uint16_t buflen,
- loff_t object_size)
-{
- struct afs_vnode *vnode = cookie_netfs_data;
- struct afs_vnode_cache_aux aux;
-
- _enter("{%llx,%x,%llx},%p,%u",
- vnode->fid.vnode, vnode->fid.unique, vnode->status.data_version,
- buffer, buflen);
-
- memcpy(&aux, buffer, sizeof(aux));
-
- /* check the size of the data is what we're expecting */
- if (buflen != sizeof(aux)) {
- _leave(" = OBSOLETE [len %hx != %zx]", buflen, sizeof(aux));
- return FSCACHE_CHECKAUX_OBSOLETE;
- }
-
- if (vnode->status.data_version != aux.data_version) {
- _leave(" = OBSOLETE [vers %llx != %llx]",
- aux.data_version, vnode->status.data_version);
- return FSCACHE_CHECKAUX_OBSOLETE;
- }
-
- _leave(" = SUCCESS");
- return FSCACHE_CHECKAUX_OKAY;
-}
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index 78ba5f932287..c9fa573126b3 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -593,9 +593,10 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
#ifdef CONFIG_AFS_FSCACHE
cell->cache = fscache_acquire_cookie(afs_cache_netfs.primary_index,
&afs_cell_cache_index_def,
+ NULL,
cell->name, strlen(cell->name),
NULL, 0,
- cell, 0, true);
+ 0, true);
#endif
ret = afs_proc_cell_setup(cell);
if (ret < 0)
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 67ccfbab683c..79e36491d8b4 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -492,9 +492,10 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
vnode->cache = fscache_acquire_cookie(vnode->volume->cache,
&afs_vnode_cache_index_def,
+ NULL,
&key, sizeof(key),
&aux, sizeof(aux),
- vnode, vnode->status.size, true);
+ vnode->status.size, true);
#endif
}
diff --git a/fs/afs/volume.c b/fs/afs/volume.c
index 4310336b9bb8..77d909e7fc60 100644
--- a/fs/afs/volume.c
+++ b/fs/afs/volume.c
@@ -184,9 +184,9 @@ void afs_activate_volume(struct afs_volume *volume)
#ifdef CONFIG_AFS_FSCACHE
volume->cache = fscache_acquire_cookie(volume->cell->cache,
&afs_volume_cache_index_def,
+ NULL,
&volume->vid, sizeof(volume->vid),
- NULL, 0,
- volume, 0, true);
+ NULL, 0, 0, true);
#endif
write_lock(&volume->cell->proc_lock);
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 04d92ad402a4..530ed11bb213 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -7,13 +7,9 @@
#include <linux/slab.h>
#include <linux/mount.h>
+#include <linux/xattr.h>
#include "internal.h"
-struct cachefiles_lookup_data {
- struct cachefiles_xattr *auxdata; /* auxiliary data */
- char *key; /* key path */
-};
-
static int cachefiles_attr_changed(struct fscache_object *_object);
/*
@@ -23,11 +19,9 @@ static struct fscache_object *cachefiles_alloc_object(
struct fscache_cache *_cache,
struct fscache_cookie *cookie)
{
- struct cachefiles_lookup_data *lookup_data;
struct cachefiles_object *object;
struct cachefiles_cache *cache;
- struct cachefiles_xattr *auxdata;
- unsigned keylen, auxlen;
+ unsigned keylen;
void *buffer, *p;
char *key;
@@ -35,10 +29,6 @@ static struct fscache_object *cachefiles_alloc_object(
_enter("{%s},%p,", cache->cache.identifier, cookie);
- lookup_data = kmalloc(sizeof(*lookup_data), cachefiles_gfp);
- if (!lookup_data)
- goto nomem_lookup_data;
-
/* create a new object record and a temporary leaf image */
object = kmem_cache_alloc(cachefiles_object_jar, cachefiles_gfp);
if (!object)
@@ -62,10 +52,7 @@ static struct fscache_object *cachefiles_alloc_object(
goto nomem_buffer;
keylen = cookie->key_len;
- if (keylen <= sizeof(cookie->inline_key))
- p = cookie->inline_key;
- else
- p = cookie->key;
+ p = fscache_get_key(cookie);
memcpy(buffer + 2, p, keylen);
*(uint16_t *)buffer = keylen;
@@ -75,28 +62,13 @@ static struct fscache_object *cachefiles_alloc_object(
/* turn the raw key into something that can work with as a filename */
key = cachefiles_cook_key(buffer, keylen + 2, object->type);
+ kfree(buffer);
if (!key)
goto nomem_key;
- /* get hold of the auxiliary data and prepend the object type */
- auxdata = buffer;
- auxlen = cookie->aux_len;
- if (auxlen) {
- if (auxlen <= sizeof(cookie->inline_aux))
- p = cookie->inline_aux;
- else
- p = cookie->aux;
- memcpy(auxdata->data, p, auxlen);
- }
-
- auxdata->len = auxlen + 1;
- auxdata->type = cookie->type;
-
- lookup_data->auxdata = auxdata;
- lookup_data->key = key;
- object->lookup_data = lookup_data;
+ object->lookup_key = key;
- _leave(" = %p [%p]", &object->fscache, lookup_data);
+ _leave(" = %p [%s]", &object->fscache, key);
return &object->fscache;
nomem_key:
@@ -106,8 +78,6 @@ static struct fscache_object *cachefiles_alloc_object(
kmem_cache_free(cachefiles_object_jar, object);
fscache_object_destroyed(&cache->cache);
nomem_object:
- kfree(lookup_data);
-nomem_lookup_data:
_leave(" = -ENOMEM");
return ERR_PTR(-ENOMEM);
}
@@ -118,7 +88,6 @@ static struct fscache_object *cachefiles_alloc_object(
*/
static int cachefiles_lookup_object(struct fscache_object *_object)
{
- struct cachefiles_lookup_data *lookup_data;
struct cachefiles_object *parent, *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
@@ -130,15 +99,12 @@ static int cachefiles_lookup_object(struct fscache_object *_object)
parent = container_of(_object->parent,
struct cachefiles_object, fscache);
object = container_of(_object, struct cachefiles_object, fscache);
- lookup_data = object->lookup_data;
- ASSERTCMP(lookup_data, !=, NULL);
+ ASSERTCMP(object->lookup_key, !=, NULL);
/* look up the key, creating any missing bits */
cachefiles_begin_secure(cache, &saved_cred);
- ret = cachefiles_walk_to_object(parent, object,
- lookup_data->key,
- lookup_data->auxdata);
+ ret = cachefiles_walk_to_object(parent, object, object->lookup_key);
cachefiles_end_secure(cache, saved_cred);
/* polish off by setting the attributes of non-index files */
@@ -165,14 +131,10 @@ static void cachefiles_lookup_complete(struct fscache_object *_object)
object = container_of(_object, struct cachefiles_object, fscache);
- _enter("{OBJ%x,%p}", object->fscache.debug_id, object->lookup_data);
+ _enter("{OBJ%x}", object->fscache.debug_id);
- if (object->lookup_data) {
- kfree(object->lookup_data->key);
- kfree(object->lookup_data->auxdata);
- kfree(object->lookup_data);
- object->lookup_data = NULL;
- }
+ kfree(object->lookup_key);
+ object->lookup_key = NULL;
}
/*
@@ -204,12 +166,8 @@ struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
static void cachefiles_update_object(struct fscache_object *_object)
{
struct cachefiles_object *object;
- struct cachefiles_xattr *auxdata;
struct cachefiles_cache *cache;
- struct fscache_cookie *cookie;
const struct cred *saved_cred;
- const void *aux;
- unsigned auxlen;
_enter("{OBJ%x}", _object->debug_id);
@@ -217,40 +175,9 @@ static void cachefiles_update_object(struct fscache_object *_object)
cache = container_of(object->fscache.cache, struct cachefiles_cache,
cache);
- if (!fscache_use_cookie(_object)) {
- _leave(" [relinq]");
- return;
- }
-
- cookie = object->fscache.cookie;
- auxlen = cookie->aux_len;
-
- if (!auxlen) {
- fscache_unuse_cookie(_object);
- _leave(" [no aux]");
- return;
- }
-
- auxdata = kmalloc(2 + auxlen + 3, cachefiles_gfp);
- if (!auxdata) {
- fscache_unuse_cookie(_object);
- _leave(" [nomem]");
- return;
- }
-
- aux = (auxlen <= sizeof(cookie->inline_aux)) ?
- cookie->inline_aux : cookie->aux;
-
- memcpy(auxdata->data, aux, auxlen);
- fscache_unuse_cookie(_object);
-
- auxdata->len = auxlen + 1;
- auxdata->type = cookie->type;
-
cachefiles_begin_secure(cache, &saved_cred);
- cachefiles_update_object_xattr(object, auxdata);
+ cachefiles_set_object_xattr(object, XATTR_REPLACE);
cachefiles_end_secure(cache, saved_cred);
- kfree(auxdata);
_leave("");
}
@@ -354,12 +281,7 @@ static void cachefiles_put_object(struct fscache_object *_object,
ASSERTCMP(object->fscache.n_ops, ==, 0);
ASSERTCMP(object->fscache.n_children, ==, 0);
- if (object->lookup_data) {
- kfree(object->lookup_data->key);
- kfree(object->lookup_data->auxdata);
- kfree(object->lookup_data);
- object->lookup_data = NULL;
- }
+ kfree(object->lookup_key);
cache = object->fscache.cache;
fscache_object_destroy(&object->fscache);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index aca73c8403ab..a5d48f271ce1 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -34,7 +34,7 @@ extern unsigned cachefiles_debug;
*/
struct cachefiles_object {
struct fscache_object fscache; /* fscache handle */
- struct cachefiles_lookup_data *lookup_data; /* cached lookup data */
+ char *lookup_key; /* key to look up */
struct dentry *dentry; /* the file/dir representing this object */
struct dentry *backer; /* backing file */
loff_t i_size; /* object size */
@@ -88,15 +88,6 @@ struct cachefiles_cache {
char *tag; /* cache binding tag */
};
-/*
- * auxiliary data xattr buffer
- */
-struct cachefiles_xattr {
- uint16_t len;
- uint8_t type;
- uint8_t data[];
-};
-
#include <trace/events/cachefiles.h>
/*
@@ -142,8 +133,7 @@ extern int cachefiles_delete_object(struct cachefiles_cache *cache,
struct cachefiles_object *object);
extern int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct cachefiles_object *object,
- const char *key,
- struct cachefiles_xattr *auxdata);
+ const char *key);
extern struct dentry *cachefiles_get_directory(struct cachefiles_cache *cache,
struct dentry *dir,
const char *name);
@@ -204,12 +194,8 @@ static inline void cachefiles_end_secure(struct cachefiles_cache *cache,
*/
extern int cachefiles_check_object_type(struct cachefiles_object *object);
extern int cachefiles_set_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata);
-extern int cachefiles_update_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata);
+ unsigned int xattr_flags);
extern int cachefiles_check_auxdata(struct cachefiles_object *object);
-extern int cachefiles_check_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata);
extern int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct dentry *dentry);
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index ecc8ecbbfa5a..432002080b83 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -45,11 +45,10 @@ void __cachefiles_printk_object(struct cachefiles_object *object,
spin_lock(&object->fscache.lock);
cookie = object->fscache.cookie;
if (cookie) {
- pr_err("%scookie=%p [pr=%p nd=%p fl=%lx]\n",
+ pr_err("%scookie=%p [pr=%p fl=%lx]\n",
prefix,
object->fscache.cookie,
object->fscache.cookie->parent,
- object->fscache.cookie->netfs_data,
object->fscache.cookie->flags);
pr_err("%skey=[%u] '", prefix, cookie->key_len);
k = (cookie->key_len <= sizeof(cookie->inline_key)) ?
@@ -481,8 +480,7 @@ int cachefiles_delete_object(struct cachefiles_cache *cache,
*/
int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct cachefiles_object *object,
- const char *key,
- struct cachefiles_xattr *auxdata)
+ const char *key)
{
struct cachefiles_cache *cache;
struct dentry *dir, *next = NULL;
@@ -636,7 +634,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
if (!object->new) {
_debug("validate '%pd'", next);
- ret = cachefiles_check_object_xattr(object, auxdata);
+ ret = cachefiles_check_auxdata(object);
if (ret == -ESTALE) {
/* delete the object (the deleter drops the directory
* mutex) */
@@ -671,7 +669,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
if (object->new) {
/* attach data to a newly constructed terminal object */
- ret = cachefiles_set_object_xattr(object, auxdata);
+ ret = cachefiles_set_object_xattr(object, XATTR_CREATE);
if (ret < 0)
goto check_error;
} else {
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 72e42438f3d7..5ab8643ca97a 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -15,6 +15,11 @@
#include <linux/slab.h>
#include "internal.h"
+struct cachefiles_xattr {
+ uint8_t type;
+ uint8_t data[];
+} __packed;
+
static const char cachefiles_xattr_cache[] =
XATTR_USER_PREFIX "CacheFiles.cache";
@@ -97,56 +102,35 @@ int cachefiles_check_object_type(struct cachefiles_object *object)
* set the state xattr on a cache file
*/
int cachefiles_set_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata)
-{
- struct dentry *dentry = object->dentry;
- int ret;
-
- ASSERT(dentry);
-
- _enter("%p,#%d", object, auxdata->len);
-
- /* attempt to install the cache metadata directly */
- _debug("SET #%u", auxdata->len);
-
- clear_bit(FSCACHE_COOKIE_AUX_UPDATED, &object->fscache.cookie->flags);
- ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
- &auxdata->type, auxdata->len,
- XATTR_CREATE);
- if (ret < 0 && ret != -ENOMEM)
- cachefiles_io_error_obj(
- object,
- "Failed to set xattr with error %d", ret);
-
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * update the state xattr on a cache file
- */
-int cachefiles_update_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata)
+ unsigned int xattr_flags)
{
+ struct cachefiles_xattr *buf;
struct dentry *dentry = object->dentry;
+ unsigned int len = object->fscache.cookie->aux_len;
int ret;
if (!dentry)
return -ESTALE;
- _enter("%p,#%d", object, auxdata->len);
+ _enter("%p,#%d", object, len);
- /* attempt to install the cache metadata directly */
- _debug("SET #%u", auxdata->len);
+ buf = kmalloc(sizeof(struct cachefiles_xattr) + len, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ buf->type = object->fscache.cookie->def->type;
+ if (len > 0)
+ memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
clear_bit(FSCACHE_COOKIE_AUX_UPDATED, &object->fscache.cookie->flags);
ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
- &auxdata->type, auxdata->len,
- XATTR_REPLACE);
+ buf, sizeof(struct cachefiles_xattr) + len,
+ xattr_flags);
+ kfree(buf);
if (ret < 0 && ret != -ENOMEM)
cachefiles_io_error_obj(
object,
- "Failed to update xattr with error %d", ret);
+ "Failed to set xattr with error %d", ret);
_leave(" = %d", ret);
return ret;
@@ -157,148 +141,30 @@ int cachefiles_update_object_xattr(struct cachefiles_object *object,
*/
int cachefiles_check_auxdata(struct cachefiles_object *object)
{
- struct cachefiles_xattr *auxbuf;
- enum fscache_checkaux validity;
+ struct cachefiles_xattr *buf;
struct dentry *dentry = object->dentry;
- ssize_t xlen;
- int ret;
+ unsigned int len = object->fscache.cookie->aux_len, tlen;
+ const void *p = fscache_get_aux(object->fscache.cookie);
+ ssize_t ret;
ASSERT(dentry);
ASSERT(d_backing_inode(dentry));
- ASSERT(object->fscache.cookie->def->check_aux);
- auxbuf = kmalloc(sizeof(struct cachefiles_xattr) + 512, GFP_KERNEL);
- if (!auxbuf)
+ tlen = sizeof(struct cachefiles_xattr) + len;
+ buf = kmalloc(tlen, GFP_KERNEL);
+ if (!buf)
return -ENOMEM;
- xlen = vfs_getxattr(dentry, cachefiles_xattr_cache,
- &auxbuf->type, 512 + 1);
- ret = -ESTALE;
- if (xlen < 1 ||
- auxbuf->type != object->fscache.cookie->def->type)
- goto error;
-
- xlen--;
- validity = fscache_check_aux(&object->fscache, &auxbuf->data, xlen,
- i_size_read(d_backing_inode(dentry)));
- if (validity != FSCACHE_CHECKAUX_OKAY)
- goto error;
-
- ret = 0;
-error:
- kfree(auxbuf);
- return ret;
-}
-
-/*
- * check the state xattr on a cache file
- * - return -ESTALE if the object should be deleted
- */
-int cachefiles_check_object_xattr(struct cachefiles_object *object,
- struct cachefiles_xattr *auxdata)
-{
- struct cachefiles_xattr *auxbuf;
- struct dentry *dentry = object->dentry;
- int ret;
-
- _enter("%p,#%d", object, auxdata->len);
-
- ASSERT(dentry);
- ASSERT(d_backing_inode(dentry));
-
- auxbuf = kmalloc(sizeof(struct cachefiles_xattr) + 512, cachefiles_gfp);
- if (!auxbuf) {
- _leave(" = -ENOMEM");
- return -ENOMEM;
- }
-
- /* read the current type label */
- ret = vfs_getxattr(dentry, cachefiles_xattr_cache,
- &auxbuf->type, 512 + 1);
- if (ret < 0) {
- if (ret == -ENODATA)
- goto stale; /* no attribute - power went off
- * mid-cull? */
-
- if (ret == -ERANGE)
- goto bad_type_length;
-
- cachefiles_io_error_obj(object,
- "Can't read xattr on %lu (err %d)",
- d_backing_inode(dentry)->i_ino, -ret);
- goto error;
- }
-
- /* check the on-disk object */
- if (ret < 1)
- goto bad_type_length;
-
- if (auxbuf->type != auxdata->type)
- goto stale;
-
- auxbuf->len = ret;
-
- /* consult the netfs */
- if (object->fscache.cookie->def->check_aux) {
- enum fscache_checkaux result;
- unsigned int dlen;
-
- dlen = auxbuf->len - 1;
-
- _debug("checkaux %s #%u",
- object->fscache.cookie->def->name, dlen);
-
- result = fscache_check_aux(&object->fscache,
- &auxbuf->data, dlen,
- i_size_read(d_backing_inode(dentry)));
-
- switch (result) {
- /* entry okay as is */
- case FSCACHE_CHECKAUX_OKAY:
- goto okay;
-
- /* entry requires update */
- case FSCACHE_CHECKAUX_NEEDS_UPDATE:
- break;
-
- /* entry requires deletion */
- case FSCACHE_CHECKAUX_OBSOLETE:
- goto stale;
-
- default:
- BUG();
- }
-
- /* update the current label */
- ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
- &auxdata->type, auxdata->len,
- XATTR_REPLACE);
- if (ret < 0) {
- cachefiles_io_error_obj(object,
- "Can't update xattr on %lu"
- " (error %d)",
- d_backing_inode(dentry)->i_ino, -ret);
- goto error;
- }
- }
-
-okay:
- ret = 0;
+ ret = vfs_getxattr(dentry, cachefiles_xattr_cache, buf, tlen);
+ if (ret == tlen &&
+ buf->type == object->fscache.cookie->def->type &&
+ memcmp(buf->data, p, len) == 0)
+ ret = 0;
+ else
+ ret = -ESTALE;
-error:
- kfree(auxbuf);
- _leave(" = %d", ret);
+ kfree(buf);
return ret;
-
-bad_type_length:
- pr_err("Cache object %lu xattr length incorrect\n",
- d_backing_inode(dentry)->i_ino);
- ret = -EIO;
- goto error;
-
-stale:
- ret = -ESTALE;
- goto error;
}
/*
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index 7ec80456a700..1c91ecfb8360 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -30,6 +30,7 @@ struct fscache_cache_tag *__fscache_lookup_cache_tag(const char *name)
list_for_each_entry(tag, &fscache_cache_tag_list, link) {
if (strcmp(tag->name, name) == 0) {
atomic_inc(&tag->usage);
+ refcount_inc(&tag->ref);
up_read(&fscache_addremove_sem);
return tag;
}
@@ -44,6 +45,7 @@ struct fscache_cache_tag *__fscache_lookup_cache_tag(const char *name)
return ERR_PTR(-ENOMEM);
atomic_set(&xtag->usage, 1);
+ refcount_set(&xtag->ref, 1);
strcpy(xtag->name, name);
/* write lock, search again and add if still not present */
@@ -52,6 +54,7 @@ struct fscache_cache_tag *__fscache_lookup_cache_tag(const char *name)
list_for_each_entry(tag, &fscache_cache_tag_list, link) {
if (strcmp(tag->name, name) == 0) {
atomic_inc(&tag->usage);
+ refcount_inc(&tag->ref);
up_write(&fscache_addremove_sem);
kfree(xtag);
return tag;
@@ -64,7 +67,7 @@ struct fscache_cache_tag *__fscache_lookup_cache_tag(const char *name)
}
/*
- * release a reference to a cache tag
+ * Unuse a cache tag
*/
void __fscache_release_cache_tag(struct fscache_cache_tag *tag)
{
@@ -77,8 +80,7 @@ void __fscache_release_cache_tag(struct fscache_cache_tag *tag)
tag = NULL;
up_write(&fscache_addremove_sem);
-
- kfree(tag);
+ fscache_put_cache_tag(tag);
}
}
@@ -130,20 +132,10 @@ struct fscache_cache *fscache_select_cache_for_object(
spin_unlock(&cookie->lock);
- if (!cookie->def->select_cache)
- goto no_preference;
-
- /* ask the netfs for its preference */
- tag = cookie->def->select_cache(cookie->parent->netfs_data,
- cookie->netfs_data);
+ tag = cookie->preferred_cache;
if (!tag)
goto no_preference;
- if (tag == ERR_PTR(-ENOMEM)) {
- _leave(" = NULL [nomem tag]");
- return NULL;
- }
-
if (!tag->cache) {
_leave(" = NULL [unbacked tag]");
return NULL;
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index fa002ef7766d..58fa939d67ba 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -43,11 +43,10 @@ static void fscache_print_cookie(struct fscache_cookie *cookie, char prefix)
cookie->flags,
atomic_read(&cookie->n_children),
atomic_read(&cookie->n_active));
- pr_err("%c-cookie d=%p{%s} n=%p\n",
+ pr_err("%c-cookie d=%p{%s}\n",
prefix,
cookie->def,
- cookie->def ? cookie->def->name : "?",
- cookie->netfs_data);
+ cookie->def ? cookie->def->name : "?");
o = READ_ONCE(cookie->backing_objects.first);
if (o) {
@@ -74,6 +73,7 @@ void fscache_free_cookie(struct fscache_cookie *cookie)
kfree(cookie->aux);
if (cookie->key_len > sizeof(cookie->inline_key))
kfree(cookie->key);
+ fscache_put_cache_tag(cookie->preferred_cache);
kmem_cache_free(fscache_cookie_jar, cookie);
}
}
@@ -150,9 +150,9 @@ static atomic_t fscache_cookie_debug_id = ATOMIC_INIT(1);
struct fscache_cookie *fscache_alloc_cookie(
struct fscache_cookie *parent,
const struct fscache_cookie_def *def,
+ struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
- void *netfs_data,
loff_t object_size)
{
struct fscache_cookie *cookie;
@@ -187,7 +187,9 @@ struct fscache_cookie *fscache_alloc_cookie(
cookie->def = def;
cookie->parent = parent;
- cookie->netfs_data = netfs_data;
+
+ cookie->preferred_cache = fscache_get_cache_tag(preferred_cache);
+
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
cookie->type = def->type;
spin_lock_init(&cookie->lock);
@@ -252,7 +254,6 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
* - the top level index cookie for each netfs is stored in the fscache_netfs
* struct upon registration
* - def points to the definition
- * - the netfs_data will be passed to the functions pointed to in *def
* - all attached caches will be searched to see if they contain this object
* - index objects aren't stored on disk until there's a dependent file that
* needs storing
@@ -264,9 +265,9 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *parent,
const struct fscache_cookie_def *def,
+ struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
- void *netfs_data,
loff_t object_size,
bool enable)
{
@@ -274,9 +275,9 @@ struct fscache_cookie *__fscache_acquire_cookie(
BUG_ON(!def);
- _enter("{%s},{%s},%p,%u",
+ _enter("{%s},{%s},%u",
parent ? (char *) parent->def->name : "<no-parent>",
- def->name, netfs_data, enable);
+ def->name, enable);
if (!index_key || !index_key_len || index_key_len > 255 || aux_data_len > 255)
return NULL;
@@ -300,10 +301,10 @@ struct fscache_cookie *__fscache_acquire_cookie(
BUG_ON(def->type == FSCACHE_COOKIE_TYPE_INDEX &&
parent->type != FSCACHE_COOKIE_TYPE_INDEX);
- candidate = fscache_alloc_cookie(parent, def,
+ candidate = fscache_alloc_cookie(parent, def, preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
- netfs_data, object_size);
+ object_size);
if (!candidate) {
fscache_stat(&fscache_n_acquires_oom);
_leave(" [ENOMEM]");
@@ -814,8 +815,8 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
return;
}
- _enter("%p{%s,%p,%d},%d",
- cookie, cookie->def->name, cookie->netfs_data,
+ _enter("%p{%s,%d},%d",
+ cookie, cookie->def->name,
atomic_read(&cookie->n_active), retire);
trace_fscache_relinquish(cookie, retire);
@@ -827,7 +828,6 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
__fscache_disable_cookie(cookie, aux_data, retire);
/* Clear pointers back to the netfs */
- cookie->netfs_data = NULL;
cookie->def = NULL;
if (cookie->parent) {
@@ -981,8 +981,8 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
if (v == &fscache_cookies) {
seq_puts(m,
- "COOKIE PARENT USAGE CHILD ACT TY FL DEF NETFS_DATA\n"
- "======== ======== ===== ===== === == === ================ ==========\n"
+ "COOKIE PARENT USAGE CHILD ACT TY FL DEF \n"
+ "======== ======== ===== ===== === == === ================\n"
);
return 0;
}
@@ -1004,7 +1004,7 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
}
seq_printf(m,
- "%08x %08x %5u %5u %3u %s %03lx %-16s %px",
+ "%08x %08x %5u %5u %3u %s %03lx %-16s",
cookie->debug_id,
cookie->parent ? cookie->parent->debug_id : 0,
atomic_read(&cookie->usage),
@@ -1012,8 +1012,7 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
atomic_read(&cookie->n_active),
type,
cookie->flags,
- cookie->def->name,
- cookie->netfs_data);
+ cookie->def->name);
keylen = cookie->key_len;
auxlen = cookie->aux_len;
diff --git a/fs/fscache/fsdef.c b/fs/fscache/fsdef.c
index 5f8f6fe243fe..0e2e242121fe 100644
--- a/fs/fscache/fsdef.c
+++ b/fs/fscache/fsdef.c
@@ -9,12 +9,6 @@
#include <linux/module.h>
#include "internal.h"
-static
-enum fscache_checkaux fscache_fsdef_netfs_check_aux(void *cookie_netfs_data,
- const void *data,
- uint16_t datalen,
- loff_t object_size);
-
/*
* The root index is owned by FS-Cache itself.
*
@@ -64,35 +58,4 @@ EXPORT_SYMBOL(fscache_fsdef_index);
struct fscache_cookie_def fscache_fsdef_netfs_def = {
.name = "FSDEF.netfs",
.type = FSCACHE_COOKIE_TYPE_INDEX,
- .check_aux = fscache_fsdef_netfs_check_aux,
};
-
-/*
- * check that the index structure version number stored in the auxiliary data
- * matches the one the netfs gave us
- */
-static enum fscache_checkaux fscache_fsdef_netfs_check_aux(
- void *cookie_netfs_data,
- const void *data,
- uint16_t datalen,
- loff_t object_size)
-{
- struct fscache_netfs *netfs = cookie_netfs_data;
- uint32_t version;
-
- _enter("{%s},,%hu", netfs->name, datalen);
-
- if (datalen != sizeof(version)) {
- _leave(" = OBSOLETE [dl=%d v=%zu]", datalen, sizeof(version));
- return FSCACHE_CHECKAUX_OBSOLETE;
- }
-
- memcpy(&version, data, sizeof(version));
- if (version != netfs->version) {
- _leave(" = OBSOLETE [ver=%x net=%x]", version, netfs->version);
- return FSCACHE_CHECKAUX_OBSOLETE;
- }
-
- _leave(" = OKAY");
- return FSCACHE_CHECKAUX_OKAY;
-}
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 83dfbe0e3964..7dc447f7f3b0 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -24,6 +24,7 @@
#define pr_fmt(fmt) "FS-Cache: " fmt
+#include <linux/slab.h>
#include <linux/fscache-cache.h>
#include <trace/events/fscache.h>
#include <linux/sched.h>
@@ -41,6 +42,20 @@ extern struct rw_semaphore fscache_addremove_sem;
extern struct fscache_cache *fscache_select_cache_for_object(
struct fscache_cookie *);
+static inline
+struct fscache_cache_tag *fscache_get_cache_tag(struct fscache_cache_tag *tag)
+{
+ if (tag)
+ refcount_inc(&tag->ref);
+ return tag;
+}
+
+static inline void fscache_put_cache_tag(struct fscache_cache_tag *tag)
+{
+ if (tag && refcount_dec_and_test(&tag->ref))
+ kfree(tag);
+}
+
/*
* cookie.c
*/
@@ -50,9 +65,10 @@ extern const struct seq_operations fscache_cookies_seq_ops;
extern void fscache_free_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
const struct fscache_cookie_def *,
+ struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
- void *, loff_t);
+ loff_t);
extern struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *);
extern void fscache_cookie_put(struct fscache_cookie *,
enum fscache_cookie_trace);
@@ -329,16 +345,9 @@ static inline void fscache_cookie_get(struct fscache_cookie *cookie,
static inline
void fscache_update_aux(struct fscache_cookie *cookie, const void *aux_data)
{
- void *p;
-
- if (!aux_data)
- return;
- if (cookie->aux_len <= sizeof(cookie->inline_aux))
- p = cookie->inline_aux;
- else
- p = cookie->aux;
+ void *p = fscache_get_aux(cookie);
- if (memcmp(p, aux_data, cookie->aux_len) != 0) {
+ if (p && memcmp(p, aux_data, cookie->aux_len) != 0) {
memcpy(p, aux_data, cookie->aux_len);
set_bit(FSCACHE_COOKIE_AUX_UPDATED, &cookie->flags);
}
diff --git a/fs/fscache/netfs.c b/fs/fscache/netfs.c
index cce92216fa28..8ce4c7fa11a0 100644
--- a/fs/fscache/netfs.c
+++ b/fs/fscache/netfs.c
@@ -22,9 +22,10 @@ int __fscache_register_netfs(struct fscache_netfs *netfs)
/* allocate a cookie for the primary index */
candidate = fscache_alloc_cookie(&fscache_fsdef_index,
&fscache_fsdef_netfs_def,
+ NULL,
netfs->name, strlen(netfs->name),
&netfs->version, sizeof(netfs->version),
- netfs, 0);
+ 0);
if (!candidate) {
_leave(" = -ENOMEM");
return -ENOMEM;
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 1a0dc32c0a33..240e71d3faf2 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -170,7 +170,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
" EM EV FL S"
- " | COOKIE NETFS_COOKIE_DEF TY FL NETFS_DATA");
+ " | COOKIE NETFS_COOKIE_DEF TY FL");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ");
@@ -189,7 +189,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 2) {
seq_puts(m, "======== ======== ==== ===== === === === == ====="
" == == == ="
- " | ======== ================ == === ================");
+ " | ======== ================ == ===");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ================");
@@ -263,12 +263,11 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
break;
}
- seq_printf(m, "%08x %-16s %s %3lx %16p",
+ seq_printf(m, "%08x %-16s %s %3lx",
cookie->debug_id,
cookie->def->name,
type,
- cookie->flags,
- cookie->netfs_data);
+ cookie->flags);
if (config & FSCACHE_OBJLIST_CONFIG_KEY)
keylen = cookie->key_len;
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index 6e4ade2f1e6f..c95eeeaf7870 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -910,54 +910,6 @@ static void fscache_dequeue_object(struct fscache_object *object)
_leave("");
}
-/**
- * fscache_check_aux - Ask the netfs whether an object on disk is still valid
- * @object: The object to ask about
- * @data: The auxiliary data for the object
- * @datalen: The size of the auxiliary data
- *
- * This function consults the netfs about the coherency state of an object.
- * The caller must be holding a ref on cookie->n_active (held by
- * fscache_look_up_object() on behalf of the cache backend during object lookup
- * and creation).
- */
-enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
- const void *data, uint16_t datalen,
- loff_t object_size)
-{
- enum fscache_checkaux result;
-
- if (!object->cookie->def->check_aux) {
- fscache_stat(&fscache_n_checkaux_none);
- return FSCACHE_CHECKAUX_OKAY;
- }
-
- result = object->cookie->def->check_aux(object->cookie->netfs_data,
- data, datalen, object_size);
- switch (result) {
- /* entry okay as is */
- case FSCACHE_CHECKAUX_OKAY:
- fscache_stat(&fscache_n_checkaux_okay);
- break;
-
- /* entry requires update */
- case FSCACHE_CHECKAUX_NEEDS_UPDATE:
- fscache_stat(&fscache_n_checkaux_update);
- break;
-
- /* entry requires deletion */
- case FSCACHE_CHECKAUX_OBSOLETE:
- fscache_stat(&fscache_n_checkaux_obsolete);
- break;
-
- default:
- BUG();
- }
-
- return result;
-}
-EXPORT_SYMBOL(fscache_check_aux);
-
/*
* Asynchronously invalidate an object.
*/
diff --git a/fs/nfs/fscache-index.c b/fs/nfs/fscache-index.c
index 573b1da9342c..b1049815729e 100644
--- a/fs/nfs/fscache-index.c
+++ b/fs/nfs/fscache-index.c
@@ -106,7 +106,7 @@ enum fscache_checkaux nfs_fscache_inode_check_aux(void *cookie_netfs_data,
* cache fails with EIO - in which case the server must be contacted to
* retrieve the data, which requires the read context for security.
*/
-static void nfs_fh_get_context(void *cookie_netfs_data, void *context)
+static void nfs_fh_get_context(void *context)
{
get_nfs_open_context(context);
}
@@ -116,7 +116,7 @@ static void nfs_fh_get_context(void *cookie_netfs_data, void *context)
* - This function can be absent if the completion function doesn't require a
* context.
*/
-static void nfs_fh_put_context(void *cookie_netfs_data, void *context)
+static void nfs_fh_put_context(void *context)
{
if (context)
put_nfs_open_context(context);
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 2db4ffbf4523..0a87e82a1657 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -45,7 +45,8 @@ struct fscache_cache_tag {
struct fscache_cache *cache; /* cache referred to by this tag */
unsigned long flags;
#define FSCACHE_TAG_RESERVED 0 /* T if tag is reserved for a cache */
- atomic_t usage;
+ atomic_t usage; /* Number of using netfs's */
+ refcount_t ref; /* Reference count on structure */
char name[0]; /* tag name */
};
@@ -414,11 +415,6 @@ extern void fscache_io_error(struct fscache_cache *cache);
extern bool fscache_object_sleep_till_congested(signed long *timeoutp);
-extern enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
- const void *data,
- uint16_t datalen,
- loff_t object_size);
-
extern void fscache_object_retrying_stale(struct fscache_object *object);
enum fscache_why_object_killed {
@@ -430,4 +426,26 @@ enum fscache_why_object_killed {
extern void fscache_object_mark_killed(struct fscache_object *object,
enum fscache_why_object_killed why);
+/*
+ * Find the key on a cookie.
+ */
+static inline void *fscache_get_key(struct fscache_cookie *cookie)
+{
+ if (cookie->key_len <= sizeof(cookie->inline_key))
+ return cookie->inline_key;
+ else
+ return cookie->key;
+}
+
+/*
+ * Find the auxiliary data on a cookie.
+ */
+static inline void *fscache_get_aux(struct fscache_cookie *cookie)
+{
+ if (cookie->aux_len <= sizeof(cookie->inline_aux))
+ return cookie->inline_aux;
+ else
+ return cookie->aux;
+}
+
#endif /* _LINUX_FSCACHE_CACHE_H */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 029a2ddd9d93..7be340e1f20a 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -47,13 +47,6 @@ struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
-/* result of index entry consultation */
-enum fscache_checkaux {
- FSCACHE_CHECKAUX_OKAY, /* entry okay as is */
- FSCACHE_CHECKAUX_NEEDS_UPDATE, /* entry requires update */
- FSCACHE_CHECKAUX_OBSOLETE, /* entry requires deletion */
-};
-
/*
* fscache cookie definition
*/
@@ -65,26 +58,6 @@ struct fscache_cookie_def {
uint8_t type;
#define FSCACHE_COOKIE_TYPE_INDEX 0
#define FSCACHE_COOKIE_TYPE_DATAFILE 1
-
- /* select the cache into which to insert an entry in this index
- * - optional
- * - should return a cache identifier or NULL to cause the cache to be
- * inherited from the parent if possible or the first cache picked
- * for a non-index file if not
- */
- struct fscache_cache_tag *(*select_cache)(
- const void *parent_netfs_data,
- const void *cookie_netfs_data);
-
- /* consult the netfs about the state of an object
- * - this function can be absent if the index carries no state data
- * - the netfs data from the cookie being used as the target is
- * presented, as is the auxiliary data and the object size
- */
- enum fscache_checkaux (*check_aux)(void *cookie_netfs_data,
- const void *data,
- uint16_t datalen,
- loff_t object_size);
};
/*
@@ -114,9 +87,9 @@ struct fscache_cookie {
struct hlist_head backing_objects; /* object(s) backing this file/index */
const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
+ struct fscache_cache_tag *preferred_cache; /* The preferred cache or NULL */
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
- void *netfs_data; /* back pointer to netfs */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
@@ -164,9 +137,10 @@ extern void __fscache_release_cache_tag(struct fscache_cache_tag *);
extern struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *,
const struct fscache_cookie_def *,
+ struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
- void *, loff_t, bool);
+ loff_t, bool);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, const void *, bool);
extern int __fscache_check_consistency(struct fscache_cookie *, const void *);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
@@ -252,6 +226,7 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
* fscache_acquire_cookie - Acquire a cookie to represent a cache object
* @parent: The cookie that's to be the parent of this one
* @def: A description of the cache object, including callback operations
+ * @preferred_cache: The cache to use (or NULL)
* @index_key: The index key for this cookie
* @index_key_len: Size of the index key
* @aux_data: The auxiliary data for the cookie (may be NULL)
@@ -272,19 +247,19 @@ static inline
struct fscache_cookie *fscache_acquire_cookie(
struct fscache_cookie *parent,
const struct fscache_cookie_def *def,
+ struct fscache_cache_tag *preferred_cache,
const void *index_key,
size_t index_key_len,
const void *aux_data,
size_t aux_data_len,
- void *netfs_data,
loff_t object_size,
bool enable)
{
if (fscache_cookie_valid(parent) && fscache_cookie_enabled(parent))
- return __fscache_acquire_cookie(parent, def,
+ return __fscache_acquire_cookie(parent, def, preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
- netfs_data, object_size, enable);
+ object_size, enable);
else
return NULL;
}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 12/61] fscache: Remove struct fscache_cookie_def
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the cookie definition structure so that there's no pointers from
that back into the network filesystem. All of the method pointers that
were in there have been removed anyway. Any remaining information is
stashed in the cookie.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/cache.c | 15 ------------
fs/afs/cell.c | 4 ++-
fs/afs/inode.c | 15 +++++++-----
fs/afs/internal.h | 7 -----
fs/afs/volume.c | 5 ++--
fs/cachefiles/interface.c | 4 ++-
fs/cachefiles/xattr.c | 6 ++---
fs/fscache/cookie.c | 51 ++++++++++++++++++----------------------
fs/fscache/fsdef.c | 17 +------------
fs/fscache/internal.h | 5 ++--
fs/fscache/netfs.c | 5 ++--
fs/fscache/object-list.c | 16 +++----------
fs/fscache/object.c | 2 +-
fs/fscache/page.c | 2 +-
include/linux/fscache.h | 37 +++++++++++++++--------------
include/trace/events/fscache.h | 2 +-
16 files changed, 76 insertions(+), 117 deletions(-)
diff --git a/fs/afs/cache.c b/fs/afs/cache.c
index 9b2de3014dbf..0ee9ede6fc67 100644
--- a/fs/afs/cache.c
+++ b/fs/afs/cache.c
@@ -12,18 +12,3 @@ struct fscache_netfs afs_cache_netfs = {
.name = "afs",
.version = 2,
};
-
-struct fscache_cookie_def afs_cell_cache_index_def = {
- .name = "AFS.cell",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-struct fscache_cookie_def afs_volume_cache_index_def = {
- .name = "AFS.volume",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-struct fscache_cookie_def afs_vnode_cache_index_def = {
- .name = "AFS.vnode",
- .type = FSCACHE_COOKIE_TYPE_DATAFILE,
-};
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index c9fa573126b3..31877638f673 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -592,7 +592,9 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
#ifdef CONFIG_AFS_FSCACHE
cell->cache = fscache_acquire_cookie(afs_cache_netfs.primary_index,
- &afs_cell_cache_index_def,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.cell",
+ 0,
NULL,
cell->name, strlen(cell->name),
NULL, 0,
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 79e36491d8b4..cf843af5f65e 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -490,12 +490,15 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
key.vnode_id_ext[1] = vnode->fid.vnode_hi;
aux.data_version = vnode->status.data_version;
- vnode->cache = fscache_acquire_cookie(vnode->volume->cache,
- &afs_vnode_cache_index_def,
- NULL,
- &key, sizeof(key),
- &aux, sizeof(aux),
- vnode->status.size, true);
+ vnode->cache = fscache_acquire_cookie(
+ vnode->volume->cache,
+ FSCACHE_COOKIE_TYPE_DATAFILE,
+ "AFS.vnode",
+ vnode->status.type == AFS_FTYPE_FILE ? 0 : FSCACHE_ADV_SINGLE_CHUNK,
+ NULL,
+ &key, sizeof(key),
+ &aux, sizeof(aux),
+ vnode->status.size, true);
#endif
}
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 03487f1bef07..633c0f49bb55 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -821,13 +821,6 @@ extern void afs_merge_fs_addr6(struct afs_addr_list *, __be32 *, u16);
*/
#ifdef CONFIG_AFS_FSCACHE
extern struct fscache_netfs afs_cache_netfs;
-extern struct fscache_cookie_def afs_cell_cache_index_def;
-extern struct fscache_cookie_def afs_volume_cache_index_def;
-extern struct fscache_cookie_def afs_vnode_cache_index_def;
-#else
-#define afs_cell_cache_index_def (*(struct fscache_cookie_def *) NULL)
-#define afs_volume_cache_index_def (*(struct fscache_cookie_def *) NULL)
-#define afs_vnode_cache_index_def (*(struct fscache_cookie_def *) NULL)
#endif
/*
diff --git a/fs/afs/volume.c b/fs/afs/volume.c
index 77d909e7fc60..5fefb0ba85bf 100644
--- a/fs/afs/volume.c
+++ b/fs/afs/volume.c
@@ -183,8 +183,9 @@ void afs_activate_volume(struct afs_volume *volume)
{
#ifdef CONFIG_AFS_FSCACHE
volume->cache = fscache_acquire_cookie(volume->cell->cache,
- &afs_volume_cache_index_def,
- NULL,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.vol",
+ 0, NULL,
&volume->vid, sizeof(volume->vid),
NULL, 0, 0, true);
#endif
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 530ed11bb213..b7aa5c733cb7 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -41,7 +41,7 @@ static struct fscache_object *cachefiles_alloc_object(
fscache_object_init(&object->fscache, cookie, &cache->cache);
- object->type = cookie->def->type;
+ object->type = cookie->type;
/* get hold of the raw key
* - stick the length on the front and leave space on the back for the
@@ -109,7 +109,7 @@ static int cachefiles_lookup_object(struct fscache_object *_object)
/* polish off by setting the attributes of non-index files */
if (ret == 0 &&
- object->fscache.cookie->def->type != FSCACHE_COOKIE_TYPE_INDEX)
+ object->fscache.cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
cachefiles_attr_changed(&object->fscache);
if (ret < 0 && ret != -ETIMEDOUT) {
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 5ab8643ca97a..a4f1eddebe6f 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -39,7 +39,7 @@ int cachefiles_check_object_type(struct cachefiles_object *object)
if (!object->fscache.cookie)
strcpy(type, "C3");
else
- snprintf(type, 3, "%02x", object->fscache.cookie->def->type);
+ snprintf(type, 3, "%02x", object->fscache.cookie->type);
_enter("%p{%s}", object, type);
@@ -118,7 +118,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
if (!buf)
return -ENOMEM;
- buf->type = object->fscache.cookie->def->type;
+ buf->type = object->fscache.cookie->type;
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
@@ -157,7 +157,7 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
ret = vfs_getxattr(dentry, cachefiles_xattr_cache, buf, tlen);
if (ret == tlen &&
- buf->type == object->fscache.cookie->def->type &&
+ buf->type == object->fscache.cookie->type &&
memcmp(buf->data, p, len) == 0)
ret = 0;
else
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 58fa939d67ba..0ffccb238e69 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -43,10 +43,9 @@ static void fscache_print_cookie(struct fscache_cookie *cookie, char prefix)
cookie->flags,
atomic_read(&cookie->n_children),
atomic_read(&cookie->n_active));
- pr_err("%c-cookie d=%p{%s}\n",
+ pr_err("%c-cookie d=%s\n",
prefix,
- cookie->def,
- cookie->def ? cookie->def->name : "?");
+ cookie->type_name);
o = READ_ONCE(cookie->backing_objects.first);
if (o) {
@@ -149,7 +148,9 @@ static atomic_t fscache_cookie_debug_id = ATOMIC_INIT(1);
*/
struct fscache_cookie *fscache_alloc_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
@@ -162,8 +163,11 @@ struct fscache_cookie *fscache_alloc_cookie(
if (!cookie)
return NULL;
+ cookie->type = type;
+ cookie->advice = advice;
cookie->key_len = index_key_len;
cookie->aux_len = aux_data_len;
+ strlcpy(cookie->type_name, type_name, sizeof(cookie->type_name));
if (fscache_set_key(cookie, index_key, index_key_len) < 0)
goto nomem;
@@ -185,13 +189,10 @@ struct fscache_cookie *fscache_alloc_cookie(
*/
atomic_set(&cookie->n_active, 1);
- cookie->def = def;
cookie->parent = parent;
-
cookie->preferred_cache = fscache_get_cache_tag(preferred_cache);
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
- cookie->type = def->type;
spin_lock_init(&cookie->lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
@@ -253,7 +254,6 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
* - parent specifies the parent object
* - the top level index cookie for each netfs is stored in the fscache_netfs
* struct upon registration
- * - def points to the definition
* - all attached caches will be searched to see if they contain this object
* - index objects aren't stored on disk until there's a dependent file that
* needs storing
@@ -264,7 +264,9 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
*/
struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
@@ -273,11 +275,8 @@ struct fscache_cookie *__fscache_acquire_cookie(
{
struct fscache_cookie *candidate, *cookie;
- BUG_ON(!def);
-
_enter("{%s},{%s},%u",
- parent ? (char *) parent->def->name : "<no-parent>",
- def->name, enable);
+ parent ? parent->type_name : "<no-parent>", type_name, enable);
if (!index_key || !index_key_len || index_key_len > 255 || aux_data_len > 255)
return NULL;
@@ -296,12 +295,11 @@ struct fscache_cookie *__fscache_acquire_cookie(
}
/* validate the definition */
- BUG_ON(!def->name[0]);
-
- BUG_ON(def->type == FSCACHE_COOKIE_TYPE_INDEX &&
+ BUG_ON(type == FSCACHE_COOKIE_TYPE_INDEX &&
parent->type != FSCACHE_COOKIE_TYPE_INDEX);
- candidate = fscache_alloc_cookie(parent, def, preferred_cache,
+ candidate = fscache_alloc_cookie(parent, type, type_name, advice,
+ preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
object_size);
@@ -496,7 +494,7 @@ static int fscache_alloc_object(struct fscache_cache *cache,
struct fscache_object *object;
int ret;
- _enter("%p,%p{%s}", cache, cookie, cookie->def->name);
+ _enter("%p,%p{%s}", cache, cookie, cookie->type_name);
spin_lock(&cookie->lock);
hlist_for_each_entry(object, &cookie->backing_objects,
@@ -523,7 +521,7 @@ static int fscache_alloc_object(struct fscache_cache *cache,
object->debug_id = atomic_inc_return(&fscache_object_debug_id);
_debug("ALLOC OBJ%x: %s {%lx}",
- object->debug_id, cookie->def->name, object->events);
+ object->debug_id, cookie->type_name, object->events);
ret = fscache_alloc_object(cache, cookie->parent);
if (ret < 0)
@@ -571,7 +569,7 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
struct fscache_cache *cache = object->cache;
int ret;
- _enter("{%s},{OBJ%x}", cookie->def->name, object->debug_id);
+ _enter("{%s},{OBJ%x}", cookie->type_name, object->debug_id);
ASSERTCMP(object->cookie, ==, cookie);
@@ -633,7 +631,7 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
{
struct fscache_object *object;
- _enter("{%s}", cookie->def->name);
+ _enter("{%s}", cookie->type_name);
fscache_stat(&fscache_n_invalidates);
@@ -698,7 +696,7 @@ void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data
return;
}
- _enter("{%s}", cookie->def->name);
+ _enter("{%s}", cookie->type_name);
spin_lock(&cookie->lock);
@@ -737,7 +735,7 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
if (atomic_read(&cookie->n_children) != 0) {
pr_err("Cookie '%s' still has children\n",
- cookie->def->name);
+ cookie->type_name);
BUG();
}
@@ -816,7 +814,7 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
}
_enter("%p{%s,%d},%d",
- cookie, cookie->def->name,
+ cookie, cookie->type_name,
atomic_read(&cookie->n_active), retire);
trace_fscache_relinquish(cookie, retire);
@@ -827,9 +825,6 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
__fscache_disable_cookie(cookie, aux_data, retire);
- /* Clear pointers back to the netfs */
- cookie->def = NULL;
-
if (cookie->parent) {
ASSERTCMP(atomic_read(&cookie->parent->usage), >, 0);
ASSERTCMP(atomic_read(&cookie->parent->n_children), >, 0);
@@ -1012,7 +1007,7 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
atomic_read(&cookie->n_active),
type,
cookie->flags,
- cookie->def->name);
+ cookie->type_name);
keylen = cookie->key_len;
auxlen = cookie->aux_len;
diff --git a/fs/fscache/fsdef.c b/fs/fscache/fsdef.c
index 0e2e242121fe..b802cbddb578 100644
--- a/fs/fscache/fsdef.c
+++ b/fs/fscache/fsdef.c
@@ -33,29 +33,14 @@
* cache. It can create whatever objects it likes in that index, including
* further indices.
*/
-static struct fscache_cookie_def fscache_fsdef_index_def = {
- .name = ".FS-Cache",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
struct fscache_cookie fscache_fsdef_index = {
.debug_id = 1,
.usage = ATOMIC_INIT(1),
.n_active = ATOMIC_INIT(1),
.lock = __SPIN_LOCK_UNLOCKED(fscache_fsdef_index.lock),
.backing_objects = HLIST_HEAD_INIT,
- .def = &fscache_fsdef_index_def,
+ .type_name = ".fscach",
.flags = 1 << FSCACHE_COOKIE_ENABLED,
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
EXPORT_SYMBOL(fscache_fsdef_index);
-
-/*
- * Definition of an entry in the root index. Each entry is an index, keyed to
- * a specific netfs and only applicable to a particular version of the index
- * structure used by that netfs.
- */
-struct fscache_cookie_def fscache_fsdef_netfs_def = {
- .name = "FSDEF.netfs",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 7dc447f7f3b0..bc66bf7182ed 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -64,7 +64,9 @@ extern const struct seq_operations fscache_cookies_seq_ops;
extern void fscache_free_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
- const struct fscache_cookie_def *,
+ enum fscache_cookie_type,
+ const char *,
+ u8,
struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
@@ -77,7 +79,6 @@ extern void fscache_cookie_put(struct fscache_cookie *,
* fsdef.c
*/
extern struct fscache_cookie fscache_fsdef_index;
-extern struct fscache_cookie_def fscache_fsdef_netfs_def;
/*
* histogram.c
diff --git a/fs/fscache/netfs.c b/fs/fscache/netfs.c
index 8ce4c7fa11a0..f8a816f844f6 100644
--- a/fs/fscache/netfs.c
+++ b/fs/fscache/netfs.c
@@ -21,8 +21,9 @@ int __fscache_register_netfs(struct fscache_netfs *netfs)
/* allocate a cookie for the primary index */
candidate = fscache_alloc_cookie(&fscache_fsdef_index,
- &fscache_fsdef_netfs_def,
- NULL,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ ".netfs",
+ 0, NULL,
netfs->name, strlen(netfs->name),
&netfs->version, sizeof(netfs->version),
0);
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 240e71d3faf2..fc28de4738ec 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -21,8 +21,6 @@ struct fscache_objlist_data {
unsigned long config; /* display configuration */
#define FSCACHE_OBJLIST_CONFIG_KEY 0x00000001 /* show object keys */
#define FSCACHE_OBJLIST_CONFIG_AUX 0x00000002 /* show object auxdata */
-#define FSCACHE_OBJLIST_CONFIG_COOKIE 0x00000004 /* show objects with cookies */
-#define FSCACHE_OBJLIST_CONFIG_NOCOOKIE 0x00000008 /* show objects without cookies */
#define FSCACHE_OBJLIST_CONFIG_BUSY 0x00000010 /* show busy objects */
#define FSCACHE_OBJLIST_CONFIG_IDLE 0x00000020 /* show idle objects */
#define FSCACHE_OBJLIST_CONFIG_PENDWR 0x00000040 /* show objects with pending writes */
@@ -170,7 +168,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
" EM EV FL S"
- " | COOKIE NETFS_COOKIE_DEF TY FL");
+ " | COOKIE TYPE TY FL");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ");
@@ -189,7 +187,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 2) {
seq_puts(m, "======== ======== ==== ===== === === === == ====="
" == == == ="
- " | ======== ================ == ===");
+ " | ======== ======= == ===");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ================");
@@ -213,8 +211,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
cookie = obj->cookie;
if (~config) {
- FILTER(cookie->def,
- COOKIE, NOCOOKIE);
FILTER(fscache_object_is_active(obj) ||
obj->n_ops != 0 ||
obj->n_obj_ops != 0 ||
@@ -263,9 +259,9 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
break;
}
- seq_printf(m, "%08x %-16s %s %3lx",
+ seq_printf(m, "%08x %-7s %s %3lx",
cookie->debug_id,
- cookie->def->name,
+ cookie->type_name,
type,
cookie->flags);
@@ -338,8 +334,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
switch (buf[len]) {
case 'K': config |= FSCACHE_OBJLIST_CONFIG_KEY; break;
case 'A': config |= FSCACHE_OBJLIST_CONFIG_AUX; break;
- case 'C': config |= FSCACHE_OBJLIST_CONFIG_COOKIE; break;
- case 'c': config |= FSCACHE_OBJLIST_CONFIG_NOCOOKIE; break;
case 'B': config |= FSCACHE_OBJLIST_CONFIG_BUSY; break;
case 'b': config |= FSCACHE_OBJLIST_CONFIG_IDLE; break;
case 'W': config |= FSCACHE_OBJLIST_CONFIG_PENDWR; break;
@@ -354,8 +348,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
rcu_read_unlock();
key_put(key);
- if (!(config & (FSCACHE_OBJLIST_CONFIG_COOKIE | FSCACHE_OBJLIST_CONFIG_NOCOOKIE)))
- config |= FSCACHE_OBJLIST_CONFIG_COOKIE | FSCACHE_OBJLIST_CONFIG_NOCOOKIE;
if (!(config & (FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE)))
config |= FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE;
if (!(config & (FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR)))
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index c95eeeaf7870..fa74b3c94f88 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -474,7 +474,7 @@ static const struct fscache_state *fscache_look_up_object(struct fscache_object
}
_debug("LOOKUP \"%s\" in \"%s\"",
- cookie->def->name, object->cache->tag->name);
+ cookie->type_name, object->cache->tag->name);
fscache_stat(&fscache_n_object_lookups);
fscache_stat(&fscache_n_cop_lookup_object);
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index 1beffb071205..fd9cc16abc18 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -50,7 +50,7 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
_enter("%p", cookie);
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
+ ASSERTCMP(cookie->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
fscache_stat(&fscache_n_attr_changed);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 7be340e1f20a..85f9cb4ac826 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -42,24 +42,17 @@
/* pattern used to fill dead space in an index entry */
#define FSCACHE_INDEX_DEADFILL_PATTERN 0x79
-struct pagevec;
struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
-/*
- * fscache cookie definition
- */
-struct fscache_cookie_def {
- /* name of cookie type */
- char name[16];
-
- /* cookie type */
- uint8_t type;
-#define FSCACHE_COOKIE_TYPE_INDEX 0
-#define FSCACHE_COOKIE_TYPE_DATAFILE 1
+enum fscache_cookie_type {
+ FSCACHE_COOKIE_TYPE_INDEX,
+ FSCACHE_COOKIE_TYPE_DATAFILE,
};
+#define FSCACHE_ADV_SINGLE_CHUNK 0x01 /* The object is a single chunk of data */
+
/*
* fscache cached network filesystem type
* - name, version and ops must be filled in before registration
@@ -85,11 +78,11 @@ struct fscache_cookie {
unsigned int debug_id;
spinlock_t lock;
struct hlist_head backing_objects; /* object(s) backing this file/index */
- const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
struct fscache_cache_tag *preferred_cache; /* The preferred cache or NULL */
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
+ char type_name[8]; /* Cookie type name */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
@@ -103,7 +96,8 @@ struct fscache_cookie {
#define FSCACHE_COOKIE_ACQUIRED 9 /* T if cookie is in use */
#define FSCACHE_COOKIE_RELINQUISHING 10 /* T if cookie is being relinquished */
- u8 type; /* Type of object */
+ enum fscache_cookie_type type:8;
+ u8 advice; /* FSCACHE_COOKIE_ADV_* */
u8 key_len; /* Length of index key */
u8 aux_len; /* Length of auxiliary data */
u32 key_hash; /* Hash of parent, type, key, len */
@@ -136,7 +130,9 @@ extern void __fscache_release_cache_tag(struct fscache_cache_tag *);
extern struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *,
- const struct fscache_cookie_def *,
+ enum fscache_cookie_type,
+ const char *,
+ u8,
struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
@@ -225,7 +221,9 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
/**
* fscache_acquire_cookie - Acquire a cookie to represent a cache object
* @parent: The cookie that's to be the parent of this one
- * @def: A description of the cache object, including callback operations
+ * @type: Type of the cookie
+ * @type_name: Name of cookie type (max 7 chars)
+ * @advice: Advice flags (FSCACHE_COOKIE_ADV_*)
* @preferred_cache: The cache to use (or NULL)
* @index_key: The index key for this cookie
* @index_key_len: Size of the index key
@@ -246,7 +244,9 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
static inline
struct fscache_cookie *fscache_acquire_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key,
size_t index_key_len,
@@ -256,7 +256,8 @@ struct fscache_cookie *fscache_acquire_cookie(
bool enable)
{
if (fscache_cookie_valid(parent) && fscache_cookie_enabled(parent))
- return __fscache_acquire_cookie(parent, def, preferred_cache,
+ return __fscache_acquire_cookie(parent, type, type_name, advice,
+ preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
object_size, enable);
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 0b9e058aba4d..953e15fad063 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -233,7 +233,7 @@ TRACE_EVENT(fscache_acquire,
__entry->p_usage = atomic_read(&cookie->parent->usage);
__entry->p_n_children = atomic_read(&cookie->parent->n_children);
__entry->p_flags = cookie->parent->flags;
- memcpy(__entry->name, cookie->def->name, 8);
+ memcpy(__entry->name, cookie->type_name, 8);
__entry->name[7] = 0;
),
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 12/61] fscache: Remove struct fscache_cookie_def
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove the cookie definition structure so that there's no pointers from
that back into the network filesystem. All of the method pointers that
were in there have been removed anyway. Any remaining information is
stashed in the cookie.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/afs/cache.c | 15 ------------
fs/afs/cell.c | 4 ++-
fs/afs/inode.c | 15 +++++++-----
fs/afs/internal.h | 7 -----
fs/afs/volume.c | 5 ++--
fs/cachefiles/interface.c | 4 ++-
fs/cachefiles/xattr.c | 6 ++---
fs/fscache/cookie.c | 51 ++++++++++++++++++----------------------
fs/fscache/fsdef.c | 17 +------------
fs/fscache/internal.h | 5 ++--
fs/fscache/netfs.c | 5 ++--
fs/fscache/object-list.c | 16 +++----------
fs/fscache/object.c | 2 +-
fs/fscache/page.c | 2 +-
include/linux/fscache.h | 37 +++++++++++++++--------------
include/trace/events/fscache.h | 2 +-
16 files changed, 76 insertions(+), 117 deletions(-)
diff --git a/fs/afs/cache.c b/fs/afs/cache.c
index 9b2de3014dbf..0ee9ede6fc67 100644
--- a/fs/afs/cache.c
+++ b/fs/afs/cache.c
@@ -12,18 +12,3 @@ struct fscache_netfs afs_cache_netfs = {
.name = "afs",
.version = 2,
};
-
-struct fscache_cookie_def afs_cell_cache_index_def = {
- .name = "AFS.cell",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-struct fscache_cookie_def afs_volume_cache_index_def = {
- .name = "AFS.volume",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-struct fscache_cookie_def afs_vnode_cache_index_def = {
- .name = "AFS.vnode",
- .type = FSCACHE_COOKIE_TYPE_DATAFILE,
-};
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index c9fa573126b3..31877638f673 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -592,7 +592,9 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
#ifdef CONFIG_AFS_FSCACHE
cell->cache = fscache_acquire_cookie(afs_cache_netfs.primary_index,
- &afs_cell_cache_index_def,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.cell",
+ 0,
NULL,
cell->name, strlen(cell->name),
NULL, 0,
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 79e36491d8b4..cf843af5f65e 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -490,12 +490,15 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
key.vnode_id_ext[1] = vnode->fid.vnode_hi;
aux.data_version = vnode->status.data_version;
- vnode->cache = fscache_acquire_cookie(vnode->volume->cache,
- &afs_vnode_cache_index_def,
- NULL,
- &key, sizeof(key),
- &aux, sizeof(aux),
- vnode->status.size, true);
+ vnode->cache = fscache_acquire_cookie(
+ vnode->volume->cache,
+ FSCACHE_COOKIE_TYPE_DATAFILE,
+ "AFS.vnode",
+ vnode->status.type == AFS_FTYPE_FILE ? 0 : FSCACHE_ADV_SINGLE_CHUNK,
+ NULL,
+ &key, sizeof(key),
+ &aux, sizeof(aux),
+ vnode->status.size, true);
#endif
}
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 03487f1bef07..633c0f49bb55 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -821,13 +821,6 @@ extern void afs_merge_fs_addr6(struct afs_addr_list *, __be32 *, u16);
*/
#ifdef CONFIG_AFS_FSCACHE
extern struct fscache_netfs afs_cache_netfs;
-extern struct fscache_cookie_def afs_cell_cache_index_def;
-extern struct fscache_cookie_def afs_volume_cache_index_def;
-extern struct fscache_cookie_def afs_vnode_cache_index_def;
-#else
-#define afs_cell_cache_index_def (*(struct fscache_cookie_def *) NULL)
-#define afs_volume_cache_index_def (*(struct fscache_cookie_def *) NULL)
-#define afs_vnode_cache_index_def (*(struct fscache_cookie_def *) NULL)
#endif
/*
diff --git a/fs/afs/volume.c b/fs/afs/volume.c
index 77d909e7fc60..5fefb0ba85bf 100644
--- a/fs/afs/volume.c
+++ b/fs/afs/volume.c
@@ -183,8 +183,9 @@ void afs_activate_volume(struct afs_volume *volume)
{
#ifdef CONFIG_AFS_FSCACHE
volume->cache = fscache_acquire_cookie(volume->cell->cache,
- &afs_volume_cache_index_def,
- NULL,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.vol",
+ 0, NULL,
&volume->vid, sizeof(volume->vid),
NULL, 0, 0, true);
#endif
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 530ed11bb213..b7aa5c733cb7 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -41,7 +41,7 @@ static struct fscache_object *cachefiles_alloc_object(
fscache_object_init(&object->fscache, cookie, &cache->cache);
- object->type = cookie->def->type;
+ object->type = cookie->type;
/* get hold of the raw key
* - stick the length on the front and leave space on the back for the
@@ -109,7 +109,7 @@ static int cachefiles_lookup_object(struct fscache_object *_object)
/* polish off by setting the attributes of non-index files */
if (ret == 0 &&
- object->fscache.cookie->def->type != FSCACHE_COOKIE_TYPE_INDEX)
+ object->fscache.cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
cachefiles_attr_changed(&object->fscache);
if (ret < 0 && ret != -ETIMEDOUT) {
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 5ab8643ca97a..a4f1eddebe6f 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -39,7 +39,7 @@ int cachefiles_check_object_type(struct cachefiles_object *object)
if (!object->fscache.cookie)
strcpy(type, "C3");
else
- snprintf(type, 3, "%02x", object->fscache.cookie->def->type);
+ snprintf(type, 3, "%02x", object->fscache.cookie->type);
_enter("%p{%s}", object, type);
@@ -118,7 +118,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
if (!buf)
return -ENOMEM;
- buf->type = object->fscache.cookie->def->type;
+ buf->type = object->fscache.cookie->type;
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
@@ -157,7 +157,7 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
ret = vfs_getxattr(dentry, cachefiles_xattr_cache, buf, tlen);
if (ret == tlen &&
- buf->type == object->fscache.cookie->def->type &&
+ buf->type == object->fscache.cookie->type &&
memcmp(buf->data, p, len) == 0)
ret = 0;
else
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 58fa939d67ba..0ffccb238e69 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -43,10 +43,9 @@ static void fscache_print_cookie(struct fscache_cookie *cookie, char prefix)
cookie->flags,
atomic_read(&cookie->n_children),
atomic_read(&cookie->n_active));
- pr_err("%c-cookie d=%p{%s}\n",
+ pr_err("%c-cookie d=%s\n",
prefix,
- cookie->def,
- cookie->def ? cookie->def->name : "?");
+ cookie->type_name);
o = READ_ONCE(cookie->backing_objects.first);
if (o) {
@@ -149,7 +148,9 @@ static atomic_t fscache_cookie_debug_id = ATOMIC_INIT(1);
*/
struct fscache_cookie *fscache_alloc_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
@@ -162,8 +163,11 @@ struct fscache_cookie *fscache_alloc_cookie(
if (!cookie)
return NULL;
+ cookie->type = type;
+ cookie->advice = advice;
cookie->key_len = index_key_len;
cookie->aux_len = aux_data_len;
+ strlcpy(cookie->type_name, type_name, sizeof(cookie->type_name));
if (fscache_set_key(cookie, index_key, index_key_len) < 0)
goto nomem;
@@ -185,13 +189,10 @@ struct fscache_cookie *fscache_alloc_cookie(
*/
atomic_set(&cookie->n_active, 1);
- cookie->def = def;
cookie->parent = parent;
-
cookie->preferred_cache = fscache_get_cache_tag(preferred_cache);
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
- cookie->type = def->type;
spin_lock_init(&cookie->lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
@@ -253,7 +254,6 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
* - parent specifies the parent object
* - the top level index cookie for each netfs is stored in the fscache_netfs
* struct upon registration
- * - def points to the definition
* - all attached caches will be searched to see if they contain this object
* - index objects aren't stored on disk until there's a dependent file that
* needs storing
@@ -264,7 +264,9 @@ struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *candidate)
*/
struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
@@ -273,11 +275,8 @@ struct fscache_cookie *__fscache_acquire_cookie(
{
struct fscache_cookie *candidate, *cookie;
- BUG_ON(!def);
-
_enter("{%s},{%s},%u",
- parent ? (char *) parent->def->name : "<no-parent>",
- def->name, enable);
+ parent ? parent->type_name : "<no-parent>", type_name, enable);
if (!index_key || !index_key_len || index_key_len > 255 || aux_data_len > 255)
return NULL;
@@ -296,12 +295,11 @@ struct fscache_cookie *__fscache_acquire_cookie(
}
/* validate the definition */
- BUG_ON(!def->name[0]);
-
- BUG_ON(def->type == FSCACHE_COOKIE_TYPE_INDEX &&
+ BUG_ON(type == FSCACHE_COOKIE_TYPE_INDEX &&
parent->type != FSCACHE_COOKIE_TYPE_INDEX);
- candidate = fscache_alloc_cookie(parent, def, preferred_cache,
+ candidate = fscache_alloc_cookie(parent, type, type_name, advice,
+ preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
object_size);
@@ -496,7 +494,7 @@ static int fscache_alloc_object(struct fscache_cache *cache,
struct fscache_object *object;
int ret;
- _enter("%p,%p{%s}", cache, cookie, cookie->def->name);
+ _enter("%p,%p{%s}", cache, cookie, cookie->type_name);
spin_lock(&cookie->lock);
hlist_for_each_entry(object, &cookie->backing_objects,
@@ -523,7 +521,7 @@ static int fscache_alloc_object(struct fscache_cache *cache,
object->debug_id = atomic_inc_return(&fscache_object_debug_id);
_debug("ALLOC OBJ%x: %s {%lx}",
- object->debug_id, cookie->def->name, object->events);
+ object->debug_id, cookie->type_name, object->events);
ret = fscache_alloc_object(cache, cookie->parent);
if (ret < 0)
@@ -571,7 +569,7 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
struct fscache_cache *cache = object->cache;
int ret;
- _enter("{%s},{OBJ%x}", cookie->def->name, object->debug_id);
+ _enter("{%s},{OBJ%x}", cookie->type_name, object->debug_id);
ASSERTCMP(object->cookie, ==, cookie);
@@ -633,7 +631,7 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
{
struct fscache_object *object;
- _enter("{%s}", cookie->def->name);
+ _enter("{%s}", cookie->type_name);
fscache_stat(&fscache_n_invalidates);
@@ -698,7 +696,7 @@ void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data
return;
}
- _enter("{%s}", cookie->def->name);
+ _enter("{%s}", cookie->type_name);
spin_lock(&cookie->lock);
@@ -737,7 +735,7 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
if (atomic_read(&cookie->n_children) != 0) {
pr_err("Cookie '%s' still has children\n",
- cookie->def->name);
+ cookie->type_name);
BUG();
}
@@ -816,7 +814,7 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
}
_enter("%p{%s,%d},%d",
- cookie, cookie->def->name,
+ cookie, cookie->type_name,
atomic_read(&cookie->n_active), retire);
trace_fscache_relinquish(cookie, retire);
@@ -827,9 +825,6 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
__fscache_disable_cookie(cookie, aux_data, retire);
- /* Clear pointers back to the netfs */
- cookie->def = NULL;
-
if (cookie->parent) {
ASSERTCMP(atomic_read(&cookie->parent->usage), >, 0);
ASSERTCMP(atomic_read(&cookie->parent->n_children), >, 0);
@@ -1012,7 +1007,7 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
atomic_read(&cookie->n_active),
type,
cookie->flags,
- cookie->def->name);
+ cookie->type_name);
keylen = cookie->key_len;
auxlen = cookie->aux_len;
diff --git a/fs/fscache/fsdef.c b/fs/fscache/fsdef.c
index 0e2e242121fe..b802cbddb578 100644
--- a/fs/fscache/fsdef.c
+++ b/fs/fscache/fsdef.c
@@ -33,29 +33,14 @@
* cache. It can create whatever objects it likes in that index, including
* further indices.
*/
-static struct fscache_cookie_def fscache_fsdef_index_def = {
- .name = ".FS-Cache",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
-
struct fscache_cookie fscache_fsdef_index = {
.debug_id = 1,
.usage = ATOMIC_INIT(1),
.n_active = ATOMIC_INIT(1),
.lock = __SPIN_LOCK_UNLOCKED(fscache_fsdef_index.lock),
.backing_objects = HLIST_HEAD_INIT,
- .def = &fscache_fsdef_index_def,
+ .type_name = ".fscach",
.flags = 1 << FSCACHE_COOKIE_ENABLED,
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
EXPORT_SYMBOL(fscache_fsdef_index);
-
-/*
- * Definition of an entry in the root index. Each entry is an index, keyed to
- * a specific netfs and only applicable to a particular version of the index
- * structure used by that netfs.
- */
-struct fscache_cookie_def fscache_fsdef_netfs_def = {
- .name = "FSDEF.netfs",
- .type = FSCACHE_COOKIE_TYPE_INDEX,
-};
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 7dc447f7f3b0..bc66bf7182ed 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -64,7 +64,9 @@ extern const struct seq_operations fscache_cookies_seq_ops;
extern void fscache_free_cookie(struct fscache_cookie *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
- const struct fscache_cookie_def *,
+ enum fscache_cookie_type,
+ const char *,
+ u8,
struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
@@ -77,7 +79,6 @@ extern void fscache_cookie_put(struct fscache_cookie *,
* fsdef.c
*/
extern struct fscache_cookie fscache_fsdef_index;
-extern struct fscache_cookie_def fscache_fsdef_netfs_def;
/*
* histogram.c
diff --git a/fs/fscache/netfs.c b/fs/fscache/netfs.c
index 8ce4c7fa11a0..f8a816f844f6 100644
--- a/fs/fscache/netfs.c
+++ b/fs/fscache/netfs.c
@@ -21,8 +21,9 @@ int __fscache_register_netfs(struct fscache_netfs *netfs)
/* allocate a cookie for the primary index */
candidate = fscache_alloc_cookie(&fscache_fsdef_index,
- &fscache_fsdef_netfs_def,
- NULL,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ ".netfs",
+ 0, NULL,
netfs->name, strlen(netfs->name),
&netfs->version, sizeof(netfs->version),
0);
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 240e71d3faf2..fc28de4738ec 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -21,8 +21,6 @@ struct fscache_objlist_data {
unsigned long config; /* display configuration */
#define FSCACHE_OBJLIST_CONFIG_KEY 0x00000001 /* show object keys */
#define FSCACHE_OBJLIST_CONFIG_AUX 0x00000002 /* show object auxdata */
-#define FSCACHE_OBJLIST_CONFIG_COOKIE 0x00000004 /* show objects with cookies */
-#define FSCACHE_OBJLIST_CONFIG_NOCOOKIE 0x00000008 /* show objects without cookies */
#define FSCACHE_OBJLIST_CONFIG_BUSY 0x00000010 /* show busy objects */
#define FSCACHE_OBJLIST_CONFIG_IDLE 0x00000020 /* show idle objects */
#define FSCACHE_OBJLIST_CONFIG_PENDWR 0x00000040 /* show objects with pending writes */
@@ -170,7 +168,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
" EM EV FL S"
- " | COOKIE NETFS_COOKIE_DEF TY FL");
+ " | COOKIE TYPE TY FL");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ");
@@ -189,7 +187,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
if ((unsigned long) v == 2) {
seq_puts(m, "======== ======== ==== ===== === === === == ====="
" == == == ="
- " | ======== ================ == ===");
+ " | ======== ======= == ===");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
FSCACHE_OBJLIST_CONFIG_AUX))
seq_puts(m, " ================");
@@ -213,8 +211,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
cookie = obj->cookie;
if (~config) {
- FILTER(cookie->def,
- COOKIE, NOCOOKIE);
FILTER(fscache_object_is_active(obj) ||
obj->n_ops != 0 ||
obj->n_obj_ops != 0 ||
@@ -263,9 +259,9 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
break;
}
- seq_printf(m, "%08x %-16s %s %3lx",
+ seq_printf(m, "%08x %-7s %s %3lx",
cookie->debug_id,
- cookie->def->name,
+ cookie->type_name,
type,
cookie->flags);
@@ -338,8 +334,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
switch (buf[len]) {
case 'K': config |= FSCACHE_OBJLIST_CONFIG_KEY; break;
case 'A': config |= FSCACHE_OBJLIST_CONFIG_AUX; break;
- case 'C': config |= FSCACHE_OBJLIST_CONFIG_COOKIE; break;
- case 'c': config |= FSCACHE_OBJLIST_CONFIG_NOCOOKIE; break;
case 'B': config |= FSCACHE_OBJLIST_CONFIG_BUSY; break;
case 'b': config |= FSCACHE_OBJLIST_CONFIG_IDLE; break;
case 'W': config |= FSCACHE_OBJLIST_CONFIG_PENDWR; break;
@@ -354,8 +348,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
rcu_read_unlock();
key_put(key);
- if (!(config & (FSCACHE_OBJLIST_CONFIG_COOKIE | FSCACHE_OBJLIST_CONFIG_NOCOOKIE)))
- config |= FSCACHE_OBJLIST_CONFIG_COOKIE | FSCACHE_OBJLIST_CONFIG_NOCOOKIE;
if (!(config & (FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE)))
config |= FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE;
if (!(config & (FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR)))
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index c95eeeaf7870..fa74b3c94f88 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -474,7 +474,7 @@ static const struct fscache_state *fscache_look_up_object(struct fscache_object
}
_debug("LOOKUP \"%s\" in \"%s\"",
- cookie->def->name, object->cache->tag->name);
+ cookie->type_name, object->cache->tag->name);
fscache_stat(&fscache_n_object_lookups);
fscache_stat(&fscache_n_cop_lookup_object);
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index 1beffb071205..fd9cc16abc18 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -50,7 +50,7 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
_enter("%p", cookie);
- ASSERTCMP(cookie->def->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
+ ASSERTCMP(cookie->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
fscache_stat(&fscache_n_attr_changed);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 7be340e1f20a..85f9cb4ac826 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -42,24 +42,17 @@
/* pattern used to fill dead space in an index entry */
#define FSCACHE_INDEX_DEADFILL_PATTERN 0x79
-struct pagevec;
struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
-/*
- * fscache cookie definition
- */
-struct fscache_cookie_def {
- /* name of cookie type */
- char name[16];
-
- /* cookie type */
- uint8_t type;
-#define FSCACHE_COOKIE_TYPE_INDEX 0
-#define FSCACHE_COOKIE_TYPE_DATAFILE 1
+enum fscache_cookie_type {
+ FSCACHE_COOKIE_TYPE_INDEX,
+ FSCACHE_COOKIE_TYPE_DATAFILE,
};
+#define FSCACHE_ADV_SINGLE_CHUNK 0x01 /* The object is a single chunk of data */
+
/*
* fscache cached network filesystem type
* - name, version and ops must be filled in before registration
@@ -85,11 +78,11 @@ struct fscache_cookie {
unsigned int debug_id;
spinlock_t lock;
struct hlist_head backing_objects; /* object(s) backing this file/index */
- const struct fscache_cookie_def *def; /* definition */
struct fscache_cookie *parent; /* parent of this entry */
struct fscache_cache_tag *preferred_cache; /* The preferred cache or NULL */
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
+ char type_name[8]; /* Cookie type name */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
@@ -103,7 +96,8 @@ struct fscache_cookie {
#define FSCACHE_COOKIE_ACQUIRED 9 /* T if cookie is in use */
#define FSCACHE_COOKIE_RELINQUISHING 10 /* T if cookie is being relinquished */
- u8 type; /* Type of object */
+ enum fscache_cookie_type type:8;
+ u8 advice; /* FSCACHE_COOKIE_ADV_* */
u8 key_len; /* Length of index key */
u8 aux_len; /* Length of auxiliary data */
u32 key_hash; /* Hash of parent, type, key, len */
@@ -136,7 +130,9 @@ extern void __fscache_release_cache_tag(struct fscache_cache_tag *);
extern struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cookie *,
- const struct fscache_cookie_def *,
+ enum fscache_cookie_type,
+ const char *,
+ u8,
struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
@@ -225,7 +221,9 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
/**
* fscache_acquire_cookie - Acquire a cookie to represent a cache object
* @parent: The cookie that's to be the parent of this one
- * @def: A description of the cache object, including callback operations
+ * @type: Type of the cookie
+ * @type_name: Name of cookie type (max 7 chars)
+ * @advice: Advice flags (FSCACHE_COOKIE_ADV_*)
* @preferred_cache: The cache to use (or NULL)
* @index_key: The index key for this cookie
* @index_key_len: Size of the index key
@@ -246,7 +244,9 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
static inline
struct fscache_cookie *fscache_acquire_cookie(
struct fscache_cookie *parent,
- const struct fscache_cookie_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
struct fscache_cache_tag *preferred_cache,
const void *index_key,
size_t index_key_len,
@@ -256,7 +256,8 @@ struct fscache_cookie *fscache_acquire_cookie(
bool enable)
{
if (fscache_cookie_valid(parent) && fscache_cookie_enabled(parent))
- return __fscache_acquire_cookie(parent, def, preferred_cache,
+ return __fscache_acquire_cookie(parent, type, type_name, advice,
+ preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
object_size, enable);
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 0b9e058aba4d..953e15fad063 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -233,7 +233,7 @@ TRACE_EVENT(fscache_acquire,
__entry->p_usage = atomic_read(&cookie->parent->usage);
__entry->p_n_children = atomic_read(&cookie->parent->n_children);
__entry->p_flags = cookie->parent->flags;
- memcpy(__entry->name, cookie->def->name, 8);
+ memcpy(__entry->name, cookie->type_name, 8);
__entry->name[7] = 0;
),
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 13/61] fscache: Remove store_limit* from struct fscache_object
2020-05-04 17:07 ` David Howells
` (12 preceding siblings ...)
(?)
@ 2020-05-04 17:09 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the store_limit values from struct fscache_object and store the
object size in the cookie. The netfs can update this at will, and we don't
want to call back into the netfs to fetch it.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 10 ++--------
fs/fscache/cookie.c | 14 ++++++--------
fs/fscache/object.c | 2 --
include/linux/fscache-cache.h | 22 ----------------------
include/linux/fscache.h | 1 +
5 files changed, 9 insertions(+), 40 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index b7aa5c733cb7..a5d348581bcc 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -359,7 +359,7 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
loff_t oi_size;
int ret;
- ni_size = _object->store_limit_l;
+ ni_size = _object->cookie->object_size;
_enter("{OBJ%x},[%llu]",
_object->debug_id, (unsigned long long) ni_size);
@@ -376,8 +376,6 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
ASSERT(d_is_reg(object->backer));
- fscache_set_store_limit(&object->fscache, ni_size);
-
oi_size = i_size_read(d_backing_inode(object->backer));
if (oi_size == ni_size)
return 0;
@@ -406,7 +404,6 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
cachefiles_end_secure(cache, saved_cred);
if (ret == -EIO) {
- fscache_set_store_limit(&object->fscache, 0);
cachefiles_io_error_obj(object, "Size set failed");
ret = -ENOBUFS;
}
@@ -431,7 +428,7 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
- ni_size = op->object->store_limit_l;
+ ni_size = op->object->cookie->object_size;
_enter("{OBJ%x},[%llu]",
op->object->debug_id, (unsigned long long)ni_size);
@@ -439,8 +436,6 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
if (object->backer) {
ASSERT(d_is_reg(object->backer));
- fscache_set_store_limit(&object->fscache, ni_size);
-
path.dentry = object->backer;
path.mnt = cache->mnt;
@@ -451,7 +446,6 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
cachefiles_end_secure(cache, saved_cred);
if (ret != 0) {
- fscache_set_store_limit(&object->fscache, 0);
if (ret == -EIO)
cachefiles_io_error_obj(object,
"Invalidate failed");
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 0ffccb238e69..bcaadbcaa0b2 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -22,8 +22,7 @@ static struct hlist_bl_head fscache_cookie_hash[1 << fscache_cookie_hash_shift];
static LIST_HEAD(fscache_cookies);
static DEFINE_RWLOCK(fscache_cookies_lock);
-static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie,
- loff_t object_size);
+static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie);
static int fscache_alloc_object(struct fscache_cache *cache,
struct fscache_cookie *cookie);
static int fscache_attach_object(struct fscache_cookie *cookie,
@@ -167,6 +166,7 @@ struct fscache_cookie *fscache_alloc_cookie(
cookie->advice = advice;
cookie->key_len = index_key_len;
cookie->aux_len = aux_data_len;
+ cookie->object_size = object_size;
strlcpy(cookie->type_name, type_name, sizeof(cookie->type_name));
if (fscache_set_key(cookie, index_key, index_key_len) < 0)
@@ -337,7 +337,7 @@ struct fscache_cookie *__fscache_acquire_cookie(
* - we create indices on disk when we need them as an index
* may exist in multiple caches */
if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX) {
- if (fscache_acquire_non_index_cookie(cookie, object_size) == 0) {
+ if (fscache_acquire_non_index_cookie(cookie) == 0) {
set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
} else {
atomic_dec(&parent->n_children);
@@ -376,6 +376,7 @@ void __fscache_enable_cookie(struct fscache_cookie *cookie,
wait_on_bit_lock(&cookie->flags, FSCACHE_COOKIE_ENABLEMENT_LOCK,
TASK_UNINTERRUPTIBLE);
+ cookie->object_size = object_size;
fscache_update_aux(cookie, aux_data);
if (test_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags))
@@ -387,7 +388,7 @@ void __fscache_enable_cookie(struct fscache_cookie *cookie,
/* Wait for outstanding disablement to complete */
__fscache_wait_on_invalidate(cookie);
- if (fscache_acquire_non_index_cookie(cookie, object_size) == 0)
+ if (fscache_acquire_non_index_cookie(cookie) == 0)
set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
} else {
set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
@@ -404,8 +405,7 @@ EXPORT_SYMBOL(__fscache_enable_cookie);
* - this must make sure the index chain is instantiated and instantiate the
* object representation too
*/
-static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie,
- loff_t object_size)
+static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie)
{
struct fscache_object *object;
struct fscache_cache *cache;
@@ -456,8 +456,6 @@ static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie,
object = hlist_entry(cookie->backing_objects.first,
struct fscache_object, cookie_link);
- fscache_set_store_limit(object, object_size);
-
/* initiate the process of looking up all the objects in the chain
* (done by fscache_initialise_object()) */
fscache_raise_event(object, FSCACHE_OBJECT_EV_NEW_CHILD);
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index fa74b3c94f88..ede38bd4774a 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -319,8 +319,6 @@ void fscache_object_init(struct fscache_object *object,
object->n_children = 0;
object->n_ops = object->n_in_progress = object->n_exclusive = 0;
object->events = 0;
- object->store_limit = 0;
- object->store_limit_l = 0;
object->cache = cache;
object->cookie = cookie;
fscache_cookie_get(cookie, fscache_cookie_get_attach_object);
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 0a87e82a1657..81418056f43f 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -274,8 +274,6 @@ struct fscache_object {
#ifdef CONFIG_FSCACHE_OBJECT_LIST
struct rb_node objlist_link; /* link in global object list */
#endif
- pgoff_t store_limit; /* current storage limit */
- loff_t store_limit_l; /* current storage limit */
};
extern void fscache_object_init(struct fscache_object *, struct fscache_cookie *,
@@ -336,26 +334,6 @@ static inline void fscache_object_lookup_error(struct fscache_object *object)
set_bit(FSCACHE_OBJECT_EV_ERROR, &object->events);
}
-/**
- * fscache_set_store_limit - Set the maximum size to be stored in an object
- * @object: The object to set the maximum on
- * @i_size: The limit to set in bytes
- *
- * Set the maximum size an object is permitted to reach, implying the highest
- * byte that may be written. Intended to be called by the attr_changed() op.
- *
- * See Documentation/filesystems/caching/backend-api.txt for a complete
- * description.
- */
-static inline
-void fscache_set_store_limit(struct fscache_object *object, loff_t i_size)
-{
- object->store_limit_l = i_size;
- object->store_limit = i_size >> PAGE_SHIFT;
- if (i_size & ~PAGE_MASK)
- object->store_limit++;
-}
-
static inline void __fscache_use_cookie(struct fscache_cookie *cookie)
{
atomic_inc(&cookie->n_active);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 85f9cb4ac826..64d9ef34da49 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -83,6 +83,7 @@ struct fscache_cookie {
struct hlist_bl_node hash_link; /* Link in hash table */
struct list_head proc_link; /* Link in proc list */
char type_name[8]; /* Cookie type name */
+ loff_t object_size; /* Size of the netfs object */
unsigned long flags;
#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 14/61] fscache: Remove fscache_check_consistency()
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove fscache_check_consistency() as that allows the netfs to pry into the
inner working of the cache - and what's in the cookie should be taken as
consistent with the disk (possibly lazily).
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 26 -------------
fs/fscache/cookie.c | 79 ----------------------------------------
fs/fscache/internal.h | 9 -----
fs/fscache/page.c | 82 -----------------------------------------
include/linux/fscache-cache.h | 4 --
include/linux/fscache.h | 23 ------------
6 files changed, 223 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index a5d348581bcc..a3837ed090a8 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -320,31 +320,6 @@ static void cachefiles_sync_cache(struct fscache_cache *_cache)
ret);
}
-/*
- * check if the backing cache is updated to FS-Cache
- * - called by FS-Cache when evaluates if need to invalidate the cache
- */
-static int cachefiles_check_consistency(struct fscache_operation *op)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- const struct cred *saved_cred;
- int ret;
-
- _enter("{OBJ%x}", op->object->debug_id);
-
- object = container_of(op->object, struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- cachefiles_begin_secure(cache, &saved_cred);
- ret = cachefiles_check_auxdata(object);
- cachefiles_end_secure(cache, saved_cred);
-
- _leave(" = %d", ret);
- return ret;
-}
-
/*
* notification the attributes on an object have changed
* - called with reads/writes excluded by FS-Cache
@@ -468,5 +443,4 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
.attr_changed = cachefiles_attr_changed,
- .check_consistency = cachefiles_check_consistency,
};
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index bcaadbcaa0b2..cd2e166d5f29 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -883,85 +883,6 @@ void fscache_cookie_put(struct fscache_cookie *cookie,
_leave("");
}
-/*
- * check the consistency between the netfs inode and the backing cache
- *
- * NOTE: it only serves no-index type
- */
-int __fscache_check_consistency(struct fscache_cookie *cookie,
- const void *aux_data)
-{
- struct fscache_operation *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,", cookie);
-
- ASSERTCMP(cookie->type, ==, FSCACHE_COOKIE_TYPE_DATAFILE);
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- if (hlist_empty(&cookie->backing_objects))
- return 0;
-
- op = kzalloc(sizeof(*op), GFP_NOIO | __GFP_NOMEMALLOC | __GFP_NORETRY);
- if (!op)
- return -ENOMEM;
-
- fscache_operation_init(cookie, op, NULL, NULL, NULL);
- op->flags = FSCACHE_OP_MYTHREAD |
- (1 << FSCACHE_OP_WAITING) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
- trace_fscache_page_op(cookie, NULL, op, fscache_page_op_check_consistency);
-
- spin_lock(&cookie->lock);
-
- fscache_update_aux(cookie, aux_data);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto inconsistent;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
- if (test_bit(FSCACHE_IOERROR, &object->cache->flags))
- goto inconsistent;
-
- op->debug_id = atomic_inc_return(&fscache_op_debug_id);
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, op) < 0)
- goto submit_failed;
-
- /* the work queue now carries its own ref on the object */
- spin_unlock(&cookie->lock);
-
- ret = fscache_wait_for_operation_activation(object, op, NULL, NULL);
- if (ret == 0) {
- /* ask the cache to honour the operation */
- ret = object->cache->ops->check_consistency(op);
- fscache_op_complete(op, false);
- } else if (ret == -ENOBUFS) {
- ret = 0;
- }
-
- fscache_put_operation(op);
- _leave(" = %d", ret);
- return ret;
-
-submit_failed:
- wake_cookie = __fscache_unuse_cookie(cookie);
-inconsistent:
- spin_unlock(&cookie->lock);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- kfree(op);
- _leave(" = -ESTALE");
- return -ESTALE;
-}
-EXPORT_SYMBOL(__fscache_check_consistency);
-
/*
* Generate a list of extant cookies in /proc/fs/fscache/cookies
*/
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index bc66bf7182ed..20cbd1288b5a 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -151,15 +151,6 @@ extern void fscache_abort_object(struct fscache_object *);
extern void fscache_start_operations(struct fscache_object *);
extern void fscache_operation_gc(struct work_struct *);
-/*
- * page.c
- */
-extern int fscache_wait_for_deferred_lookup(struct fscache_cookie *);
-extern int fscache_wait_for_operation_activation(struct fscache_object *,
- struct fscache_operation *,
- atomic_t *,
- atomic_t *);
-
/*
* proc.c
*/
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index fd9cc16abc18..73636e9d652d 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -96,85 +96,3 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
return -ENOBUFS;
}
EXPORT_SYMBOL(__fscache_attr_changed);
-
-/*
- * wait for a deferred lookup to complete
- */
-int fscache_wait_for_deferred_lookup(struct fscache_cookie *cookie)
-{
- unsigned long jif;
-
- _enter("");
-
- if (!test_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags)) {
- _leave(" = 0 [imm]");
- return 0;
- }
-
- fscache_stat(&fscache_n_retrievals_wait);
-
- jif = jiffies;
- if (wait_on_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP,
- TASK_INTERRUPTIBLE) != 0) {
- fscache_stat(&fscache_n_retrievals_intr);
- _leave(" = -ERESTARTSYS");
- return -ERESTARTSYS;
- }
-
- ASSERT(!test_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags));
-
- smp_rmb();
- fscache_hist(fscache_retrieval_delay_histogram, jif);
- _leave(" = 0 [dly]");
- return 0;
-}
-
-/*
- * wait for an object to become active (or dead)
- */
-int fscache_wait_for_operation_activation(struct fscache_object *object,
- struct fscache_operation *op,
- atomic_t *stat_op_waits,
- atomic_t *stat_object_dead)
-{
- int ret;
-
- if (!test_bit(FSCACHE_OP_WAITING, &op->flags))
- goto check_if_dead;
-
- _debug(">>> WT");
- if (stat_op_waits)
- fscache_stat(stat_op_waits);
- if (wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
- TASK_INTERRUPTIBLE) != 0) {
- trace_fscache_op(object->cookie, op, fscache_op_signal);
- ret = fscache_cancel_op(op, false);
- if (ret == 0)
- return -ERESTARTSYS;
-
- /* it's been removed from the pending queue by another party,
- * so we should get to run shortly */
- wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
- TASK_UNINTERRUPTIBLE);
- }
- _debug("<<< GO");
-
-check_if_dead:
- if (op->state == FSCACHE_OP_ST_CANCELLED) {
- if (stat_object_dead)
- fscache_stat(stat_object_dead);
- _leave(" = -ENOBUFS [cancelled]");
- return -ENOBUFS;
- }
- if (unlikely(fscache_object_is_dying(object) ||
- fscache_cache_is_broken(object))) {
- enum fscache_operation_state state = op->state;
- trace_fscache_op(object->cookie, op, fscache_op_signal);
- fscache_cancel_op(op, true);
- if (stat_object_dead)
- fscache_stat(stat_object_dead);
- _leave(" = -ENOBUFS [obj dead %d]", state);
- return -ENOBUFS;
- }
- return 0;
-}
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 81418056f43f..3a78e41d2338 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -167,10 +167,6 @@ struct fscache_cache_ops {
/* unpin an object in the cache */
void (*unpin_object)(struct fscache_object *object);
- /* check the consistency between the backing cache and the FS-Cache
- * cookie */
- int (*check_consistency)(struct fscache_operation *op);
-
/* store the updated auxiliary data on an object */
void (*update_object)(struct fscache_object *object);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 64d9ef34da49..82e871a3dc6a 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -139,7 +139,6 @@ extern struct fscache_cookie *__fscache_acquire_cookie(
const void *, size_t,
loff_t, bool);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, const void *, bool);
-extern int __fscache_check_consistency(struct fscache_cookie *, const void *);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
extern int __fscache_attr_changed(struct fscache_cookie *);
extern void __fscache_invalidate(struct fscache_cookie *);
@@ -290,28 +289,6 @@ void fscache_relinquish_cookie(struct fscache_cookie *cookie,
__fscache_relinquish_cookie(cookie, aux_data, retire);
}
-/**
- * fscache_check_consistency - Request validation of a cache's auxiliary data
- * @cookie: The cookie representing the cache object
- * @aux_data: The updated auxiliary data for the cookie (may be NULL)
- *
- * Request an consistency check from fscache, which passes the request to the
- * backing cache. The auxiliary data on the cookie will be updated first if
- * @aux_data is set.
- *
- * Returns 0 if consistent and -ESTALE if inconsistent. May also
- * return -ENOMEM and -ERESTARTSYS.
- */
-static inline
-int fscache_check_consistency(struct fscache_cookie *cookie,
- const void *aux_data)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_check_consistency(cookie, aux_data);
- else
- return 0;
-}
-
/**
* fscache_update_cookie - Request that a cache object be updated
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 14/61] fscache: Remove fscache_check_consistency()
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove fscache_check_consistency() as that allows the netfs to pry into the
inner working of the cache - and what's in the cookie should be taken as
consistent with the disk (possibly lazily).
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/interface.c | 26 -------------
fs/fscache/cookie.c | 79 ----------------------------------------
fs/fscache/internal.h | 9 -----
fs/fscache/page.c | 82 -----------------------------------------
include/linux/fscache-cache.h | 4 --
include/linux/fscache.h | 23 ------------
6 files changed, 223 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index a5d348581bcc..a3837ed090a8 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -320,31 +320,6 @@ static void cachefiles_sync_cache(struct fscache_cache *_cache)
ret);
}
-/*
- * check if the backing cache is updated to FS-Cache
- * - called by FS-Cache when evaluates if need to invalidate the cache
- */
-static int cachefiles_check_consistency(struct fscache_operation *op)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- const struct cred *saved_cred;
- int ret;
-
- _enter("{OBJ%x}", op->object->debug_id);
-
- object = container_of(op->object, struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache,
- struct cachefiles_cache, cache);
-
- cachefiles_begin_secure(cache, &saved_cred);
- ret = cachefiles_check_auxdata(object);
- cachefiles_end_secure(cache, saved_cred);
-
- _leave(" = %d", ret);
- return ret;
-}
-
/*
* notification the attributes on an object have changed
* - called with reads/writes excluded by FS-Cache
@@ -468,5 +443,4 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
.attr_changed = cachefiles_attr_changed,
- .check_consistency = cachefiles_check_consistency,
};
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index bcaadbcaa0b2..cd2e166d5f29 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -883,85 +883,6 @@ void fscache_cookie_put(struct fscache_cookie *cookie,
_leave("");
}
-/*
- * check the consistency between the netfs inode and the backing cache
- *
- * NOTE: it only serves no-index type
- */
-int __fscache_check_consistency(struct fscache_cookie *cookie,
- const void *aux_data)
-{
- struct fscache_operation *op;
- struct fscache_object *object;
- bool wake_cookie = false;
- int ret;
-
- _enter("%p,", cookie);
-
- ASSERTCMP(cookie->type, ==, FSCACHE_COOKIE_TYPE_DATAFILE);
-
- if (fscache_wait_for_deferred_lookup(cookie) < 0)
- return -ERESTARTSYS;
-
- if (hlist_empty(&cookie->backing_objects))
- return 0;
-
- op = kzalloc(sizeof(*op), GFP_NOIO | __GFP_NOMEMALLOC | __GFP_NORETRY);
- if (!op)
- return -ENOMEM;
-
- fscache_operation_init(cookie, op, NULL, NULL, NULL);
- op->flags = FSCACHE_OP_MYTHREAD |
- (1 << FSCACHE_OP_WAITING) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
- trace_fscache_page_op(cookie, NULL, op, fscache_page_op_check_consistency);
-
- spin_lock(&cookie->lock);
-
- fscache_update_aux(cookie, aux_data);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto inconsistent;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
- if (test_bit(FSCACHE_IOERROR, &object->cache->flags))
- goto inconsistent;
-
- op->debug_id = atomic_inc_return(&fscache_op_debug_id);
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_op(object, op) < 0)
- goto submit_failed;
-
- /* the work queue now carries its own ref on the object */
- spin_unlock(&cookie->lock);
-
- ret = fscache_wait_for_operation_activation(object, op, NULL, NULL);
- if (ret == 0) {
- /* ask the cache to honour the operation */
- ret = object->cache->ops->check_consistency(op);
- fscache_op_complete(op, false);
- } else if (ret == -ENOBUFS) {
- ret = 0;
- }
-
- fscache_put_operation(op);
- _leave(" = %d", ret);
- return ret;
-
-submit_failed:
- wake_cookie = __fscache_unuse_cookie(cookie);
-inconsistent:
- spin_unlock(&cookie->lock);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- kfree(op);
- _leave(" = -ESTALE");
- return -ESTALE;
-}
-EXPORT_SYMBOL(__fscache_check_consistency);
-
/*
* Generate a list of extant cookies in /proc/fs/fscache/cookies
*/
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index bc66bf7182ed..20cbd1288b5a 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -151,15 +151,6 @@ extern void fscache_abort_object(struct fscache_object *);
extern void fscache_start_operations(struct fscache_object *);
extern void fscache_operation_gc(struct work_struct *);
-/*
- * page.c
- */
-extern int fscache_wait_for_deferred_lookup(struct fscache_cookie *);
-extern int fscache_wait_for_operation_activation(struct fscache_object *,
- struct fscache_operation *,
- atomic_t *,
- atomic_t *);
-
/*
* proc.c
*/
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
index fd9cc16abc18..73636e9d652d 100644
--- a/fs/fscache/page.c
+++ b/fs/fscache/page.c
@@ -96,85 +96,3 @@ int __fscache_attr_changed(struct fscache_cookie *cookie)
return -ENOBUFS;
}
EXPORT_SYMBOL(__fscache_attr_changed);
-
-/*
- * wait for a deferred lookup to complete
- */
-int fscache_wait_for_deferred_lookup(struct fscache_cookie *cookie)
-{
- unsigned long jif;
-
- _enter("");
-
- if (!test_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags)) {
- _leave(" = 0 [imm]");
- return 0;
- }
-
- fscache_stat(&fscache_n_retrievals_wait);
-
- jif = jiffies;
- if (wait_on_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP,
- TASK_INTERRUPTIBLE) != 0) {
- fscache_stat(&fscache_n_retrievals_intr);
- _leave(" = -ERESTARTSYS");
- return -ERESTARTSYS;
- }
-
- ASSERT(!test_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags));
-
- smp_rmb();
- fscache_hist(fscache_retrieval_delay_histogram, jif);
- _leave(" = 0 [dly]");
- return 0;
-}
-
-/*
- * wait for an object to become active (or dead)
- */
-int fscache_wait_for_operation_activation(struct fscache_object *object,
- struct fscache_operation *op,
- atomic_t *stat_op_waits,
- atomic_t *stat_object_dead)
-{
- int ret;
-
- if (!test_bit(FSCACHE_OP_WAITING, &op->flags))
- goto check_if_dead;
-
- _debug(">>> WT");
- if (stat_op_waits)
- fscache_stat(stat_op_waits);
- if (wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
- TASK_INTERRUPTIBLE) != 0) {
- trace_fscache_op(object->cookie, op, fscache_op_signal);
- ret = fscache_cancel_op(op, false);
- if (ret == 0)
- return -ERESTARTSYS;
-
- /* it's been removed from the pending queue by another party,
- * so we should get to run shortly */
- wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
- TASK_UNINTERRUPTIBLE);
- }
- _debug("<<< GO");
-
-check_if_dead:
- if (op->state == FSCACHE_OP_ST_CANCELLED) {
- if (stat_object_dead)
- fscache_stat(stat_object_dead);
- _leave(" = -ENOBUFS [cancelled]");
- return -ENOBUFS;
- }
- if (unlikely(fscache_object_is_dying(object) ||
- fscache_cache_is_broken(object))) {
- enum fscache_operation_state state = op->state;
- trace_fscache_op(object->cookie, op, fscache_op_signal);
- fscache_cancel_op(op, true);
- if (stat_object_dead)
- fscache_stat(stat_object_dead);
- _leave(" = -ENOBUFS [obj dead %d]", state);
- return -ENOBUFS;
- }
- return 0;
-}
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 81418056f43f..3a78e41d2338 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -167,10 +167,6 @@ struct fscache_cache_ops {
/* unpin an object in the cache */
void (*unpin_object)(struct fscache_object *object);
- /* check the consistency between the backing cache and the FS-Cache
- * cookie */
- int (*check_consistency)(struct fscache_operation *op);
-
/* store the updated auxiliary data on an object */
void (*update_object)(struct fscache_object *object);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 64d9ef34da49..82e871a3dc6a 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -139,7 +139,6 @@ extern struct fscache_cookie *__fscache_acquire_cookie(
const void *, size_t,
loff_t, bool);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, const void *, bool);
-extern int __fscache_check_consistency(struct fscache_cookie *, const void *);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
extern int __fscache_attr_changed(struct fscache_cookie *);
extern void __fscache_invalidate(struct fscache_cookie *);
@@ -290,28 +289,6 @@ void fscache_relinquish_cookie(struct fscache_cookie *cookie,
__fscache_relinquish_cookie(cookie, aux_data, retire);
}
-/**
- * fscache_check_consistency - Request validation of a cache's auxiliary data
- * @cookie: The cookie representing the cache object
- * @aux_data: The updated auxiliary data for the cookie (may be NULL)
- *
- * Request an consistency check from fscache, which passes the request to the
- * backing cache. The auxiliary data on the cookie will be updated first if
- * @aux_data is set.
- *
- * Returns 0 if consistent and -ESTALE if inconsistent. May also
- * return -ENOMEM and -ERESTARTSYS.
- */
-static inline
-int fscache_check_consistency(struct fscache_cookie *cookie,
- const void *aux_data)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_check_consistency(cookie, aux_data);
- else
- return 0;
-}
-
/**
* fscache_update_cookie - Request that a cache object be updated
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 15/61] fscache: Remove fscache_attr_changed()
2020-05-04 17:07 ` David Howells
` (14 preceding siblings ...)
(?)
@ 2020-05-04 17:09 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove fscache_attr_changed() as it's unused.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 13 ++---
fs/fscache/Makefile | 3 -
fs/fscache/page.c | 98 -----------------------------------------
include/linux/fscache-cache.h | 4 --
include/linux/fscache.h | 21 ---------
5 files changed, 6 insertions(+), 133 deletions(-)
delete mode 100644 fs/fscache/page.c
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index a3837ed090a8..81322e3acadd 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -10,7 +10,7 @@
#include <linux/xattr.h>
#include "internal.h"
-static int cachefiles_attr_changed(struct fscache_object *_object);
+static int cachefiles_attr_changed(struct cachefiles_object *object);
/*
* allocate an object record for a cookie lookup and prepare the lookup data
@@ -110,7 +110,7 @@ static int cachefiles_lookup_object(struct fscache_object *_object)
/* polish off by setting the attributes of non-index files */
if (ret == 0 &&
object->fscache.cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
- cachefiles_attr_changed(&object->fscache);
+ cachefiles_attr_changed(object);
if (ret < 0 && ret != -ETIMEDOUT) {
if (ret != -ENOBUFS)
@@ -324,9 +324,8 @@ static void cachefiles_sync_cache(struct fscache_cache *_cache)
* notification the attributes on an object have changed
* - called with reads/writes excluded by FS-Cache
*/
-static int cachefiles_attr_changed(struct fscache_object *_object)
+static int cachefiles_attr_changed(struct cachefiles_object *object)
{
- struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
struct iattr newattrs;
@@ -334,12 +333,11 @@ static int cachefiles_attr_changed(struct fscache_object *_object)
loff_t oi_size;
int ret;
- ni_size = _object->cookie->object_size;
+ ni_size = object->fscache.cookie->object_size;
_enter("{OBJ%x},[%llu]",
- _object->debug_id, (unsigned long long) ni_size);
+ object->fscache.debug_id, (unsigned long long) ni_size);
- object = container_of(_object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
@@ -442,5 +440,4 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.drop_object = cachefiles_drop_object,
.put_object = cachefiles_put_object,
.sync_cache = cachefiles_sync_cache,
- .attr_changed = cachefiles_attr_changed,
};
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 79e08e05ef84..565a3441d31d 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -10,8 +10,7 @@ fscache-y := \
main.o \
netfs.o \
object.o \
- operation.o \
- page.o
+ operation.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/page.c b/fs/fscache/page.c
deleted file mode 100644
index 73636e9d652d..000000000000
--- a/fs/fscache/page.c
+++ /dev/null
@@ -1,98 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Cache page management and data I/O routines
- *
- * Copyright (C) 2004-2008 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- */
-
-#define FSCACHE_DEBUG_LEVEL PAGE
-#include <linux/module.h>
-#include <linux/fscache-cache.h>
-#include <linux/buffer_head.h>
-#include <linux/pagevec.h>
-#include <linux/slab.h>
-#include "internal.h"
-
-/*
- * actually apply the changed attributes to a cache object
- */
-static void fscache_attr_changed_op(struct fscache_operation *op)
-{
- struct fscache_object *object = op->object;
- int ret;
-
- _enter("{OBJ%x OP%x}", object->debug_id, op->debug_id);
-
- fscache_stat(&fscache_n_attr_changed_calls);
-
- if (fscache_object_is_active(object)) {
- fscache_stat(&fscache_n_cop_attr_changed);
- ret = object->cache->ops->attr_changed(object);
- fscache_stat_d(&fscache_n_cop_attr_changed);
- if (ret < 0)
- fscache_abort_object(object);
- fscache_op_complete(op, ret < 0);
- } else {
- fscache_op_complete(op, true);
- }
-
- _leave("");
-}
-
-/*
- * notification that the attributes on an object have changed
- */
-int __fscache_attr_changed(struct fscache_cookie *cookie)
-{
- struct fscache_operation *op;
- struct fscache_object *object;
- bool wake_cookie = false;
-
- _enter("%p", cookie);
-
- ASSERTCMP(cookie->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
-
- fscache_stat(&fscache_n_attr_changed);
-
- op = kzalloc(sizeof(*op), GFP_KERNEL);
- if (!op) {
- fscache_stat(&fscache_n_attr_changed_nomem);
- _leave(" = -ENOMEM");
- return -ENOMEM;
- }
-
- fscache_operation_init(cookie, op, fscache_attr_changed_op, NULL, NULL);
- trace_fscache_page_op(cookie, NULL, op, fscache_page_op_attr_changed);
- op->flags = FSCACHE_OP_ASYNC |
- (1 << FSCACHE_OP_EXCLUSIVE) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
-
- spin_lock(&cookie->lock);
-
- if (!fscache_cookie_enabled(cookie) ||
- hlist_empty(&cookie->backing_objects))
- goto nobufs;
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
-
- __fscache_use_cookie(cookie);
- if (fscache_submit_exclusive_op(object, op) < 0)
- goto nobufs_dec;
- spin_unlock(&cookie->lock);
- fscache_stat(&fscache_n_attr_changed_ok);
- fscache_put_operation(op);
- _leave(" = 0");
- return 0;
-
-nobufs_dec:
- wake_cookie = __fscache_unuse_cookie(cookie);
-nobufs:
- spin_unlock(&cookie->lock);
- fscache_put_operation(op);
- if (wake_cookie)
- __fscache_wake_unused_cookie(cookie);
- fscache_stat(&fscache_n_attr_changed_nobufs);
- _leave(" = %d", -ENOBUFS);
- return -ENOBUFS;
-}
-EXPORT_SYMBOL(__fscache_attr_changed);
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 3a78e41d2338..627e1ab7123d 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -184,10 +184,6 @@ struct fscache_cache_ops {
/* sync a cache */
void (*sync_cache)(struct fscache_cache *cache);
- /* notification that the attributes of a non-index object (such as
- * i_size) have changed */
- int (*attr_changed)(struct fscache_object *object);
-
/* reserve space for an object's data and associated metadata */
int (*reserve_space)(struct fscache_object *object, loff_t i_size);
};
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 82e871a3dc6a..5eb2ac5f2bf1 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -140,7 +140,6 @@ extern struct fscache_cookie *__fscache_acquire_cookie(
loff_t, bool);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, const void *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
-extern int __fscache_attr_changed(struct fscache_cookie *);
extern void __fscache_invalidate(struct fscache_cookie *);
extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
extern void __fscache_disable_cookie(struct fscache_cookie *, const void *, bool);
@@ -337,26 +336,6 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
{
}
-/**
- * fscache_attr_changed - Notify cache that an object's attributes changed
- * @cookie: The cookie representing the cache object
- *
- * Send a notification to the cache indicating that an object's attributes have
- * changed. This includes the data size. These attributes will be obtained
- * through the get_attr() cookie definition op.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-int fscache_attr_changed(struct fscache_cookie *cookie)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- return __fscache_attr_changed(cookie);
- else
- return -ENOBUFS;
-}
-
/**
* fscache_invalidate - Notify cache that an object needs invalidation
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 16/61] fscache: Remove obsolete stats
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove a bunch of now-unused fscache stats counters that were obsoleted by
the removal of the old I/O routines.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/internal.h | 61 -----------------------
fs/fscache/stats.c | 129 -------------------------------------------------
2 files changed, 1 insertion(+), 189 deletions(-)
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 20cbd1288b5a..360137fd19a7 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -166,9 +166,6 @@ extern void fscache_proc_cleanup(void);
* stats.c
*/
#ifdef CONFIG_FSCACHE_STATS
-extern atomic_t fscache_n_ops_processed[FSCACHE_MAX_THREADS];
-extern atomic_t fscache_n_objs_processed[FSCACHE_MAX_THREADS];
-
extern atomic_t fscache_n_op_pend;
extern atomic_t fscache_n_op_run;
extern atomic_t fscache_n_op_enqueue;
@@ -179,52 +176,6 @@ extern atomic_t fscache_n_op_gc;
extern atomic_t fscache_n_op_cancelled;
extern atomic_t fscache_n_op_rejected;
-extern atomic_t fscache_n_attr_changed;
-extern atomic_t fscache_n_attr_changed_ok;
-extern atomic_t fscache_n_attr_changed_nobufs;
-extern atomic_t fscache_n_attr_changed_nomem;
-extern atomic_t fscache_n_attr_changed_calls;
-
-extern atomic_t fscache_n_allocs;
-extern atomic_t fscache_n_allocs_ok;
-extern atomic_t fscache_n_allocs_wait;
-extern atomic_t fscache_n_allocs_nobufs;
-extern atomic_t fscache_n_allocs_intr;
-extern atomic_t fscache_n_allocs_object_dead;
-extern atomic_t fscache_n_alloc_ops;
-extern atomic_t fscache_n_alloc_op_waits;
-
-extern atomic_t fscache_n_retrievals;
-extern atomic_t fscache_n_retrievals_ok;
-extern atomic_t fscache_n_retrievals_wait;
-extern atomic_t fscache_n_retrievals_nodata;
-extern atomic_t fscache_n_retrievals_nobufs;
-extern atomic_t fscache_n_retrievals_intr;
-extern atomic_t fscache_n_retrievals_nomem;
-extern atomic_t fscache_n_retrievals_object_dead;
-extern atomic_t fscache_n_retrieval_ops;
-extern atomic_t fscache_n_retrieval_op_waits;
-
-extern atomic_t fscache_n_stores;
-extern atomic_t fscache_n_stores_ok;
-extern atomic_t fscache_n_stores_again;
-extern atomic_t fscache_n_stores_nobufs;
-extern atomic_t fscache_n_stores_oom;
-extern atomic_t fscache_n_store_ops;
-extern atomic_t fscache_n_store_calls;
-extern atomic_t fscache_n_store_pages;
-extern atomic_t fscache_n_store_radix_deletes;
-extern atomic_t fscache_n_store_pages_over_limit;
-
-extern atomic_t fscache_n_store_vmscan_not_storing;
-extern atomic_t fscache_n_store_vmscan_gone;
-extern atomic_t fscache_n_store_vmscan_busy;
-extern atomic_t fscache_n_store_vmscan_cancelled;
-extern atomic_t fscache_n_store_vmscan_wait;
-
-extern atomic_t fscache_n_marks;
-extern atomic_t fscache_n_uncaches;
-
extern atomic_t fscache_n_acquires;
extern atomic_t fscache_n_acquires_null;
extern atomic_t fscache_n_acquires_no_cache;
@@ -241,7 +192,6 @@ extern atomic_t fscache_n_updates_run;
extern atomic_t fscache_n_relinquishes;
extern atomic_t fscache_n_relinquishes_null;
-extern atomic_t fscache_n_relinquishes_waitcrt;
extern atomic_t fscache_n_relinquishes_retire;
extern atomic_t fscache_n_cookie_index;
@@ -258,11 +208,6 @@ extern atomic_t fscache_n_object_created;
extern atomic_t fscache_n_object_avail;
extern atomic_t fscache_n_object_dead;
-extern atomic_t fscache_n_checkaux_none;
-extern atomic_t fscache_n_checkaux_okay;
-extern atomic_t fscache_n_checkaux_update;
-extern atomic_t fscache_n_checkaux_obsolete;
-
extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
extern atomic_t fscache_n_cop_lookup_complete;
@@ -273,12 +218,6 @@ extern atomic_t fscache_n_cop_drop_object;
extern atomic_t fscache_n_cop_put_object;
extern atomic_t fscache_n_cop_sync_cache;
extern atomic_t fscache_n_cop_attr_changed;
-extern atomic_t fscache_n_cop_read_or_alloc_page;
-extern atomic_t fscache_n_cop_read_or_alloc_pages;
-extern atomic_t fscache_n_cop_allocate_page;
-extern atomic_t fscache_n_cop_allocate_pages;
-extern atomic_t fscache_n_cop_write_page;
-extern atomic_t fscache_n_cop_uncache_page;
extern atomic_t fscache_n_cache_no_space_reject;
extern atomic_t fscache_n_cache_stale_objects;
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 281022871e70..5b1cec456199 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -24,52 +24,6 @@ atomic_t fscache_n_op_gc;
atomic_t fscache_n_op_cancelled;
atomic_t fscache_n_op_rejected;
-atomic_t fscache_n_attr_changed;
-atomic_t fscache_n_attr_changed_ok;
-atomic_t fscache_n_attr_changed_nobufs;
-atomic_t fscache_n_attr_changed_nomem;
-atomic_t fscache_n_attr_changed_calls;
-
-atomic_t fscache_n_allocs;
-atomic_t fscache_n_allocs_ok;
-atomic_t fscache_n_allocs_wait;
-atomic_t fscache_n_allocs_nobufs;
-atomic_t fscache_n_allocs_intr;
-atomic_t fscache_n_allocs_object_dead;
-atomic_t fscache_n_alloc_ops;
-atomic_t fscache_n_alloc_op_waits;
-
-atomic_t fscache_n_retrievals;
-atomic_t fscache_n_retrievals_ok;
-atomic_t fscache_n_retrievals_wait;
-atomic_t fscache_n_retrievals_nodata;
-atomic_t fscache_n_retrievals_nobufs;
-atomic_t fscache_n_retrievals_intr;
-atomic_t fscache_n_retrievals_nomem;
-atomic_t fscache_n_retrievals_object_dead;
-atomic_t fscache_n_retrieval_ops;
-atomic_t fscache_n_retrieval_op_waits;
-
-atomic_t fscache_n_stores;
-atomic_t fscache_n_stores_ok;
-atomic_t fscache_n_stores_again;
-atomic_t fscache_n_stores_nobufs;
-atomic_t fscache_n_stores_oom;
-atomic_t fscache_n_store_ops;
-atomic_t fscache_n_store_calls;
-atomic_t fscache_n_store_pages;
-atomic_t fscache_n_store_radix_deletes;
-atomic_t fscache_n_store_pages_over_limit;
-
-atomic_t fscache_n_store_vmscan_not_storing;
-atomic_t fscache_n_store_vmscan_gone;
-atomic_t fscache_n_store_vmscan_busy;
-atomic_t fscache_n_store_vmscan_cancelled;
-atomic_t fscache_n_store_vmscan_wait;
-
-atomic_t fscache_n_marks;
-atomic_t fscache_n_uncaches;
-
atomic_t fscache_n_acquires;
atomic_t fscache_n_acquires_null;
atomic_t fscache_n_acquires_no_cache;
@@ -86,7 +40,6 @@ atomic_t fscache_n_updates_run;
atomic_t fscache_n_relinquishes;
atomic_t fscache_n_relinquishes_null;
-atomic_t fscache_n_relinquishes_waitcrt;
atomic_t fscache_n_relinquishes_retire;
atomic_t fscache_n_cookie_index;
@@ -103,11 +56,6 @@ atomic_t fscache_n_object_created;
atomic_t fscache_n_object_avail;
atomic_t fscache_n_object_dead;
-atomic_t fscache_n_checkaux_none;
-atomic_t fscache_n_checkaux_okay;
-atomic_t fscache_n_checkaux_update;
-atomic_t fscache_n_checkaux_obsolete;
-
atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
atomic_t fscache_n_cop_lookup_complete;
@@ -118,12 +66,6 @@ atomic_t fscache_n_cop_drop_object;
atomic_t fscache_n_cop_put_object;
atomic_t fscache_n_cop_sync_cache;
atomic_t fscache_n_cop_attr_changed;
-atomic_t fscache_n_cop_read_or_alloc_page;
-atomic_t fscache_n_cop_read_or_alloc_pages;
-atomic_t fscache_n_cop_allocate_page;
-atomic_t fscache_n_cop_allocate_pages;
-atomic_t fscache_n_cop_write_page;
-atomic_t fscache_n_cop_uncache_page;
atomic_t fscache_n_cache_no_space_reject;
atomic_t fscache_n_cache_stale_objects;
@@ -147,15 +89,6 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_object_no_alloc),
atomic_read(&fscache_n_object_avail),
atomic_read(&fscache_n_object_dead));
- seq_printf(m, "ChkAux : non=%u ok=%u upd=%u obs=%u\n",
- atomic_read(&fscache_n_checkaux_none),
- atomic_read(&fscache_n_checkaux_okay),
- atomic_read(&fscache_n_checkaux_update),
- atomic_read(&fscache_n_checkaux_obsolete));
-
- seq_printf(m, "Pages : mrk=%u unc=%u\n",
- atomic_read(&fscache_n_marks),
- atomic_read(&fscache_n_uncaches));
seq_printf(m, "Acquire: n=%u nul=%u noc=%u ok=%u nbf=%u"
" oom=%u\n",
@@ -182,64 +115,11 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_updates_null),
atomic_read(&fscache_n_updates_run));
- seq_printf(m, "Relinqs: n=%u nul=%u wcr=%u rtr=%u\n",
+ seq_printf(m, "Relinqs: n=%u nul=%u rtr=%u\n",
atomic_read(&fscache_n_relinquishes),
atomic_read(&fscache_n_relinquishes_null),
- atomic_read(&fscache_n_relinquishes_waitcrt),
atomic_read(&fscache_n_relinquishes_retire));
- seq_printf(m, "AttrChg: n=%u ok=%u nbf=%u oom=%u run=%u\n",
- atomic_read(&fscache_n_attr_changed),
- atomic_read(&fscache_n_attr_changed_ok),
- atomic_read(&fscache_n_attr_changed_nobufs),
- atomic_read(&fscache_n_attr_changed_nomem),
- atomic_read(&fscache_n_attr_changed_calls));
-
- seq_printf(m, "Allocs : n=%u ok=%u wt=%u nbf=%u int=%u\n",
- atomic_read(&fscache_n_allocs),
- atomic_read(&fscache_n_allocs_ok),
- atomic_read(&fscache_n_allocs_wait),
- atomic_read(&fscache_n_allocs_nobufs),
- atomic_read(&fscache_n_allocs_intr));
- seq_printf(m, "Allocs : ops=%u owt=%u abt=%u\n",
- atomic_read(&fscache_n_alloc_ops),
- atomic_read(&fscache_n_alloc_op_waits),
- atomic_read(&fscache_n_allocs_object_dead));
-
- seq_printf(m, "Retrvls: n=%u ok=%u wt=%u nod=%u nbf=%u"
- " int=%u oom=%u\n",
- atomic_read(&fscache_n_retrievals),
- atomic_read(&fscache_n_retrievals_ok),
- atomic_read(&fscache_n_retrievals_wait),
- atomic_read(&fscache_n_retrievals_nodata),
- atomic_read(&fscache_n_retrievals_nobufs),
- atomic_read(&fscache_n_retrievals_intr),
- atomic_read(&fscache_n_retrievals_nomem));
- seq_printf(m, "Retrvls: ops=%u owt=%u abt=%u\n",
- atomic_read(&fscache_n_retrieval_ops),
- atomic_read(&fscache_n_retrieval_op_waits),
- atomic_read(&fscache_n_retrievals_object_dead));
-
- seq_printf(m, "Stores : n=%u ok=%u agn=%u nbf=%u oom=%u\n",
- atomic_read(&fscache_n_stores),
- atomic_read(&fscache_n_stores_ok),
- atomic_read(&fscache_n_stores_again),
- atomic_read(&fscache_n_stores_nobufs),
- atomic_read(&fscache_n_stores_oom));
- seq_printf(m, "Stores : ops=%u run=%u pgs=%u rxd=%u olm=%u\n",
- atomic_read(&fscache_n_store_ops),
- atomic_read(&fscache_n_store_calls),
- atomic_read(&fscache_n_store_pages),
- atomic_read(&fscache_n_store_radix_deletes),
- atomic_read(&fscache_n_store_pages_over_limit));
-
- seq_printf(m, "VmScan : nos=%u gon=%u bsy=%u can=%u wt=%u\n",
- atomic_read(&fscache_n_store_vmscan_not_storing),
- atomic_read(&fscache_n_store_vmscan_gone),
- atomic_read(&fscache_n_store_vmscan_busy),
- atomic_read(&fscache_n_store_vmscan_cancelled),
- atomic_read(&fscache_n_store_vmscan_wait));
-
seq_printf(m, "Ops : pend=%u run=%u enq=%u can=%u rej=%u\n",
atomic_read(&fscache_n_op_pend),
atomic_read(&fscache_n_op_run),
@@ -264,13 +144,6 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
atomic_read(&fscache_n_cop_sync_cache));
- seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d\n",
- atomic_read(&fscache_n_cop_read_or_alloc_page),
- atomic_read(&fscache_n_cop_read_or_alloc_pages),
- atomic_read(&fscache_n_cop_allocate_page),
- atomic_read(&fscache_n_cop_allocate_pages),
- atomic_read(&fscache_n_cop_write_page),
- atomic_read(&fscache_n_cop_uncache_page));
seq_printf(m, "CacheEv: nsp=%d stl=%d rtr=%d cul=%d\n",
atomic_read(&fscache_n_cache_no_space_reject),
atomic_read(&fscache_n_cache_stale_objects),
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 16/61] fscache: Remove obsolete stats
@ 2020-05-04 17:09 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:09 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove a bunch of now-unused fscache stats counters that were obsoleted by
the removal of the old I/O routines.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/fscache/internal.h | 61 -----------------------
fs/fscache/stats.c | 129 -------------------------------------------------
2 files changed, 1 insertion(+), 189 deletions(-)
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 20cbd1288b5a..360137fd19a7 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -166,9 +166,6 @@ extern void fscache_proc_cleanup(void);
* stats.c
*/
#ifdef CONFIG_FSCACHE_STATS
-extern atomic_t fscache_n_ops_processed[FSCACHE_MAX_THREADS];
-extern atomic_t fscache_n_objs_processed[FSCACHE_MAX_THREADS];
-
extern atomic_t fscache_n_op_pend;
extern atomic_t fscache_n_op_run;
extern atomic_t fscache_n_op_enqueue;
@@ -179,52 +176,6 @@ extern atomic_t fscache_n_op_gc;
extern atomic_t fscache_n_op_cancelled;
extern atomic_t fscache_n_op_rejected;
-extern atomic_t fscache_n_attr_changed;
-extern atomic_t fscache_n_attr_changed_ok;
-extern atomic_t fscache_n_attr_changed_nobufs;
-extern atomic_t fscache_n_attr_changed_nomem;
-extern atomic_t fscache_n_attr_changed_calls;
-
-extern atomic_t fscache_n_allocs;
-extern atomic_t fscache_n_allocs_ok;
-extern atomic_t fscache_n_allocs_wait;
-extern atomic_t fscache_n_allocs_nobufs;
-extern atomic_t fscache_n_allocs_intr;
-extern atomic_t fscache_n_allocs_object_dead;
-extern atomic_t fscache_n_alloc_ops;
-extern atomic_t fscache_n_alloc_op_waits;
-
-extern atomic_t fscache_n_retrievals;
-extern atomic_t fscache_n_retrievals_ok;
-extern atomic_t fscache_n_retrievals_wait;
-extern atomic_t fscache_n_retrievals_nodata;
-extern atomic_t fscache_n_retrievals_nobufs;
-extern atomic_t fscache_n_retrievals_intr;
-extern atomic_t fscache_n_retrievals_nomem;
-extern atomic_t fscache_n_retrievals_object_dead;
-extern atomic_t fscache_n_retrieval_ops;
-extern atomic_t fscache_n_retrieval_op_waits;
-
-extern atomic_t fscache_n_stores;
-extern atomic_t fscache_n_stores_ok;
-extern atomic_t fscache_n_stores_again;
-extern atomic_t fscache_n_stores_nobufs;
-extern atomic_t fscache_n_stores_oom;
-extern atomic_t fscache_n_store_ops;
-extern atomic_t fscache_n_store_calls;
-extern atomic_t fscache_n_store_pages;
-extern atomic_t fscache_n_store_radix_deletes;
-extern atomic_t fscache_n_store_pages_over_limit;
-
-extern atomic_t fscache_n_store_vmscan_not_storing;
-extern atomic_t fscache_n_store_vmscan_gone;
-extern atomic_t fscache_n_store_vmscan_busy;
-extern atomic_t fscache_n_store_vmscan_cancelled;
-extern atomic_t fscache_n_store_vmscan_wait;
-
-extern atomic_t fscache_n_marks;
-extern atomic_t fscache_n_uncaches;
-
extern atomic_t fscache_n_acquires;
extern atomic_t fscache_n_acquires_null;
extern atomic_t fscache_n_acquires_no_cache;
@@ -241,7 +192,6 @@ extern atomic_t fscache_n_updates_run;
extern atomic_t fscache_n_relinquishes;
extern atomic_t fscache_n_relinquishes_null;
-extern atomic_t fscache_n_relinquishes_waitcrt;
extern atomic_t fscache_n_relinquishes_retire;
extern atomic_t fscache_n_cookie_index;
@@ -258,11 +208,6 @@ extern atomic_t fscache_n_object_created;
extern atomic_t fscache_n_object_avail;
extern atomic_t fscache_n_object_dead;
-extern atomic_t fscache_n_checkaux_none;
-extern atomic_t fscache_n_checkaux_okay;
-extern atomic_t fscache_n_checkaux_update;
-extern atomic_t fscache_n_checkaux_obsolete;
-
extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
extern atomic_t fscache_n_cop_lookup_complete;
@@ -273,12 +218,6 @@ extern atomic_t fscache_n_cop_drop_object;
extern atomic_t fscache_n_cop_put_object;
extern atomic_t fscache_n_cop_sync_cache;
extern atomic_t fscache_n_cop_attr_changed;
-extern atomic_t fscache_n_cop_read_or_alloc_page;
-extern atomic_t fscache_n_cop_read_or_alloc_pages;
-extern atomic_t fscache_n_cop_allocate_page;
-extern atomic_t fscache_n_cop_allocate_pages;
-extern atomic_t fscache_n_cop_write_page;
-extern atomic_t fscache_n_cop_uncache_page;
extern atomic_t fscache_n_cache_no_space_reject;
extern atomic_t fscache_n_cache_stale_objects;
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 281022871e70..5b1cec456199 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -24,52 +24,6 @@ atomic_t fscache_n_op_gc;
atomic_t fscache_n_op_cancelled;
atomic_t fscache_n_op_rejected;
-atomic_t fscache_n_attr_changed;
-atomic_t fscache_n_attr_changed_ok;
-atomic_t fscache_n_attr_changed_nobufs;
-atomic_t fscache_n_attr_changed_nomem;
-atomic_t fscache_n_attr_changed_calls;
-
-atomic_t fscache_n_allocs;
-atomic_t fscache_n_allocs_ok;
-atomic_t fscache_n_allocs_wait;
-atomic_t fscache_n_allocs_nobufs;
-atomic_t fscache_n_allocs_intr;
-atomic_t fscache_n_allocs_object_dead;
-atomic_t fscache_n_alloc_ops;
-atomic_t fscache_n_alloc_op_waits;
-
-atomic_t fscache_n_retrievals;
-atomic_t fscache_n_retrievals_ok;
-atomic_t fscache_n_retrievals_wait;
-atomic_t fscache_n_retrievals_nodata;
-atomic_t fscache_n_retrievals_nobufs;
-atomic_t fscache_n_retrievals_intr;
-atomic_t fscache_n_retrievals_nomem;
-atomic_t fscache_n_retrievals_object_dead;
-atomic_t fscache_n_retrieval_ops;
-atomic_t fscache_n_retrieval_op_waits;
-
-atomic_t fscache_n_stores;
-atomic_t fscache_n_stores_ok;
-atomic_t fscache_n_stores_again;
-atomic_t fscache_n_stores_nobufs;
-atomic_t fscache_n_stores_oom;
-atomic_t fscache_n_store_ops;
-atomic_t fscache_n_store_calls;
-atomic_t fscache_n_store_pages;
-atomic_t fscache_n_store_radix_deletes;
-atomic_t fscache_n_store_pages_over_limit;
-
-atomic_t fscache_n_store_vmscan_not_storing;
-atomic_t fscache_n_store_vmscan_gone;
-atomic_t fscache_n_store_vmscan_busy;
-atomic_t fscache_n_store_vmscan_cancelled;
-atomic_t fscache_n_store_vmscan_wait;
-
-atomic_t fscache_n_marks;
-atomic_t fscache_n_uncaches;
-
atomic_t fscache_n_acquires;
atomic_t fscache_n_acquires_null;
atomic_t fscache_n_acquires_no_cache;
@@ -86,7 +40,6 @@ atomic_t fscache_n_updates_run;
atomic_t fscache_n_relinquishes;
atomic_t fscache_n_relinquishes_null;
-atomic_t fscache_n_relinquishes_waitcrt;
atomic_t fscache_n_relinquishes_retire;
atomic_t fscache_n_cookie_index;
@@ -103,11 +56,6 @@ atomic_t fscache_n_object_created;
atomic_t fscache_n_object_avail;
atomic_t fscache_n_object_dead;
-atomic_t fscache_n_checkaux_none;
-atomic_t fscache_n_checkaux_okay;
-atomic_t fscache_n_checkaux_update;
-atomic_t fscache_n_checkaux_obsolete;
-
atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
atomic_t fscache_n_cop_lookup_complete;
@@ -118,12 +66,6 @@ atomic_t fscache_n_cop_drop_object;
atomic_t fscache_n_cop_put_object;
atomic_t fscache_n_cop_sync_cache;
atomic_t fscache_n_cop_attr_changed;
-atomic_t fscache_n_cop_read_or_alloc_page;
-atomic_t fscache_n_cop_read_or_alloc_pages;
-atomic_t fscache_n_cop_allocate_page;
-atomic_t fscache_n_cop_allocate_pages;
-atomic_t fscache_n_cop_write_page;
-atomic_t fscache_n_cop_uncache_page;
atomic_t fscache_n_cache_no_space_reject;
atomic_t fscache_n_cache_stale_objects;
@@ -147,15 +89,6 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_object_no_alloc),
atomic_read(&fscache_n_object_avail),
atomic_read(&fscache_n_object_dead));
- seq_printf(m, "ChkAux : non=%u ok=%u upd=%u obs=%u\n",
- atomic_read(&fscache_n_checkaux_none),
- atomic_read(&fscache_n_checkaux_okay),
- atomic_read(&fscache_n_checkaux_update),
- atomic_read(&fscache_n_checkaux_obsolete));
-
- seq_printf(m, "Pages : mrk=%u unc=%u\n",
- atomic_read(&fscache_n_marks),
- atomic_read(&fscache_n_uncaches));
seq_printf(m, "Acquire: n=%u nul=%u noc=%u ok=%u nbf=%u"
" oom=%u\n",
@@ -182,64 +115,11 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_updates_null),
atomic_read(&fscache_n_updates_run));
- seq_printf(m, "Relinqs: n=%u nul=%u wcr=%u rtr=%u\n",
+ seq_printf(m, "Relinqs: n=%u nul=%u rtr=%u\n",
atomic_read(&fscache_n_relinquishes),
atomic_read(&fscache_n_relinquishes_null),
- atomic_read(&fscache_n_relinquishes_waitcrt),
atomic_read(&fscache_n_relinquishes_retire));
- seq_printf(m, "AttrChg: n=%u ok=%u nbf=%u oom=%u run=%u\n",
- atomic_read(&fscache_n_attr_changed),
- atomic_read(&fscache_n_attr_changed_ok),
- atomic_read(&fscache_n_attr_changed_nobufs),
- atomic_read(&fscache_n_attr_changed_nomem),
- atomic_read(&fscache_n_attr_changed_calls));
-
- seq_printf(m, "Allocs : n=%u ok=%u wt=%u nbf=%u int=%u\n",
- atomic_read(&fscache_n_allocs),
- atomic_read(&fscache_n_allocs_ok),
- atomic_read(&fscache_n_allocs_wait),
- atomic_read(&fscache_n_allocs_nobufs),
- atomic_read(&fscache_n_allocs_intr));
- seq_printf(m, "Allocs : ops=%u owt=%u abt=%u\n",
- atomic_read(&fscache_n_alloc_ops),
- atomic_read(&fscache_n_alloc_op_waits),
- atomic_read(&fscache_n_allocs_object_dead));
-
- seq_printf(m, "Retrvls: n=%u ok=%u wt=%u nod=%u nbf=%u"
- " int=%u oom=%u\n",
- atomic_read(&fscache_n_retrievals),
- atomic_read(&fscache_n_retrievals_ok),
- atomic_read(&fscache_n_retrievals_wait),
- atomic_read(&fscache_n_retrievals_nodata),
- atomic_read(&fscache_n_retrievals_nobufs),
- atomic_read(&fscache_n_retrievals_intr),
- atomic_read(&fscache_n_retrievals_nomem));
- seq_printf(m, "Retrvls: ops=%u owt=%u abt=%u\n",
- atomic_read(&fscache_n_retrieval_ops),
- atomic_read(&fscache_n_retrieval_op_waits),
- atomic_read(&fscache_n_retrievals_object_dead));
-
- seq_printf(m, "Stores : n=%u ok=%u agn=%u nbf=%u oom=%u\n",
- atomic_read(&fscache_n_stores),
- atomic_read(&fscache_n_stores_ok),
- atomic_read(&fscache_n_stores_again),
- atomic_read(&fscache_n_stores_nobufs),
- atomic_read(&fscache_n_stores_oom));
- seq_printf(m, "Stores : ops=%u run=%u pgs=%u rxd=%u olm=%u\n",
- atomic_read(&fscache_n_store_ops),
- atomic_read(&fscache_n_store_calls),
- atomic_read(&fscache_n_store_pages),
- atomic_read(&fscache_n_store_radix_deletes),
- atomic_read(&fscache_n_store_pages_over_limit));
-
- seq_printf(m, "VmScan : nos=%u gon=%u bsy=%u can=%u wt=%u\n",
- atomic_read(&fscache_n_store_vmscan_not_storing),
- atomic_read(&fscache_n_store_vmscan_gone),
- atomic_read(&fscache_n_store_vmscan_busy),
- atomic_read(&fscache_n_store_vmscan_cancelled),
- atomic_read(&fscache_n_store_vmscan_wait));
-
seq_printf(m, "Ops : pend=%u run=%u enq=%u can=%u rej=%u\n",
atomic_read(&fscache_n_op_pend),
atomic_read(&fscache_n_op_run),
@@ -264,13 +144,6 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
atomic_read(&fscache_n_cop_sync_cache));
- seq_printf(m, "CacheOp: rap=%d ras=%d alp=%d als=%d wrp=%d ucp=%d\n",
- atomic_read(&fscache_n_cop_read_or_alloc_page),
- atomic_read(&fscache_n_cop_read_or_alloc_pages),
- atomic_read(&fscache_n_cop_allocate_page),
- atomic_read(&fscache_n_cop_allocate_pages),
- atomic_read(&fscache_n_cop_write_page),
- atomic_read(&fscache_n_cop_uncache_page));
seq_printf(m, "CacheEv: nsp=%d stl=%d rtr=%d cul=%d\n",
atomic_read(&fscache_n_cache_no_space_reject),
atomic_read(&fscache_n_cache_stale_objects),
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 17/61] fscache: Remove old I/O tracepoints
2020-05-04 17:07 ` David Howells
` (16 preceding siblings ...)
(?)
@ 2020-05-04 17:10 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove now-unused fscache tracepoints that have been obsoleted by the
removal of the old I/O code.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/object.c | 1
include/trace/events/fscache.h | 195 ----------------------------------------
2 files changed, 196 deletions(-)
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index ede38bd4774a..a90451cdbdde 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -942,7 +942,6 @@ static const struct fscache_state *_fscache_invalidate_object(struct fscache_obj
op->flags = FSCACHE_OP_ASYNC |
(1 << FSCACHE_OP_EXCLUSIVE) |
(1 << FSCACHE_OP_UNUSE_COOKIE);
- trace_fscache_page_op(cookie, NULL, op, fscache_page_op_invalidate);
spin_lock(&cookie->lock);
if (fscache_submit_exclusive_op(object, op) < 0)
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 953e15fad063..2ebfd688a7c2 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -33,24 +33,6 @@ enum fscache_cookie_trace {
fscache_cookie_put_parent,
};
-enum fscache_page_trace {
- fscache_page_cached,
- fscache_page_inval,
- fscache_page_maybe_release,
- fscache_page_radix_clear_store,
- fscache_page_radix_delete,
- fscache_page_radix_insert,
- fscache_page_radix_pend2store,
- fscache_page_radix_set_pend,
- fscache_page_uncache,
- fscache_page_write,
- fscache_page_write_end,
- fscache_page_write_end_pend,
- fscache_page_write_end_noc,
- fscache_page_write_wait,
- fscache_page_trace__nr
-};
-
enum fscache_op_trace {
fscache_op_cancel,
fscache_op_cancel_all,
@@ -69,17 +51,6 @@ enum fscache_op_trace {
fscache_op_trace__nr
};
-enum fscache_page_op_trace {
- fscache_page_op_alloc_one,
- fscache_page_op_attr_changed,
- fscache_page_op_check_consistency,
- fscache_page_op_invalidate,
- fscache_page_op_retr_multi,
- fscache_page_op_retr_one,
- fscache_page_op_write_one,
- fscache_page_op_trace__nr
-};
-
#endif
/*
@@ -98,22 +69,6 @@ enum fscache_page_op_trace {
EM(fscache_cookie_put_object, "PUT obj") \
E_(fscache_cookie_put_parent, "PUT prn")
-#define fscache_page_traces \
- EM(fscache_page_cached, "Cached ") \
- EM(fscache_page_inval, "InvalPg") \
- EM(fscache_page_maybe_release, "MayRels") \
- EM(fscache_page_uncache, "Uncache") \
- EM(fscache_page_radix_clear_store, "RxCStr ") \
- EM(fscache_page_radix_delete, "RxDel ") \
- EM(fscache_page_radix_insert, "RxIns ") \
- EM(fscache_page_radix_pend2store, "RxP2S ") \
- EM(fscache_page_radix_set_pend, "RxSPend ") \
- EM(fscache_page_write, "WritePg") \
- EM(fscache_page_write_end, "EndPgWr") \
- EM(fscache_page_write_end_pend, "EndPgWP") \
- EM(fscache_page_write_end_noc, "EndPgNC") \
- E_(fscache_page_write_wait, "WtOnWrt")
-
#define fscache_op_traces \
EM(fscache_op_cancel, "Cancel1") \
EM(fscache_op_cancel_all, "CancelA") \
@@ -130,15 +85,6 @@ enum fscache_page_op_trace {
EM(fscache_op_submit_ex, "SubmitX") \
E_(fscache_op_work, "Work ")
-#define fscache_page_op_traces \
- EM(fscache_page_op_alloc_one, "Alloc1 ") \
- EM(fscache_page_op_attr_changed, "AttrChg") \
- EM(fscache_page_op_check_consistency, "CheckCn") \
- EM(fscache_page_op_invalidate, "Inval ") \
- EM(fscache_page_op_retr_multi, "RetrMul") \
- EM(fscache_page_op_retr_one, "Retr1 ") \
- E_(fscache_page_op_write_one, "Write1 ")
-
/*
* Export enum symbols via userspace.
*/
@@ -363,70 +309,6 @@ TRACE_EVENT(fscache_osm,
__entry->event_num)
);
-TRACE_EVENT(fscache_page,
- TP_PROTO(struct fscache_cookie *cookie, struct page *page,
- enum fscache_page_trace why),
-
- TP_ARGS(cookie, page, why),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(pgoff_t, page )
- __field(enum fscache_page_trace, why )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->page = page->index;
- __entry->why = why;
- ),
-
- TP_printk("c=%08x %s pg=%lx",
- __entry->cookie,
- __print_symbolic(__entry->why, fscache_page_traces),
- __entry->page)
- );
-
-TRACE_EVENT(fscache_check_page,
- TP_PROTO(struct fscache_cookie *cookie, struct page *page,
- void *val, int n),
-
- TP_ARGS(cookie, page, val, n),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(void *, page )
- __field(void *, val )
- __field(int, n )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->page = page;
- __entry->val = val;
- __entry->n = n;
- ),
-
- TP_printk("c=%08x pg=%p val=%p n=%d",
- __entry->cookie, __entry->page, __entry->val, __entry->n)
- );
-
-TRACE_EVENT(fscache_wake_cookie,
- TP_PROTO(struct fscache_cookie *cookie),
-
- TP_ARGS(cookie),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- ),
-
- TP_printk("c=%08x", __entry->cookie)
- );
-
TRACE_EVENT(fscache_op,
TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
enum fscache_op_trace why),
@@ -450,83 +332,6 @@ TRACE_EVENT(fscache_op,
__print_symbolic(__entry->why, fscache_op_traces))
);
-TRACE_EVENT(fscache_page_op,
- TP_PROTO(struct fscache_cookie *cookie, struct page *page,
- struct fscache_operation *op, enum fscache_page_op_trace what),
-
- TP_ARGS(cookie, page, op, what),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, op )
- __field(pgoff_t, page )
- __field(enum fscache_page_op_trace, what )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->page = page ? page->index : 0;
- __entry->op = op->debug_id;
- __entry->what = what;
- ),
-
- TP_printk("c=%08x %s pg=%lx op=%08x",
- __entry->cookie,
- __print_symbolic(__entry->what, fscache_page_op_traces),
- __entry->page, __entry->op)
- );
-
-TRACE_EVENT(fscache_wrote_page,
- TP_PROTO(struct fscache_cookie *cookie, struct page *page,
- struct fscache_operation *op, int ret),
-
- TP_ARGS(cookie, page, op, ret),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, op )
- __field(pgoff_t, page )
- __field(int, ret )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->page = page->index;
- __entry->op = op->debug_id;
- __entry->ret = ret;
- ),
-
- TP_printk("c=%08x pg=%lx op=%08x ret=%d",
- __entry->cookie, __entry->page, __entry->op, __entry->ret)
- );
-
-TRACE_EVENT(fscache_gang_lookup,
- TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
- void **results, int n, pgoff_t store_limit),
-
- TP_ARGS(cookie, op, results, n, store_limit),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, op )
- __field(pgoff_t, results0 )
- __field(int, n )
- __field(pgoff_t, store_limit )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->op = op->debug_id;
- __entry->results0 = results[0] ? ((struct page *)results[0])->index : (pgoff_t)-1;
- __entry->n = n;
- __entry->store_limit = store_limit;
- ),
-
- TP_printk("c=%08x op=%08x r0=%lx n=%d sl=%lx",
- __entry->cookie, __entry->op, __entry->results0, __entry->n,
- __entry->store_limit)
- );
-
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 18/61] fscache: Temporarily disable fscache_invalidate()
2020-05-04 17:07 ` David Howells
` (17 preceding siblings ...)
(?)
@ 2020-05-04 17:10 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Temporarily disable the fscache side of fscache_invalidate() so that the
operation managing code can be removed.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 9 ++---
fs/fscache/cookie.c | 4 +-
fs/fscache/object.c | 78 +----------------------------------------
include/linux/fscache-cache.h | 2 +
4 files changed, 7 insertions(+), 86 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 81322e3acadd..99f42d216ef7 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -388,7 +388,7 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
/*
* Invalidate an object
*/
-static void cachefiles_invalidate_object(struct fscache_operation *op)
+static void cachefiles_invalidate_object(struct fscache_object *_object)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
@@ -397,14 +397,14 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
uint64_t ni_size;
int ret;
- object = container_of(op->object, struct cachefiles_object, fscache);
+ object = container_of(_object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
- ni_size = op->object->cookie->object_size;
+ ni_size = object->fscache.cookie->object_size;
_enter("{OBJ%x},[%llu]",
- op->object->debug_id, (unsigned long long)ni_size);
+ object->fscache.debug_id, (unsigned long long)ni_size);
if (object->backer) {
ASSERT(d_is_reg(object->backer));
@@ -425,7 +425,6 @@ static void cachefiles_invalidate_object(struct fscache_operation *op)
}
}
- fscache_op_complete(op, true);
_leave("");
}
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index cd2e166d5f29..cd105227cb83 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -653,9 +653,7 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
object = hlist_entry(cookie->backing_objects.first,
struct fscache_object,
cookie_link);
- if (fscache_object_is_live(object))
- fscache_raise_event(
- object, FSCACHE_OBJECT_EV_INVALIDATE);
+ /* TODO: Do invalidation */
}
spin_unlock(&cookie->lock);
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index a90451cdbdde..73231fdf565c 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -908,86 +908,10 @@ static void fscache_dequeue_object(struct fscache_object *object)
_leave("");
}
-/*
- * Asynchronously invalidate an object.
- */
-static const struct fscache_state *_fscache_invalidate_object(struct fscache_object *object,
- int event)
-{
- struct fscache_operation *op;
- struct fscache_cookie *cookie = object->cookie;
-
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- /* We're going to need the cookie. If the cookie is not available then
- * retire the object instead.
- */
- if (!fscache_use_cookie(object)) {
- set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
- _leave(" [no cookie]");
- return transit_to(KILL_OBJECT);
- }
-
- /* Reject any new read/write ops and abort any that are pending. */
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- fscache_cancel_all_ops(object);
-
- /* Now we have to wait for in-progress reads and writes */
- op = kzalloc(sizeof(*op), GFP_KERNEL);
- if (!op)
- goto nomem;
-
- fscache_operation_init(cookie, op, object->cache->ops->invalidate_object,
- NULL, NULL);
- op->flags = FSCACHE_OP_ASYNC |
- (1 << FSCACHE_OP_EXCLUSIVE) |
- (1 << FSCACHE_OP_UNUSE_COOKIE);
-
- spin_lock(&cookie->lock);
- if (fscache_submit_exclusive_op(object, op) < 0)
- goto submit_op_failed;
- spin_unlock(&cookie->lock);
- fscache_put_operation(op);
-
- /* Once we've completed the invalidation, we know there will be no data
- * stored in the cache and thus we can reinstate the data-check-skip
- * optimisation.
- */
- set_bit(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
-
- /* We can allow read and write requests to come in once again. They'll
- * queue up behind our exclusive invalidation operation.
- */
- if (test_and_clear_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING);
- _leave(" [ok]");
- return transit_to(UPDATE_OBJECT);
-
-nomem:
- fscache_mark_object_dead(object);
- fscache_unuse_cookie(object);
- _leave(" [ENOMEM]");
- return transit_to(KILL_OBJECT);
-
-submit_op_failed:
- fscache_mark_object_dead(object);
- spin_unlock(&cookie->lock);
- fscache_unuse_cookie(object);
- kfree(op);
- _leave(" [EIO]");
- return transit_to(KILL_OBJECT);
-}
-
static const struct fscache_state *fscache_invalidate_object(struct fscache_object *object,
int event)
{
- const struct fscache_state *s;
-
- fscache_stat(&fscache_n_invalidates_run);
- fscache_stat(&fscache_n_cop_invalidate_object);
- s = _fscache_invalidate_object(object, event);
- fscache_stat_d(&fscache_n_cop_invalidate_object);
- return s;
+ return transit_to(UPDATE_OBJECT);
}
/*
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 627e1ab7123d..3c543f8ac663 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -171,7 +171,7 @@ struct fscache_cache_ops {
void (*update_object)(struct fscache_object *object);
/* Invalidate an object */
- void (*invalidate_object)(struct fscache_operation *op);
+ void (*invalidate_object)(struct fscache_object *object);
/* discard the resources pinned by an object and effect retirement if
* necessary */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 19/61] fscache: Remove the I/O operation manager
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the fscache I/O operation manager. Getting operation-operation
interactions and object-operation interactions correct has proven really
difficult; furthermore, the operations are being replaced with kiocb-driven
stuff on the cache front.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/namei.c | 5
fs/fscache/Makefile | 3
fs/fscache/cache.c | 3
fs/fscache/cookie.c | 1
fs/fscache/internal.h | 13 -
fs/fscache/object-list.c | 25 --
fs/fscache/object.c | 24 --
fs/fscache/operation.c | 633 ----------------------------------------
include/linux/fscache-cache.h | 71 ----
include/trace/events/fscache.h | 57 ----
10 files changed, 7 insertions(+), 828 deletions(-)
delete mode 100644 fs/fscache/operation.c
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 432002080b83..924042e8cced 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -36,9 +36,8 @@ void __cachefiles_printk_object(struct cachefiles_object *object,
prefix, object->fscache.state->name,
object->fscache.flags, work_busy(&object->fscache.work),
object->fscache.events, object->fscache.event_mask);
- pr_err("%sops=%u inp=%u exc=%u\n",
- prefix, object->fscache.n_ops, object->fscache.n_in_progress,
- object->fscache.n_exclusive);
+ pr_err("%sops=%u\n",
+ prefix, object->fscache.n_ops);
pr_err("%sparent=%p\n",
prefix, object->fscache.parent);
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 565a3441d31d..ac3fcd909fff 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -9,8 +9,7 @@ fscache-y := \
fsdef.o \
main.o \
netfs.o \
- object.o \
- operation.o
+ object.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index 1c91ecfb8360..bea53cd92a45 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -182,12 +182,9 @@ void fscache_init_cache(struct fscache_cache *cache,
vsnprintf(cache->identifier, sizeof(cache->identifier), idfmt, va);
va_end(va);
- INIT_WORK(&cache->op_gc, fscache_operation_gc);
INIT_LIST_HEAD(&cache->link);
INIT_LIST_HEAD(&cache->object_list);
- INIT_LIST_HEAD(&cache->op_gc_list);
spin_lock_init(&cache->object_list_lock);
- spin_lock_init(&cache->op_gc_list_lock);
}
EXPORT_SYMBOL(fscache_init_cache);
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index cd105227cb83..3b450c783f03 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -756,7 +756,6 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
if (invalidate)
set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
}
} else {
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 360137fd19a7..bc5539d2157b 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -138,19 +138,6 @@ extern void fscache_objlist_remove(struct fscache_object *);
#define fscache_objlist_remove(object) do {} while(0)
#endif
-/*
- * operation.c
- */
-extern int fscache_submit_exclusive_op(struct fscache_object *,
- struct fscache_operation *);
-extern int fscache_submit_op(struct fscache_object *,
- struct fscache_operation *);
-extern int fscache_cancel_op(struct fscache_operation *, bool);
-extern void fscache_cancel_all_ops(struct fscache_object *);
-extern void fscache_abort_object(struct fscache_object *);
-extern void fscache_start_operations(struct fscache_object *);
-extern void fscache_operation_gc(struct work_struct *);
-
/*
* proc.c
*/
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index fc28de4738ec..147c556ee01b 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -23,10 +23,6 @@ struct fscache_objlist_data {
#define FSCACHE_OBJLIST_CONFIG_AUX 0x00000002 /* show object auxdata */
#define FSCACHE_OBJLIST_CONFIG_BUSY 0x00000010 /* show busy objects */
#define FSCACHE_OBJLIST_CONFIG_IDLE 0x00000020 /* show idle objects */
-#define FSCACHE_OBJLIST_CONFIG_PENDWR 0x00000040 /* show objects with pending writes */
-#define FSCACHE_OBJLIST_CONFIG_NOPENDWR 0x00000080 /* show objects without pending writes */
-#define FSCACHE_OBJLIST_CONFIG_READS 0x00000100 /* show objects with active reads */
-#define FSCACHE_OBJLIST_CONFIG_NOREADS 0x00000200 /* show objects without active reads */
#define FSCACHE_OBJLIST_CONFIG_EVENTS 0x00000400 /* show objects with events */
#define FSCACHE_OBJLIST_CONFIG_NOEVENTS 0x00000800 /* show objects without no events */
#define FSCACHE_OBJLIST_CONFIG_WORK 0x00001000 /* show objects with work */
@@ -166,7 +162,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
u8 *p;
if ((unsigned long) v == 1) {
- seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
+ seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP"
" EM EV FL S"
" | COOKIE TYPE TY FL");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
@@ -185,7 +181,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
}
if ((unsigned long) v == 2) {
- seq_puts(m, "======== ======== ==== ===== === === === == ====="
+ seq_puts(m, "======== ======== ==== ===== === ==="
" == == == ="
" | ======== ======= == ===");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
@@ -217,26 +213,19 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
obj->flags ||
!list_empty(&obj->dependents),
BUSY, IDLE);
- FILTER(test_bit(FSCACHE_OBJECT_PENDING_WRITE, &obj->flags),
- PENDWR, NOPENDWR);
- FILTER(atomic_read(&obj->n_reads),
- READS, NOREADS);
FILTER(obj->events & obj->event_mask,
EVENTS, NOEVENTS);
FILTER(work_busy(&obj->work), WORK, NOWORK);
}
seq_printf(m,
- "%08x %08x %s %5u %3u %3u %3u %2u %5u %2lx %2lx %2lx %1x | ",
+ "%08x %08x %s %5u %3u %3u %2lx %2lx %2lx %1x | ",
obj->debug_id,
obj->parent ? obj->parent->debug_id : UINT_MAX,
obj->state->short_name,
obj->n_children,
obj->n_ops,
obj->n_obj_ops,
- obj->n_in_progress,
- obj->n_exclusive,
- atomic_read(&obj->n_reads),
obj->event_mask,
obj->events,
obj->flags,
@@ -336,10 +325,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
case 'A': config |= FSCACHE_OBJLIST_CONFIG_AUX; break;
case 'B': config |= FSCACHE_OBJLIST_CONFIG_BUSY; break;
case 'b': config |= FSCACHE_OBJLIST_CONFIG_IDLE; break;
- case 'W': config |= FSCACHE_OBJLIST_CONFIG_PENDWR; break;
- case 'w': config |= FSCACHE_OBJLIST_CONFIG_NOPENDWR; break;
- case 'R': config |= FSCACHE_OBJLIST_CONFIG_READS; break;
- case 'r': config |= FSCACHE_OBJLIST_CONFIG_NOREADS; break;
case 'S': config |= FSCACHE_OBJLIST_CONFIG_WORK; break;
case 's': config |= FSCACHE_OBJLIST_CONFIG_NOWORK; break;
}
@@ -350,10 +335,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
if (!(config & (FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE)))
config |= FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR)))
- config |= FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_READS | FSCACHE_OBJLIST_CONFIG_NOREADS)))
- config |= FSCACHE_OBJLIST_CONFIG_READS | FSCACHE_OBJLIST_CONFIG_NOREADS;
if (!(config & (FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS)))
config |= FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS;
if (!(config & (FSCACHE_OBJLIST_CONFIG_WORK | FSCACHE_OBJLIST_CONFIG_NOWORK)))
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index 73231fdf565c..8cb21dd4b1cc 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -315,9 +315,8 @@ void fscache_object_init(struct fscache_object *object,
INIT_WORK(&object->work, fscache_object_work_func);
INIT_LIST_HEAD(&object->dependents);
INIT_LIST_HEAD(&object->dep_link);
- INIT_LIST_HEAD(&object->pending_ops);
object->n_children = 0;
- object->n_ops = object->n_in_progress = object->n_exclusive = 0;
+ object->n_ops = 0;
object->events = 0;
object->cache = cache;
object->cookie = cookie;
@@ -580,14 +579,6 @@ static const struct fscache_state *fscache_object_available(struct fscache_objec
spin_lock(&object->lock);
fscache_done_parent_op(object);
- if (object->n_in_progress == 0) {
- if (object->n_ops > 0) {
- ASSERTCMP(object->n_ops, >=, object->n_obj_ops);
- fscache_start_operations(object);
- } else {
- ASSERT(list_empty(&object->pending_ops));
- }
- }
spin_unlock(&object->lock);
fscache_stat(&fscache_n_cop_lookup_complete);
@@ -654,24 +645,11 @@ static const struct fscache_state *fscache_kill_object(struct fscache_object *ob
fscache_mark_object_dead(object);
object->oob_event_mask = 0;
- if (test_bit(FSCACHE_OBJECT_RETIRED, &object->flags)) {
- /* Reject any new read/write ops and abort any that are pending. */
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- fscache_cancel_all_ops(object);
- }
-
if (list_empty(&object->dependents) &&
object->n_ops == 0 &&
object->n_children == 0)
return transit_to(DROP_OBJECT);
- if (object->n_in_progress == 0) {
- spin_lock(&object->lock);
- if (object->n_ops > 0 && object->n_in_progress == 0)
- fscache_start_operations(object);
- spin_unlock(&object->lock);
- }
-
if (!list_empty(&object->dependents))
return transit_to(KILL_DEPENDENTS);
diff --git a/fs/fscache/operation.c b/fs/fscache/operation.c
deleted file mode 100644
index 1a22a55f75a0..000000000000
--- a/fs/fscache/operation.c
+++ /dev/null
@@ -1,633 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* FS-Cache worker operation management routines
- *
- * Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * See Documentation/filesystems/caching/operations.txt
- */
-
-#define FSCACHE_DEBUG_LEVEL OPERATION
-#include <linux/module.h>
-#include <linux/seq_file.h>
-#include <linux/slab.h>
-#include "internal.h"
-
-atomic_t fscache_op_debug_id;
-EXPORT_SYMBOL(fscache_op_debug_id);
-
-static void fscache_operation_dummy_cancel(struct fscache_operation *op)
-{
-}
-
-/**
- * fscache_operation_init - Do basic initialisation of an operation
- * @op: The operation to initialise
- * @release: The release function to assign
- *
- * Do basic initialisation of an operation. The caller must still set flags,
- * object and processor if needed.
- */
-void fscache_operation_init(struct fscache_cookie *cookie,
- struct fscache_operation *op,
- fscache_operation_processor_t processor,
- fscache_operation_cancel_t cancel,
- fscache_operation_release_t release)
-{
- INIT_WORK(&op->work, fscache_op_work_func);
- atomic_set(&op->usage, 1);
- op->state = FSCACHE_OP_ST_INITIALISED;
- op->debug_id = atomic_inc_return(&fscache_op_debug_id);
- op->processor = processor;
- op->cancel = cancel ?: fscache_operation_dummy_cancel;
- op->release = release;
- INIT_LIST_HEAD(&op->pend_link);
- fscache_stat(&fscache_n_op_initialised);
- trace_fscache_op(cookie, op, fscache_op_init);
-}
-EXPORT_SYMBOL(fscache_operation_init);
-
-/**
- * fscache_enqueue_operation - Enqueue an operation for processing
- * @op: The operation to enqueue
- *
- * Enqueue an operation for processing by the FS-Cache thread pool.
- *
- * This will get its own ref on the object.
- */
-void fscache_enqueue_operation(struct fscache_operation *op)
-{
- struct fscache_cookie *cookie = op->object->cookie;
-
- _enter("{OBJ%x OP%x,%u}",
- op->object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- ASSERT(list_empty(&op->pend_link));
- ASSERT(op->processor != NULL);
- ASSERT(fscache_object_is_available(op->object));
- ASSERTCMP(atomic_read(&op->usage), >, 0);
- ASSERTIFCMP(op->state != FSCACHE_OP_ST_IN_PROGRESS,
- op->state, ==, FSCACHE_OP_ST_CANCELLED);
-
- fscache_stat(&fscache_n_op_enqueue);
- switch (op->flags & FSCACHE_OP_TYPE) {
- case FSCACHE_OP_ASYNC:
- trace_fscache_op(cookie, op, fscache_op_enqueue_async);
- _debug("queue async");
- atomic_inc(&op->usage);
- if (!queue_work(fscache_op_wq, &op->work))
- fscache_put_operation(op);
- break;
- case FSCACHE_OP_MYTHREAD:
- trace_fscache_op(cookie, op, fscache_op_enqueue_mythread);
- _debug("queue for caller's attention");
- break;
- default:
- pr_err("Unexpected op type %lx", op->flags);
- BUG();
- break;
- }
-}
-EXPORT_SYMBOL(fscache_enqueue_operation);
-
-/*
- * start an op running
- */
-static void fscache_run_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
-
- op->state = FSCACHE_OP_ST_IN_PROGRESS;
- object->n_in_progress++;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- if (op->processor)
- fscache_enqueue_operation(op);
- else
- trace_fscache_op(object->cookie, op, fscache_op_run);
- fscache_stat(&fscache_n_op_run);
-}
-
-/*
- * report an unexpected submission
- */
-static void fscache_report_unexpected_submission(struct fscache_object *object,
- struct fscache_operation *op,
- const struct fscache_state *ostate)
-{
- static bool once_only;
- struct fscache_operation *p;
- unsigned n;
-
- if (once_only)
- return;
- once_only = true;
-
- kdebug("unexpected submission OP%x [OBJ%x %s]",
- op->debug_id, object->debug_id, object->state->name);
- kdebug("objstate=%s [%s]", object->state->name, ostate->name);
- kdebug("objflags=%lx", object->flags);
- kdebug("objevent=%lx [%lx]", object->events, object->event_mask);
- kdebug("ops=%u inp=%u exc=%u",
- object->n_ops, object->n_in_progress, object->n_exclusive);
-
- if (!list_empty(&object->pending_ops)) {
- n = 0;
- list_for_each_entry(p, &object->pending_ops, pend_link) {
- ASSERTCMP(p->object, ==, object);
- kdebug("%p %p", op->processor, op->release);
- n++;
- }
-
- kdebug("n=%u", n);
- }
-
- dump_stack();
-}
-
-/*
- * submit an exclusive operation for an object
- * - other ops are excluded from running simultaneously with this one
- * - this gets any extra refs it needs on an op
- */
-int fscache_submit_exclusive_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- const struct fscache_state *ostate;
- unsigned long flags;
- int ret;
-
- _enter("{OBJ%x OP%x},", object->debug_id, op->debug_id);
-
- trace_fscache_op(object->cookie, op, fscache_op_submit_ex);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
- ASSERTCMP(object->n_ops, >=, object->n_in_progress);
- ASSERTCMP(object->n_ops, >=, object->n_exclusive);
- ASSERT(list_empty(&op->pend_link));
-
- ostate = object->state;
- smp_rmb();
-
- op->state = FSCACHE_OP_ST_PENDING;
- flags = READ_ONCE(object->flags);
- if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
- fscache_stat(&fscache_n_op_rejected);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else if (unlikely(fscache_cache_is_broken(object))) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -EIO;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
- op->object = object;
- object->n_ops++;
- object->n_exclusive++; /* reads and writes must wait */
-
- if (object->n_in_progress > 0) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- } else if (!list_empty(&object->pending_ops)) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- fscache_start_operations(object);
- } else {
- ASSERTCMP(object->n_in_progress, ==, 0);
- fscache_run_op(object, op);
- }
-
- /* need to issue a new write op after this */
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
- op->object = object;
- object->n_ops++;
- object->n_exclusive++; /* reads and writes must wait */
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else {
- fscache_report_unexpected_submission(object, op, ostate);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- }
-
- spin_unlock(&object->lock);
- return ret;
-}
-
-/*
- * submit an operation for an object
- * - objects may be submitted only in the following states:
- * - during object creation (write ops may be submitted)
- * - whilst the object is active
- * - after an I/O error incurred in one of the two above states (op rejected)
- * - this gets any extra refs it needs on an op
- */
-int fscache_submit_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- const struct fscache_state *ostate;
- unsigned long flags;
- int ret;
-
- _enter("{OBJ%x OP%x},{%u}",
- object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- trace_fscache_op(object->cookie, op, fscache_op_submit);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
- ASSERTCMP(object->n_ops, >=, object->n_in_progress);
- ASSERTCMP(object->n_ops, >=, object->n_exclusive);
- ASSERT(list_empty(&op->pend_link));
-
- ostate = object->state;
- smp_rmb();
-
- op->state = FSCACHE_OP_ST_PENDING;
- flags = READ_ONCE(object->flags);
- if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
- fscache_stat(&fscache_n_op_rejected);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else if (unlikely(fscache_cache_is_broken(object))) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -EIO;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
- op->object = object;
- object->n_ops++;
-
- if (object->n_exclusive > 0) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- } else if (!list_empty(&object->pending_ops)) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- fscache_start_operations(object);
- } else {
- ASSERTCMP(object->n_exclusive, ==, 0);
- fscache_run_op(object, op);
- }
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
- op->object = object;
- object->n_ops++;
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else {
- fscache_report_unexpected_submission(object, op, ostate);
- ASSERT(!fscache_object_is_active(object));
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- }
-
- spin_unlock(&object->lock);
- return ret;
-}
-
-/*
- * queue an object for withdrawal on error, aborting all following asynchronous
- * operations
- */
-void fscache_abort_object(struct fscache_object *object)
-{
- _enter("{OBJ%x}", object->debug_id);
-
- fscache_raise_event(object, FSCACHE_OBJECT_EV_ERROR);
-}
-
-/*
- * Jump start the operation processing on an object. The caller must hold
- * object->lock.
- */
-void fscache_start_operations(struct fscache_object *object)
-{
- struct fscache_operation *op;
- bool stop = false;
-
- while (!list_empty(&object->pending_ops) && !stop) {
- op = list_entry(object->pending_ops.next,
- struct fscache_operation, pend_link);
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags)) {
- if (object->n_in_progress > 0)
- break;
- stop = true;
- }
- list_del_init(&op->pend_link);
- fscache_run_op(object, op);
-
- /* the pending queue was holding a ref on the object */
- fscache_put_operation(op);
- }
-
- ASSERTCMP(object->n_in_progress, <=, object->n_ops);
-
- _debug("woke %d ops on OBJ%x",
- object->n_in_progress, object->debug_id);
-}
-
-/*
- * cancel an operation that's pending on an object
- */
-int fscache_cancel_op(struct fscache_operation *op,
- bool cancel_in_progress_op)
-{
- struct fscache_object *object = op->object;
- bool put = false;
- int ret;
-
- _enter("OBJ%x OP%x}", op->object->debug_id, op->debug_id);
-
- trace_fscache_op(object->cookie, op, fscache_op_cancel);
-
- ASSERTCMP(op->state, >=, FSCACHE_OP_ST_PENDING);
- ASSERTCMP(op->state, !=, FSCACHE_OP_ST_CANCELLED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
-
- ret = -EBUSY;
- if (op->state == FSCACHE_OP_ST_PENDING) {
- ASSERT(!list_empty(&op->pend_link));
- list_del_init(&op->pend_link);
- put = true;
-
- fscache_stat(&fscache_n_op_cancelled);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- ret = 0;
- } else if (op->state == FSCACHE_OP_ST_IN_PROGRESS && cancel_in_progress_op) {
- ASSERTCMP(object->n_in_progress, >, 0);
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- object->n_in_progress--;
- if (object->n_in_progress == 0)
- fscache_start_operations(object);
-
- fscache_stat(&fscache_n_op_cancelled);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- ret = 0;
- }
-
- if (put)
- fscache_put_operation(op);
- spin_unlock(&object->lock);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * Cancel all pending operations on an object
- */
-void fscache_cancel_all_ops(struct fscache_object *object)
-{
- struct fscache_operation *op;
-
- _enter("OBJ%x", object->debug_id);
-
- spin_lock(&object->lock);
-
- while (!list_empty(&object->pending_ops)) {
- op = list_entry(object->pending_ops.next,
- struct fscache_operation, pend_link);
- fscache_stat(&fscache_n_op_cancelled);
- list_del_init(&op->pend_link);
-
- trace_fscache_op(object->cookie, op, fscache_op_cancel_all);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- fscache_put_operation(op);
- cond_resched_lock(&object->lock);
- }
-
- spin_unlock(&object->lock);
- _leave("");
-}
-
-/*
- * Record the completion or cancellation of an in-progress operation.
- */
-void fscache_op_complete(struct fscache_operation *op, bool cancelled)
-{
- struct fscache_object *object = op->object;
-
- _enter("OBJ%x", object->debug_id);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_IN_PROGRESS);
- ASSERTCMP(object->n_in_progress, >, 0);
- ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
- object->n_exclusive, >, 0);
- ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
- object->n_in_progress, ==, 1);
-
- spin_lock(&object->lock);
-
- if (!cancelled) {
- trace_fscache_op(object->cookie, op, fscache_op_completed);
- op->state = FSCACHE_OP_ST_COMPLETE;
- } else {
- op->cancel(op);
- trace_fscache_op(object->cookie, op, fscache_op_cancelled);
- op->state = FSCACHE_OP_ST_CANCELLED;
- }
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- object->n_in_progress--;
- if (object->n_in_progress == 0)
- fscache_start_operations(object);
-
- spin_unlock(&object->lock);
- _leave("");
-}
-EXPORT_SYMBOL(fscache_op_complete);
-
-/*
- * release an operation
- * - queues pending ops if this is the last in-progress op
- */
-void fscache_put_operation(struct fscache_operation *op)
-{
- struct fscache_object *object;
- struct fscache_cache *cache;
-
- _enter("{OBJ%x OP%x,%d}",
- op->object ? op->object->debug_id : 0,
- op->debug_id, atomic_read(&op->usage));
-
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- if (!atomic_dec_and_test(&op->usage))
- return;
-
- trace_fscache_op(op->object ? op->object->cookie : NULL, op, fscache_op_put);
-
- _debug("PUT OP");
- ASSERTIFCMP(op->state != FSCACHE_OP_ST_INITIALISED &&
- op->state != FSCACHE_OP_ST_COMPLETE,
- op->state, ==, FSCACHE_OP_ST_CANCELLED);
-
- fscache_stat(&fscache_n_op_release);
-
- if (op->release) {
- op->release(op);
- op->release = NULL;
- }
- op->state = FSCACHE_OP_ST_DEAD;
-
- object = op->object;
- if (likely(object)) {
- if (test_bit(FSCACHE_OP_DEC_READ_CNT, &op->flags))
- atomic_dec(&object->n_reads);
- if (test_bit(FSCACHE_OP_UNUSE_COOKIE, &op->flags))
- fscache_unuse_cookie(object);
-
- /* now... we may get called with the object spinlock held, so we
- * complete the cleanup here only if we can immediately acquire the
- * lock, and defer it otherwise */
- if (!spin_trylock(&object->lock)) {
- _debug("defer put");
- fscache_stat(&fscache_n_op_deferred_release);
-
- cache = object->cache;
- spin_lock(&cache->op_gc_list_lock);
- list_add_tail(&op->pend_link, &cache->op_gc_list);
- spin_unlock(&cache->op_gc_list_lock);
- schedule_work(&cache->op_gc);
- _leave(" [defer]");
- return;
- }
-
- ASSERTCMP(object->n_ops, >, 0);
- object->n_ops--;
- if (object->n_ops == 0)
- fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
-
- spin_unlock(&object->lock);
- }
-
- kfree(op);
- _leave(" [done]");
-}
-EXPORT_SYMBOL(fscache_put_operation);
-
-/*
- * garbage collect operations that have had their release deferred
- */
-void fscache_operation_gc(struct work_struct *work)
-{
- struct fscache_operation *op;
- struct fscache_object *object;
- struct fscache_cache *cache =
- container_of(work, struct fscache_cache, op_gc);
- int count = 0;
-
- _enter("");
-
- do {
- spin_lock(&cache->op_gc_list_lock);
- if (list_empty(&cache->op_gc_list)) {
- spin_unlock(&cache->op_gc_list_lock);
- break;
- }
-
- op = list_entry(cache->op_gc_list.next,
- struct fscache_operation, pend_link);
- list_del(&op->pend_link);
- spin_unlock(&cache->op_gc_list_lock);
-
- object = op->object;
- trace_fscache_op(object->cookie, op, fscache_op_gc);
-
- spin_lock(&object->lock);
-
- _debug("GC DEFERRED REL OBJ%x OP%x",
- object->debug_id, op->debug_id);
- fscache_stat(&fscache_n_op_gc);
-
- ASSERTCMP(atomic_read(&op->usage), ==, 0);
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_DEAD);
-
- ASSERTCMP(object->n_ops, >, 0);
- object->n_ops--;
- if (object->n_ops == 0)
- fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
-
- spin_unlock(&object->lock);
- kfree(op);
-
- } while (count++ < 20);
-
- if (!list_empty(&cache->op_gc_list))
- schedule_work(&cache->op_gc);
-
- _leave("");
-}
-
-/*
- * execute an operation using fs_op_wq to provide processing context -
- * the caller holds a ref to this object, so we don't need to hold one
- */
-void fscache_op_work_func(struct work_struct *work)
-{
- struct fscache_operation *op =
- container_of(work, struct fscache_operation, work);
- unsigned long start;
-
- _enter("{OBJ%x OP%x,%d}",
- op->object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- trace_fscache_op(op->object->cookie, op, fscache_op_work);
-
- ASSERT(op->processor != NULL);
- start = jiffies;
- op->processor(op);
- fscache_hist(fscache_ops_histogram, start);
- fscache_put_operation(op);
-
- _leave("");
-}
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 3c543f8ac663..5e63d4a7feb0 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -23,7 +23,6 @@
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
-struct fscache_operation;
enum fscache_obj_ref_trace {
fscache_obj_get_add_to_deps,
@@ -62,11 +61,8 @@ struct fscache_cache {
char identifier[36]; /* cache label */
/* node management */
- struct work_struct op_gc; /* operation garbage collector */
struct list_head object_list; /* list of data/index objects */
- struct list_head op_gc_list; /* list of ops to be deleted */
spinlock_t object_list_lock;
- spinlock_t op_gc_list_lock;
atomic_t object_count; /* no. of live objects in this cache */
struct fscache_object *fsdef; /* object for the fsdef index */
unsigned long flags;
@@ -76,68 +72,6 @@ struct fscache_cache {
extern wait_queue_head_t fscache_cache_cleared_wq;
-/*
- * operation to be applied to a cache object
- * - retrieval initiation operations are done in the context of the process
- * that issued them, and not in an async thread pool
- */
-typedef void (*fscache_operation_release_t)(struct fscache_operation *op);
-typedef void (*fscache_operation_processor_t)(struct fscache_operation *op);
-typedef void (*fscache_operation_cancel_t)(struct fscache_operation *op);
-
-enum fscache_operation_state {
- FSCACHE_OP_ST_BLANK, /* Op is not yet submitted */
- FSCACHE_OP_ST_INITIALISED, /* Op is initialised */
- FSCACHE_OP_ST_PENDING, /* Op is blocked from running */
- FSCACHE_OP_ST_IN_PROGRESS, /* Op is in progress */
- FSCACHE_OP_ST_COMPLETE, /* Op is complete */
- FSCACHE_OP_ST_CANCELLED, /* Op has been cancelled */
- FSCACHE_OP_ST_DEAD /* Op is now dead */
-};
-
-struct fscache_operation {
- struct work_struct work; /* record for async ops */
- struct list_head pend_link; /* link in object->pending_ops */
- struct fscache_object *object; /* object to be operated upon */
-
- unsigned long flags;
-#define FSCACHE_OP_TYPE 0x000f /* operation type */
-#define FSCACHE_OP_ASYNC 0x0001 /* - async op, processor may sleep for disk */
-#define FSCACHE_OP_MYTHREAD 0x0002 /* - processing is done be issuing thread, not pool */
-#define FSCACHE_OP_WAITING 4 /* cleared when op is woken */
-#define FSCACHE_OP_EXCLUSIVE 5 /* exclusive op, other ops must wait */
-#define FSCACHE_OP_DEC_READ_CNT 6 /* decrement object->n_reads on destruction */
-#define FSCACHE_OP_UNUSE_COOKIE 7 /* call fscache_unuse_cookie() on completion */
-#define FSCACHE_OP_KEEP_FLAGS 0x00f0 /* flags to keep when repurposing an op */
-
- enum fscache_operation_state state;
- atomic_t usage;
- unsigned debug_id; /* debugging ID */
-
- /* operation processor callback
- * - can be NULL if FSCACHE_OP_WAITING is going to be used to perform
- * the op in a non-pool thread */
- fscache_operation_processor_t processor;
-
- /* Operation cancellation cleanup (optional) */
- fscache_operation_cancel_t cancel;
-
- /* operation releaser */
- fscache_operation_release_t release;
-};
-
-extern atomic_t fscache_op_debug_id;
-extern void fscache_op_work_func(struct work_struct *work);
-
-extern void fscache_enqueue_operation(struct fscache_operation *);
-extern void fscache_op_complete(struct fscache_operation *, bool);
-extern void fscache_put_operation(struct fscache_operation *);
-extern void fscache_operation_init(struct fscache_cookie *,
- struct fscache_operation *,
- fscache_operation_processor_t,
- fscache_operation_cancel_t,
- fscache_operation_release_t);
-
/*
* cache operations
*/
@@ -232,9 +166,6 @@ struct fscache_object {
int n_children; /* number of child objects */
int n_ops; /* number of extant ops on object */
int n_obj_ops; /* number of object ops outstanding on object */
- int n_in_progress; /* number of ops in progress */
- int n_exclusive; /* number of exclusive ops queued or in progress */
- atomic_t n_reads; /* number of read ops in progress */
spinlock_t lock; /* state and operations lock */
unsigned long lookup_jif; /* time at which lookup started */
@@ -245,7 +176,6 @@ struct fscache_object {
unsigned long flags;
#define FSCACHE_OBJECT_LOCK 0 /* T if object is busy being processed */
-#define FSCACHE_OBJECT_PENDING_WRITE 1 /* T if object has pending write */
#define FSCACHE_OBJECT_WAITING 2 /* T if object is waiting on its parent */
#define FSCACHE_OBJECT_IS_LIVE 3 /* T if object is not withdrawn or relinquished */
#define FSCACHE_OBJECT_IS_LOOKED_UP 4 /* T if object has been looked up */
@@ -262,7 +192,6 @@ struct fscache_object {
struct work_struct work; /* attention scheduling record */
struct list_head dependents; /* FIFO of dependent objects */
struct list_head dep_link; /* link in parent's dependents list */
- struct list_head pending_ops; /* unstarted operations on this object */
#ifdef CONFIG_FSCACHE_OBJECT_LIST
struct rb_node objlist_link; /* link in global object list */
#endif
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 2ebfd688a7c2..08d7de72409d 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -33,24 +33,6 @@ enum fscache_cookie_trace {
fscache_cookie_put_parent,
};
-enum fscache_op_trace {
- fscache_op_cancel,
- fscache_op_cancel_all,
- fscache_op_cancelled,
- fscache_op_completed,
- fscache_op_enqueue_async,
- fscache_op_enqueue_mythread,
- fscache_op_gc,
- fscache_op_init,
- fscache_op_put,
- fscache_op_run,
- fscache_op_signal,
- fscache_op_submit,
- fscache_op_submit_ex,
- fscache_op_work,
- fscache_op_trace__nr
-};
-
#endif
/*
@@ -69,22 +51,6 @@ enum fscache_op_trace {
EM(fscache_cookie_put_object, "PUT obj") \
E_(fscache_cookie_put_parent, "PUT prn")
-#define fscache_op_traces \
- EM(fscache_op_cancel, "Cancel1") \
- EM(fscache_op_cancel_all, "CancelA") \
- EM(fscache_op_cancelled, "Canclld") \
- EM(fscache_op_completed, "Complet") \
- EM(fscache_op_enqueue_async, "EnqAsyn") \
- EM(fscache_op_enqueue_mythread, "EnqMyTh") \
- EM(fscache_op_gc, "GC ") \
- EM(fscache_op_init, "Init ") \
- EM(fscache_op_put, "Put ") \
- EM(fscache_op_run, "Run ") \
- EM(fscache_op_signal, "Signal ") \
- EM(fscache_op_submit, "Submit ") \
- EM(fscache_op_submit_ex, "SubmitX") \
- E_(fscache_op_work, "Work ")
-
/*
* Export enum symbols via userspace.
*/
@@ -309,29 +275,6 @@ TRACE_EVENT(fscache_osm,
__entry->event_num)
);
-TRACE_EVENT(fscache_op,
- TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
- enum fscache_op_trace why),
-
- TP_ARGS(cookie, op, why),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, op )
- __field(enum fscache_op_trace, why )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->op = op->debug_id;
- __entry->why = why;
- ),
-
- TP_printk("c=%08x op=%08x %s",
- __entry->cookie, __entry->op,
- __print_symbolic(__entry->why, fscache_op_traces))
- );
-
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 19/61] fscache: Remove the I/O operation manager
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove the fscache I/O operation manager. Getting operation-operation
interactions and object-operation interactions correct has proven really
difficult; furthermore, the operations are being replaced with kiocb-driven
stuff on the cache front.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/namei.c | 5
fs/fscache/Makefile | 3
fs/fscache/cache.c | 3
fs/fscache/cookie.c | 1
fs/fscache/internal.h | 13 -
fs/fscache/object-list.c | 25 --
fs/fscache/object.c | 24 --
fs/fscache/operation.c | 633 ----------------------------------------
include/linux/fscache-cache.h | 71 ----
include/trace/events/fscache.h | 57 ----
10 files changed, 7 insertions(+), 828 deletions(-)
delete mode 100644 fs/fscache/operation.c
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 432002080b83..924042e8cced 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -36,9 +36,8 @@ void __cachefiles_printk_object(struct cachefiles_object *object,
prefix, object->fscache.state->name,
object->fscache.flags, work_busy(&object->fscache.work),
object->fscache.events, object->fscache.event_mask);
- pr_err("%sops=%u inp=%u exc=%u\n",
- prefix, object->fscache.n_ops, object->fscache.n_in_progress,
- object->fscache.n_exclusive);
+ pr_err("%sops=%u\n",
+ prefix, object->fscache.n_ops);
pr_err("%sparent=%p\n",
prefix, object->fscache.parent);
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 565a3441d31d..ac3fcd909fff 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -9,8 +9,7 @@ fscache-y := \
fsdef.o \
main.o \
netfs.o \
- object.o \
- operation.o
+ object.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index 1c91ecfb8360..bea53cd92a45 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -182,12 +182,9 @@ void fscache_init_cache(struct fscache_cache *cache,
vsnprintf(cache->identifier, sizeof(cache->identifier), idfmt, va);
va_end(va);
- INIT_WORK(&cache->op_gc, fscache_operation_gc);
INIT_LIST_HEAD(&cache->link);
INIT_LIST_HEAD(&cache->object_list);
- INIT_LIST_HEAD(&cache->op_gc_list);
spin_lock_init(&cache->object_list_lock);
- spin_lock_init(&cache->op_gc_list_lock);
}
EXPORT_SYMBOL(fscache_init_cache);
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index cd105227cb83..3b450c783f03 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -756,7 +756,6 @@ void __fscache_disable_cookie(struct fscache_cookie *cookie,
hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
if (invalidate)
set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
}
} else {
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 360137fd19a7..bc5539d2157b 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -138,19 +138,6 @@ extern void fscache_objlist_remove(struct fscache_object *);
#define fscache_objlist_remove(object) do {} while(0)
#endif
-/*
- * operation.c
- */
-extern int fscache_submit_exclusive_op(struct fscache_object *,
- struct fscache_operation *);
-extern int fscache_submit_op(struct fscache_object *,
- struct fscache_operation *);
-extern int fscache_cancel_op(struct fscache_operation *, bool);
-extern void fscache_cancel_all_ops(struct fscache_object *);
-extern void fscache_abort_object(struct fscache_object *);
-extern void fscache_start_operations(struct fscache_object *);
-extern void fscache_operation_gc(struct work_struct *);
-
/*
* proc.c
*/
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index fc28de4738ec..147c556ee01b 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -23,10 +23,6 @@ struct fscache_objlist_data {
#define FSCACHE_OBJLIST_CONFIG_AUX 0x00000002 /* show object auxdata */
#define FSCACHE_OBJLIST_CONFIG_BUSY 0x00000010 /* show busy objects */
#define FSCACHE_OBJLIST_CONFIG_IDLE 0x00000020 /* show idle objects */
-#define FSCACHE_OBJLIST_CONFIG_PENDWR 0x00000040 /* show objects with pending writes */
-#define FSCACHE_OBJLIST_CONFIG_NOPENDWR 0x00000080 /* show objects without pending writes */
-#define FSCACHE_OBJLIST_CONFIG_READS 0x00000100 /* show objects with active reads */
-#define FSCACHE_OBJLIST_CONFIG_NOREADS 0x00000200 /* show objects without active reads */
#define FSCACHE_OBJLIST_CONFIG_EVENTS 0x00000400 /* show objects with events */
#define FSCACHE_OBJLIST_CONFIG_NOEVENTS 0x00000800 /* show objects without no events */
#define FSCACHE_OBJLIST_CONFIG_WORK 0x00001000 /* show objects with work */
@@ -166,7 +162,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
u8 *p;
if ((unsigned long) v == 1) {
- seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP IPR EX READS"
+ seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP"
" EM EV FL S"
" | COOKIE TYPE TY FL");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
@@ -185,7 +181,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
}
if ((unsigned long) v == 2) {
- seq_puts(m, "======== ======== ==== ===== === === === == ====="
+ seq_puts(m, "======== ======== ==== ===== === ==="
" == == == ="
" | ======== ======= == ===");
if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
@@ -217,26 +213,19 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
obj->flags ||
!list_empty(&obj->dependents),
BUSY, IDLE);
- FILTER(test_bit(FSCACHE_OBJECT_PENDING_WRITE, &obj->flags),
- PENDWR, NOPENDWR);
- FILTER(atomic_read(&obj->n_reads),
- READS, NOREADS);
FILTER(obj->events & obj->event_mask,
EVENTS, NOEVENTS);
FILTER(work_busy(&obj->work), WORK, NOWORK);
}
seq_printf(m,
- "%08x %08x %s %5u %3u %3u %3u %2u %5u %2lx %2lx %2lx %1x | ",
+ "%08x %08x %s %5u %3u %3u %2lx %2lx %2lx %1x | ",
obj->debug_id,
obj->parent ? obj->parent->debug_id : UINT_MAX,
obj->state->short_name,
obj->n_children,
obj->n_ops,
obj->n_obj_ops,
- obj->n_in_progress,
- obj->n_exclusive,
- atomic_read(&obj->n_reads),
obj->event_mask,
obj->events,
obj->flags,
@@ -336,10 +325,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
case 'A': config |= FSCACHE_OBJLIST_CONFIG_AUX; break;
case 'B': config |= FSCACHE_OBJLIST_CONFIG_BUSY; break;
case 'b': config |= FSCACHE_OBJLIST_CONFIG_IDLE; break;
- case 'W': config |= FSCACHE_OBJLIST_CONFIG_PENDWR; break;
- case 'w': config |= FSCACHE_OBJLIST_CONFIG_NOPENDWR; break;
- case 'R': config |= FSCACHE_OBJLIST_CONFIG_READS; break;
- case 'r': config |= FSCACHE_OBJLIST_CONFIG_NOREADS; break;
case 'S': config |= FSCACHE_OBJLIST_CONFIG_WORK; break;
case 's': config |= FSCACHE_OBJLIST_CONFIG_NOWORK; break;
}
@@ -350,10 +335,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
if (!(config & (FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE)))
config |= FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR)))
- config |= FSCACHE_OBJLIST_CONFIG_PENDWR | FSCACHE_OBJLIST_CONFIG_NOPENDWR;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_READS | FSCACHE_OBJLIST_CONFIG_NOREADS)))
- config |= FSCACHE_OBJLIST_CONFIG_READS | FSCACHE_OBJLIST_CONFIG_NOREADS;
if (!(config & (FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS)))
config |= FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS;
if (!(config & (FSCACHE_OBJLIST_CONFIG_WORK | FSCACHE_OBJLIST_CONFIG_NOWORK)))
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
index 73231fdf565c..8cb21dd4b1cc 100644
--- a/fs/fscache/object.c
+++ b/fs/fscache/object.c
@@ -315,9 +315,8 @@ void fscache_object_init(struct fscache_object *object,
INIT_WORK(&object->work, fscache_object_work_func);
INIT_LIST_HEAD(&object->dependents);
INIT_LIST_HEAD(&object->dep_link);
- INIT_LIST_HEAD(&object->pending_ops);
object->n_children = 0;
- object->n_ops = object->n_in_progress = object->n_exclusive = 0;
+ object->n_ops = 0;
object->events = 0;
object->cache = cache;
object->cookie = cookie;
@@ -580,14 +579,6 @@ static const struct fscache_state *fscache_object_available(struct fscache_objec
spin_lock(&object->lock);
fscache_done_parent_op(object);
- if (object->n_in_progress == 0) {
- if (object->n_ops > 0) {
- ASSERTCMP(object->n_ops, >=, object->n_obj_ops);
- fscache_start_operations(object);
- } else {
- ASSERT(list_empty(&object->pending_ops));
- }
- }
spin_unlock(&object->lock);
fscache_stat(&fscache_n_cop_lookup_complete);
@@ -654,24 +645,11 @@ static const struct fscache_state *fscache_kill_object(struct fscache_object *ob
fscache_mark_object_dead(object);
object->oob_event_mask = 0;
- if (test_bit(FSCACHE_OBJECT_RETIRED, &object->flags)) {
- /* Reject any new read/write ops and abort any that are pending. */
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- fscache_cancel_all_ops(object);
- }
-
if (list_empty(&object->dependents) &&
object->n_ops == 0 &&
object->n_children == 0)
return transit_to(DROP_OBJECT);
- if (object->n_in_progress == 0) {
- spin_lock(&object->lock);
- if (object->n_ops > 0 && object->n_in_progress == 0)
- fscache_start_operations(object);
- spin_unlock(&object->lock);
- }
-
if (!list_empty(&object->dependents))
return transit_to(KILL_DEPENDENTS);
diff --git a/fs/fscache/operation.c b/fs/fscache/operation.c
deleted file mode 100644
index 1a22a55f75a0..000000000000
--- a/fs/fscache/operation.c
+++ /dev/null
@@ -1,633 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* FS-Cache worker operation management routines
- *
- * Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
- *
- * See Documentation/filesystems/caching/operations.txt
- */
-
-#define FSCACHE_DEBUG_LEVEL OPERATION
-#include <linux/module.h>
-#include <linux/seq_file.h>
-#include <linux/slab.h>
-#include "internal.h"
-
-atomic_t fscache_op_debug_id;
-EXPORT_SYMBOL(fscache_op_debug_id);
-
-static void fscache_operation_dummy_cancel(struct fscache_operation *op)
-{
-}
-
-/**
- * fscache_operation_init - Do basic initialisation of an operation
- * @op: The operation to initialise
- * @release: The release function to assign
- *
- * Do basic initialisation of an operation. The caller must still set flags,
- * object and processor if needed.
- */
-void fscache_operation_init(struct fscache_cookie *cookie,
- struct fscache_operation *op,
- fscache_operation_processor_t processor,
- fscache_operation_cancel_t cancel,
- fscache_operation_release_t release)
-{
- INIT_WORK(&op->work, fscache_op_work_func);
- atomic_set(&op->usage, 1);
- op->state = FSCACHE_OP_ST_INITIALISED;
- op->debug_id = atomic_inc_return(&fscache_op_debug_id);
- op->processor = processor;
- op->cancel = cancel ?: fscache_operation_dummy_cancel;
- op->release = release;
- INIT_LIST_HEAD(&op->pend_link);
- fscache_stat(&fscache_n_op_initialised);
- trace_fscache_op(cookie, op, fscache_op_init);
-}
-EXPORT_SYMBOL(fscache_operation_init);
-
-/**
- * fscache_enqueue_operation - Enqueue an operation for processing
- * @op: The operation to enqueue
- *
- * Enqueue an operation for processing by the FS-Cache thread pool.
- *
- * This will get its own ref on the object.
- */
-void fscache_enqueue_operation(struct fscache_operation *op)
-{
- struct fscache_cookie *cookie = op->object->cookie;
-
- _enter("{OBJ%x OP%x,%u}",
- op->object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- ASSERT(list_empty(&op->pend_link));
- ASSERT(op->processor != NULL);
- ASSERT(fscache_object_is_available(op->object));
- ASSERTCMP(atomic_read(&op->usage), >, 0);
- ASSERTIFCMP(op->state != FSCACHE_OP_ST_IN_PROGRESS,
- op->state, ==, FSCACHE_OP_ST_CANCELLED);
-
- fscache_stat(&fscache_n_op_enqueue);
- switch (op->flags & FSCACHE_OP_TYPE) {
- case FSCACHE_OP_ASYNC:
- trace_fscache_op(cookie, op, fscache_op_enqueue_async);
- _debug("queue async");
- atomic_inc(&op->usage);
- if (!queue_work(fscache_op_wq, &op->work))
- fscache_put_operation(op);
- break;
- case FSCACHE_OP_MYTHREAD:
- trace_fscache_op(cookie, op, fscache_op_enqueue_mythread);
- _debug("queue for caller's attention");
- break;
- default:
- pr_err("Unexpected op type %lx", op->flags);
- BUG();
- break;
- }
-}
-EXPORT_SYMBOL(fscache_enqueue_operation);
-
-/*
- * start an op running
- */
-static void fscache_run_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
-
- op->state = FSCACHE_OP_ST_IN_PROGRESS;
- object->n_in_progress++;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- if (op->processor)
- fscache_enqueue_operation(op);
- else
- trace_fscache_op(object->cookie, op, fscache_op_run);
- fscache_stat(&fscache_n_op_run);
-}
-
-/*
- * report an unexpected submission
- */
-static void fscache_report_unexpected_submission(struct fscache_object *object,
- struct fscache_operation *op,
- const struct fscache_state *ostate)
-{
- static bool once_only;
- struct fscache_operation *p;
- unsigned n;
-
- if (once_only)
- return;
- once_only = true;
-
- kdebug("unexpected submission OP%x [OBJ%x %s]",
- op->debug_id, object->debug_id, object->state->name);
- kdebug("objstate=%s [%s]", object->state->name, ostate->name);
- kdebug("objflags=%lx", object->flags);
- kdebug("objevent=%lx [%lx]", object->events, object->event_mask);
- kdebug("ops=%u inp=%u exc=%u",
- object->n_ops, object->n_in_progress, object->n_exclusive);
-
- if (!list_empty(&object->pending_ops)) {
- n = 0;
- list_for_each_entry(p, &object->pending_ops, pend_link) {
- ASSERTCMP(p->object, ==, object);
- kdebug("%p %p", op->processor, op->release);
- n++;
- }
-
- kdebug("n=%u", n);
- }
-
- dump_stack();
-}
-
-/*
- * submit an exclusive operation for an object
- * - other ops are excluded from running simultaneously with this one
- * - this gets any extra refs it needs on an op
- */
-int fscache_submit_exclusive_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- const struct fscache_state *ostate;
- unsigned long flags;
- int ret;
-
- _enter("{OBJ%x OP%x},", object->debug_id, op->debug_id);
-
- trace_fscache_op(object->cookie, op, fscache_op_submit_ex);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
- ASSERTCMP(object->n_ops, >=, object->n_in_progress);
- ASSERTCMP(object->n_ops, >=, object->n_exclusive);
- ASSERT(list_empty(&op->pend_link));
-
- ostate = object->state;
- smp_rmb();
-
- op->state = FSCACHE_OP_ST_PENDING;
- flags = READ_ONCE(object->flags);
- if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
- fscache_stat(&fscache_n_op_rejected);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else if (unlikely(fscache_cache_is_broken(object))) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -EIO;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
- op->object = object;
- object->n_ops++;
- object->n_exclusive++; /* reads and writes must wait */
-
- if (object->n_in_progress > 0) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- } else if (!list_empty(&object->pending_ops)) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- fscache_start_operations(object);
- } else {
- ASSERTCMP(object->n_in_progress, ==, 0);
- fscache_run_op(object, op);
- }
-
- /* need to issue a new write op after this */
- clear_bit(FSCACHE_OBJECT_PENDING_WRITE, &object->flags);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
- op->object = object;
- object->n_ops++;
- object->n_exclusive++; /* reads and writes must wait */
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else {
- fscache_report_unexpected_submission(object, op, ostate);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- }
-
- spin_unlock(&object->lock);
- return ret;
-}
-
-/*
- * submit an operation for an object
- * - objects may be submitted only in the following states:
- * - during object creation (write ops may be submitted)
- * - whilst the object is active
- * - after an I/O error incurred in one of the two above states (op rejected)
- * - this gets any extra refs it needs on an op
- */
-int fscache_submit_op(struct fscache_object *object,
- struct fscache_operation *op)
-{
- const struct fscache_state *ostate;
- unsigned long flags;
- int ret;
-
- _enter("{OBJ%x OP%x},{%u}",
- object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- trace_fscache_op(object->cookie, op, fscache_op_submit);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_INITIALISED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
- ASSERTCMP(object->n_ops, >=, object->n_in_progress);
- ASSERTCMP(object->n_ops, >=, object->n_exclusive);
- ASSERT(list_empty(&op->pend_link));
-
- ostate = object->state;
- smp_rmb();
-
- op->state = FSCACHE_OP_ST_PENDING;
- flags = READ_ONCE(object->flags);
- if (unlikely(!(flags & BIT(FSCACHE_OBJECT_IS_LIVE)))) {
- fscache_stat(&fscache_n_op_rejected);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else if (unlikely(fscache_cache_is_broken(object))) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -EIO;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_AVAILABLE)) {
- op->object = object;
- object->n_ops++;
-
- if (object->n_exclusive > 0) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- } else if (!list_empty(&object->pending_ops)) {
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- fscache_start_operations(object);
- } else {
- ASSERTCMP(object->n_exclusive, ==, 0);
- fscache_run_op(object, op);
- }
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_IS_LOOKED_UP)) {
- op->object = object;
- object->n_ops++;
- atomic_inc(&op->usage);
- list_add_tail(&op->pend_link, &object->pending_ops);
- fscache_stat(&fscache_n_op_pend);
- ret = 0;
- } else if (flags & BIT(FSCACHE_OBJECT_KILLED_BY_CACHE)) {
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- } else {
- fscache_report_unexpected_submission(object, op, ostate);
- ASSERT(!fscache_object_is_active(object));
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- ret = -ENOBUFS;
- }
-
- spin_unlock(&object->lock);
- return ret;
-}
-
-/*
- * queue an object for withdrawal on error, aborting all following asynchronous
- * operations
- */
-void fscache_abort_object(struct fscache_object *object)
-{
- _enter("{OBJ%x}", object->debug_id);
-
- fscache_raise_event(object, FSCACHE_OBJECT_EV_ERROR);
-}
-
-/*
- * Jump start the operation processing on an object. The caller must hold
- * object->lock.
- */
-void fscache_start_operations(struct fscache_object *object)
-{
- struct fscache_operation *op;
- bool stop = false;
-
- while (!list_empty(&object->pending_ops) && !stop) {
- op = list_entry(object->pending_ops.next,
- struct fscache_operation, pend_link);
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags)) {
- if (object->n_in_progress > 0)
- break;
- stop = true;
- }
- list_del_init(&op->pend_link);
- fscache_run_op(object, op);
-
- /* the pending queue was holding a ref on the object */
- fscache_put_operation(op);
- }
-
- ASSERTCMP(object->n_in_progress, <=, object->n_ops);
-
- _debug("woke %d ops on OBJ%x",
- object->n_in_progress, object->debug_id);
-}
-
-/*
- * cancel an operation that's pending on an object
- */
-int fscache_cancel_op(struct fscache_operation *op,
- bool cancel_in_progress_op)
-{
- struct fscache_object *object = op->object;
- bool put = false;
- int ret;
-
- _enter("OBJ%x OP%x}", op->object->debug_id, op->debug_id);
-
- trace_fscache_op(object->cookie, op, fscache_op_cancel);
-
- ASSERTCMP(op->state, >=, FSCACHE_OP_ST_PENDING);
- ASSERTCMP(op->state, !=, FSCACHE_OP_ST_CANCELLED);
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- spin_lock(&object->lock);
-
- ret = -EBUSY;
- if (op->state == FSCACHE_OP_ST_PENDING) {
- ASSERT(!list_empty(&op->pend_link));
- list_del_init(&op->pend_link);
- put = true;
-
- fscache_stat(&fscache_n_op_cancelled);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- ret = 0;
- } else if (op->state == FSCACHE_OP_ST_IN_PROGRESS && cancel_in_progress_op) {
- ASSERTCMP(object->n_in_progress, >, 0);
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- object->n_in_progress--;
- if (object->n_in_progress == 0)
- fscache_start_operations(object);
-
- fscache_stat(&fscache_n_op_cancelled);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- ret = 0;
- }
-
- if (put)
- fscache_put_operation(op);
- spin_unlock(&object->lock);
- _leave(" = %d", ret);
- return ret;
-}
-
-/*
- * Cancel all pending operations on an object
- */
-void fscache_cancel_all_ops(struct fscache_object *object)
-{
- struct fscache_operation *op;
-
- _enter("OBJ%x", object->debug_id);
-
- spin_lock(&object->lock);
-
- while (!list_empty(&object->pending_ops)) {
- op = list_entry(object->pending_ops.next,
- struct fscache_operation, pend_link);
- fscache_stat(&fscache_n_op_cancelled);
- list_del_init(&op->pend_link);
-
- trace_fscache_op(object->cookie, op, fscache_op_cancel_all);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_PENDING);
- op->cancel(op);
- op->state = FSCACHE_OP_ST_CANCELLED;
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- if (test_and_clear_bit(FSCACHE_OP_WAITING, &op->flags))
- wake_up_bit(&op->flags, FSCACHE_OP_WAITING);
- fscache_put_operation(op);
- cond_resched_lock(&object->lock);
- }
-
- spin_unlock(&object->lock);
- _leave("");
-}
-
-/*
- * Record the completion or cancellation of an in-progress operation.
- */
-void fscache_op_complete(struct fscache_operation *op, bool cancelled)
-{
- struct fscache_object *object = op->object;
-
- _enter("OBJ%x", object->debug_id);
-
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_IN_PROGRESS);
- ASSERTCMP(object->n_in_progress, >, 0);
- ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
- object->n_exclusive, >, 0);
- ASSERTIFCMP(test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags),
- object->n_in_progress, ==, 1);
-
- spin_lock(&object->lock);
-
- if (!cancelled) {
- trace_fscache_op(object->cookie, op, fscache_op_completed);
- op->state = FSCACHE_OP_ST_COMPLETE;
- } else {
- op->cancel(op);
- trace_fscache_op(object->cookie, op, fscache_op_cancelled);
- op->state = FSCACHE_OP_ST_CANCELLED;
- }
-
- if (test_bit(FSCACHE_OP_EXCLUSIVE, &op->flags))
- object->n_exclusive--;
- object->n_in_progress--;
- if (object->n_in_progress == 0)
- fscache_start_operations(object);
-
- spin_unlock(&object->lock);
- _leave("");
-}
-EXPORT_SYMBOL(fscache_op_complete);
-
-/*
- * release an operation
- * - queues pending ops if this is the last in-progress op
- */
-void fscache_put_operation(struct fscache_operation *op)
-{
- struct fscache_object *object;
- struct fscache_cache *cache;
-
- _enter("{OBJ%x OP%x,%d}",
- op->object ? op->object->debug_id : 0,
- op->debug_id, atomic_read(&op->usage));
-
- ASSERTCMP(atomic_read(&op->usage), >, 0);
-
- if (!atomic_dec_and_test(&op->usage))
- return;
-
- trace_fscache_op(op->object ? op->object->cookie : NULL, op, fscache_op_put);
-
- _debug("PUT OP");
- ASSERTIFCMP(op->state != FSCACHE_OP_ST_INITIALISED &&
- op->state != FSCACHE_OP_ST_COMPLETE,
- op->state, ==, FSCACHE_OP_ST_CANCELLED);
-
- fscache_stat(&fscache_n_op_release);
-
- if (op->release) {
- op->release(op);
- op->release = NULL;
- }
- op->state = FSCACHE_OP_ST_DEAD;
-
- object = op->object;
- if (likely(object)) {
- if (test_bit(FSCACHE_OP_DEC_READ_CNT, &op->flags))
- atomic_dec(&object->n_reads);
- if (test_bit(FSCACHE_OP_UNUSE_COOKIE, &op->flags))
- fscache_unuse_cookie(object);
-
- /* now... we may get called with the object spinlock held, so we
- * complete the cleanup here only if we can immediately acquire the
- * lock, and defer it otherwise */
- if (!spin_trylock(&object->lock)) {
- _debug("defer put");
- fscache_stat(&fscache_n_op_deferred_release);
-
- cache = object->cache;
- spin_lock(&cache->op_gc_list_lock);
- list_add_tail(&op->pend_link, &cache->op_gc_list);
- spin_unlock(&cache->op_gc_list_lock);
- schedule_work(&cache->op_gc);
- _leave(" [defer]");
- return;
- }
-
- ASSERTCMP(object->n_ops, >, 0);
- object->n_ops--;
- if (object->n_ops == 0)
- fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
-
- spin_unlock(&object->lock);
- }
-
- kfree(op);
- _leave(" [done]");
-}
-EXPORT_SYMBOL(fscache_put_operation);
-
-/*
- * garbage collect operations that have had their release deferred
- */
-void fscache_operation_gc(struct work_struct *work)
-{
- struct fscache_operation *op;
- struct fscache_object *object;
- struct fscache_cache *cache =
- container_of(work, struct fscache_cache, op_gc);
- int count = 0;
-
- _enter("");
-
- do {
- spin_lock(&cache->op_gc_list_lock);
- if (list_empty(&cache->op_gc_list)) {
- spin_unlock(&cache->op_gc_list_lock);
- break;
- }
-
- op = list_entry(cache->op_gc_list.next,
- struct fscache_operation, pend_link);
- list_del(&op->pend_link);
- spin_unlock(&cache->op_gc_list_lock);
-
- object = op->object;
- trace_fscache_op(object->cookie, op, fscache_op_gc);
-
- spin_lock(&object->lock);
-
- _debug("GC DEFERRED REL OBJ%x OP%x",
- object->debug_id, op->debug_id);
- fscache_stat(&fscache_n_op_gc);
-
- ASSERTCMP(atomic_read(&op->usage), ==, 0);
- ASSERTCMP(op->state, ==, FSCACHE_OP_ST_DEAD);
-
- ASSERTCMP(object->n_ops, >, 0);
- object->n_ops--;
- if (object->n_ops == 0)
- fscache_raise_event(object, FSCACHE_OBJECT_EV_CLEARED);
-
- spin_unlock(&object->lock);
- kfree(op);
-
- } while (count++ < 20);
-
- if (!list_empty(&cache->op_gc_list))
- schedule_work(&cache->op_gc);
-
- _leave("");
-}
-
-/*
- * execute an operation using fs_op_wq to provide processing context -
- * the caller holds a ref to this object, so we don't need to hold one
- */
-void fscache_op_work_func(struct work_struct *work)
-{
- struct fscache_operation *op =
- container_of(work, struct fscache_operation, work);
- unsigned long start;
-
- _enter("{OBJ%x OP%x,%d}",
- op->object->debug_id, op->debug_id, atomic_read(&op->usage));
-
- trace_fscache_op(op->object->cookie, op, fscache_op_work);
-
- ASSERT(op->processor != NULL);
- start = jiffies;
- op->processor(op);
- fscache_hist(fscache_ops_histogram, start);
- fscache_put_operation(op);
-
- _leave("");
-}
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 3c543f8ac663..5e63d4a7feb0 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -23,7 +23,6 @@
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
-struct fscache_operation;
enum fscache_obj_ref_trace {
fscache_obj_get_add_to_deps,
@@ -62,11 +61,8 @@ struct fscache_cache {
char identifier[36]; /* cache label */
/* node management */
- struct work_struct op_gc; /* operation garbage collector */
struct list_head object_list; /* list of data/index objects */
- struct list_head op_gc_list; /* list of ops to be deleted */
spinlock_t object_list_lock;
- spinlock_t op_gc_list_lock;
atomic_t object_count; /* no. of live objects in this cache */
struct fscache_object *fsdef; /* object for the fsdef index */
unsigned long flags;
@@ -76,68 +72,6 @@ struct fscache_cache {
extern wait_queue_head_t fscache_cache_cleared_wq;
-/*
- * operation to be applied to a cache object
- * - retrieval initiation operations are done in the context of the process
- * that issued them, and not in an async thread pool
- */
-typedef void (*fscache_operation_release_t)(struct fscache_operation *op);
-typedef void (*fscache_operation_processor_t)(struct fscache_operation *op);
-typedef void (*fscache_operation_cancel_t)(struct fscache_operation *op);
-
-enum fscache_operation_state {
- FSCACHE_OP_ST_BLANK, /* Op is not yet submitted */
- FSCACHE_OP_ST_INITIALISED, /* Op is initialised */
- FSCACHE_OP_ST_PENDING, /* Op is blocked from running */
- FSCACHE_OP_ST_IN_PROGRESS, /* Op is in progress */
- FSCACHE_OP_ST_COMPLETE, /* Op is complete */
- FSCACHE_OP_ST_CANCELLED, /* Op has been cancelled */
- FSCACHE_OP_ST_DEAD /* Op is now dead */
-};
-
-struct fscache_operation {
- struct work_struct work; /* record for async ops */
- struct list_head pend_link; /* link in object->pending_ops */
- struct fscache_object *object; /* object to be operated upon */
-
- unsigned long flags;
-#define FSCACHE_OP_TYPE 0x000f /* operation type */
-#define FSCACHE_OP_ASYNC 0x0001 /* - async op, processor may sleep for disk */
-#define FSCACHE_OP_MYTHREAD 0x0002 /* - processing is done be issuing thread, not pool */
-#define FSCACHE_OP_WAITING 4 /* cleared when op is woken */
-#define FSCACHE_OP_EXCLUSIVE 5 /* exclusive op, other ops must wait */
-#define FSCACHE_OP_DEC_READ_CNT 6 /* decrement object->n_reads on destruction */
-#define FSCACHE_OP_UNUSE_COOKIE 7 /* call fscache_unuse_cookie() on completion */
-#define FSCACHE_OP_KEEP_FLAGS 0x00f0 /* flags to keep when repurposing an op */
-
- enum fscache_operation_state state;
- atomic_t usage;
- unsigned debug_id; /* debugging ID */
-
- /* operation processor callback
- * - can be NULL if FSCACHE_OP_WAITING is going to be used to perform
- * the op in a non-pool thread */
- fscache_operation_processor_t processor;
-
- /* Operation cancellation cleanup (optional) */
- fscache_operation_cancel_t cancel;
-
- /* operation releaser */
- fscache_operation_release_t release;
-};
-
-extern atomic_t fscache_op_debug_id;
-extern void fscache_op_work_func(struct work_struct *work);
-
-extern void fscache_enqueue_operation(struct fscache_operation *);
-extern void fscache_op_complete(struct fscache_operation *, bool);
-extern void fscache_put_operation(struct fscache_operation *);
-extern void fscache_operation_init(struct fscache_cookie *,
- struct fscache_operation *,
- fscache_operation_processor_t,
- fscache_operation_cancel_t,
- fscache_operation_release_t);
-
/*
* cache operations
*/
@@ -232,9 +166,6 @@ struct fscache_object {
int n_children; /* number of child objects */
int n_ops; /* number of extant ops on object */
int n_obj_ops; /* number of object ops outstanding on object */
- int n_in_progress; /* number of ops in progress */
- int n_exclusive; /* number of exclusive ops queued or in progress */
- atomic_t n_reads; /* number of read ops in progress */
spinlock_t lock; /* state and operations lock */
unsigned long lookup_jif; /* time at which lookup started */
@@ -245,7 +176,6 @@ struct fscache_object {
unsigned long flags;
#define FSCACHE_OBJECT_LOCK 0 /* T if object is busy being processed */
-#define FSCACHE_OBJECT_PENDING_WRITE 1 /* T if object has pending write */
#define FSCACHE_OBJECT_WAITING 2 /* T if object is waiting on its parent */
#define FSCACHE_OBJECT_IS_LIVE 3 /* T if object is not withdrawn or relinquished */
#define FSCACHE_OBJECT_IS_LOOKED_UP 4 /* T if object has been looked up */
@@ -262,7 +192,6 @@ struct fscache_object {
struct work_struct work; /* attention scheduling record */
struct list_head dependents; /* FIFO of dependent objects */
struct list_head dep_link; /* link in parent's dependents list */
- struct list_head pending_ops; /* unstarted operations on this object */
#ifdef CONFIG_FSCACHE_OBJECT_LIST
struct rb_node objlist_link; /* link in global object list */
#endif
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 2ebfd688a7c2..08d7de72409d 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -33,24 +33,6 @@ enum fscache_cookie_trace {
fscache_cookie_put_parent,
};
-enum fscache_op_trace {
- fscache_op_cancel,
- fscache_op_cancel_all,
- fscache_op_cancelled,
- fscache_op_completed,
- fscache_op_enqueue_async,
- fscache_op_enqueue_mythread,
- fscache_op_gc,
- fscache_op_init,
- fscache_op_put,
- fscache_op_run,
- fscache_op_signal,
- fscache_op_submit,
- fscache_op_submit_ex,
- fscache_op_work,
- fscache_op_trace__nr
-};
-
#endif
/*
@@ -69,22 +51,6 @@ enum fscache_op_trace {
EM(fscache_cookie_put_object, "PUT obj") \
E_(fscache_cookie_put_parent, "PUT prn")
-#define fscache_op_traces \
- EM(fscache_op_cancel, "Cancel1") \
- EM(fscache_op_cancel_all, "CancelA") \
- EM(fscache_op_cancelled, "Canclld") \
- EM(fscache_op_completed, "Complet") \
- EM(fscache_op_enqueue_async, "EnqAsyn") \
- EM(fscache_op_enqueue_mythread, "EnqMyTh") \
- EM(fscache_op_gc, "GC ") \
- EM(fscache_op_init, "Init ") \
- EM(fscache_op_put, "Put ") \
- EM(fscache_op_run, "Run ") \
- EM(fscache_op_signal, "Signal ") \
- EM(fscache_op_submit, "Submit ") \
- EM(fscache_op_submit_ex, "SubmitX") \
- E_(fscache_op_work, "Work ")
-
/*
* Export enum symbols via userspace.
*/
@@ -309,29 +275,6 @@ TRACE_EVENT(fscache_osm,
__entry->event_num)
);
-TRACE_EVENT(fscache_op,
- TP_PROTO(struct fscache_cookie *cookie, struct fscache_operation *op,
- enum fscache_op_trace why),
-
- TP_ARGS(cookie, op, why),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, op )
- __field(enum fscache_op_trace, why )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->op = op->debug_id;
- __entry->why = why;
- ),
-
- TP_printk("c=%08x op=%08x %s",
- __entry->cookie, __entry->op,
- __print_symbolic(__entry->why, fscache_op_traces))
- );
-
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 20/61] cachefiles: Remove tree of active files and use S_CACHE_FILE inode flag
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the tree of active dentries from the cachefiles_cache struct and
instead set a flag, S_CACHE_FILE, on the backing inode to indicate that
this file is in use by the kernel so as to ward off other kernel users.
This simplifies the code a lot and also prevents two overlain caches from
fighting with each other.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/daemon.c | 4
fs/cachefiles/interface.c | 20 --
fs/cachefiles/internal.h | 10 -
fs/cachefiles/namei.c | 375 +++++++------------------------------
include/trace/events/cachefiles.h | 29 ---
5 files changed, 78 insertions(+), 360 deletions(-)
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 752c1e43416f..8a937d6d5e22 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -102,8 +102,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
}
mutex_init(&cache->daemon_mutex);
- cache->active_nodes = RB_ROOT;
- rwlock_init(&cache->active_lock);
init_waitqueue_head(&cache->daemon_pollwq);
/* set default caching limits
@@ -138,8 +136,6 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
cachefiles_daemon_unbind(cache);
- ASSERT(!cache->active_nodes.rb_node);
-
/* clean up the control file interface */
cache->cachefilesd = NULL;
file->private_data = NULL;
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 99f42d216ef7..b868afb970ad 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -36,7 +36,6 @@ static struct fscache_object *cachefiles_alloc_object(
ASSERTCMP(object->backer, ==, NULL);
- BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
atomic_set(&object->usage, 1);
fscache_object_init(&object->fscache, cookie, &cache->cache);
@@ -74,7 +73,6 @@ static struct fscache_object *cachefiles_alloc_object(
nomem_key:
kfree(buffer);
nomem_buffer:
- BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
kmem_cache_free(cachefiles_object_jar, object);
fscache_object_destroyed(&cache->cache);
nomem_object:
@@ -190,8 +188,6 @@ static void cachefiles_drop_object(struct fscache_object *_object)
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
- struct inode *inode;
- blkcnt_t i_blocks = 0;
ASSERT(_object);
@@ -218,10 +214,6 @@ static void cachefiles_drop_object(struct fscache_object *_object)
_object != cache->cache.fsdef
) {
_debug("- retire object OBJ%x", object->fscache.debug_id);
- inode = d_backing_inode(object->dentry);
- if (inode)
- i_blocks = inode->i_blocks;
-
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_delete_object(cache, object);
cachefiles_end_secure(cache, saved_cred);
@@ -231,14 +223,11 @@ static void cachefiles_drop_object(struct fscache_object *_object)
if (object->backer != object->dentry)
dput(object->backer);
object->backer = NULL;
- }
- /* note that the object is now inactive */
- if (test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags))
- cachefiles_mark_object_inactive(cache, object, i_blocks);
-
- dput(object->dentry);
- object->dentry = NULL;
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ dput(object->dentry);
+ object->dentry = NULL;
+ }
_leave("");
}
@@ -274,7 +263,6 @@ static void cachefiles_put_object(struct fscache_object *_object,
if (u == 0) {
_debug("- kill object OBJ%x", object->fscache.debug_id);
- ASSERT(!test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
ASSERTCMP(object->fscache.parent, ==, NULL);
ASSERTCMP(object->backer, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index a5d48f271ce1..f8f308ce7385 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -38,12 +38,9 @@ struct cachefiles_object {
struct dentry *dentry; /* the file/dir representing this object */
struct dentry *backer; /* backing file */
loff_t i_size; /* object size */
- unsigned long flags;
-#define CACHEFILES_OBJECT_ACTIVE 0 /* T if marked active */
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
uint8_t new; /* T if object new */
- struct rb_node active_node; /* link in active tree (dentry is key) */
};
extern struct kmem_cache *cachefiles_object_jar;
@@ -59,8 +56,6 @@ struct cachefiles_cache {
const struct cred *cache_cred; /* security override for accessing cache */
struct mutex daemon_mutex; /* command serialisation mutex */
wait_queue_head_t daemon_pollwq; /* poll waitqueue for daemon */
- struct rb_root active_nodes; /* active nodes (can't be culled) */
- rwlock_t active_lock; /* lock for active_nodes */
atomic_t gravecounter; /* graveyard uniquifier */
atomic_t f_released; /* number of objects released lately */
atomic_long_t b_released; /* number of blocks released lately */
@@ -126,9 +121,8 @@ extern char *cachefiles_cook_key(const u8 *raw, int keylen, uint8_t type);
/*
* namei.c
*/
-extern void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
- struct cachefiles_object *object,
- blkcnt_t i_blocks);
+extern void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry);
extern int cachefiles_delete_object(struct cachefiles_cache *cache,
struct cachefiles_object *object);
extern int cachefiles_walk_to_object(struct cachefiles_object *parent,
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 924042e8cced..818d1bca1904 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -21,251 +21,51 @@
#define CACHEFILES_KEYBUF_SIZE 512
/*
- * dump debugging info about an object
+ * Mark the backing file as being a cache file if it's not already in use so.
*/
-static noinline
-void __cachefiles_printk_object(struct cachefiles_object *object,
- const char *prefix)
+static bool cachefiles_mark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry)
{
- struct fscache_cookie *cookie;
- const u8 *k;
- unsigned loop;
-
- pr_err("%sobject: OBJ%x\n", prefix, object->fscache.debug_id);
- pr_err("%sobjstate=%s fl=%lx wbusy=%x ev=%lx[%lx]\n",
- prefix, object->fscache.state->name,
- object->fscache.flags, work_busy(&object->fscache.work),
- object->fscache.events, object->fscache.event_mask);
- pr_err("%sops=%u\n",
- prefix, object->fscache.n_ops);
- pr_err("%sparent=%p\n",
- prefix, object->fscache.parent);
-
- spin_lock(&object->fscache.lock);
- cookie = object->fscache.cookie;
- if (cookie) {
- pr_err("%scookie=%p [pr=%p fl=%lx]\n",
- prefix,
- object->fscache.cookie,
- object->fscache.cookie->parent,
- object->fscache.cookie->flags);
- pr_err("%skey=[%u] '", prefix, cookie->key_len);
- k = (cookie->key_len <= sizeof(cookie->inline_key)) ?
- cookie->inline_key : cookie->key;
- for (loop = 0; loop < cookie->key_len; loop++)
- pr_cont("%02x", k[loop]);
- pr_cont("'\n");
- } else {
- pr_err("%scookie=NULL\n", prefix);
- }
- spin_unlock(&object->fscache.lock);
-}
-
-/*
- * dump debugging info about a pair of objects
- */
-static noinline void cachefiles_printk_object(struct cachefiles_object *object,
- struct cachefiles_object *xobject)
-{
- if (object)
- __cachefiles_printk_object(object, "");
- if (xobject)
- __cachefiles_printk_object(xobject, "x");
-}
-
-/*
- * mark the owner of a dentry, if there is one, to indicate that that dentry
- * has been preemptively deleted
- * - the caller must hold the i_mutex on the dentry's parent as required to
- * call vfs_unlink(), vfs_rmdir() or vfs_rename()
- */
-static void cachefiles_mark_object_buried(struct cachefiles_cache *cache,
- struct dentry *dentry,
- enum fscache_why_object_killed why)
-{
- struct cachefiles_object *object;
- struct rb_node *p;
-
- _enter(",'%pd'", dentry);
-
- write_lock(&cache->active_lock);
-
- p = cache->active_nodes.rb_node;
- while (p) {
- object = rb_entry(p, struct cachefiles_object, active_node);
- if (object->dentry > dentry)
- p = p->rb_left;
- else if (object->dentry < dentry)
- p = p->rb_right;
- else
- goto found_dentry;
- }
-
- write_unlock(&cache->active_lock);
- trace_cachefiles_mark_buried(NULL, dentry, why);
- _leave(" [no owner]");
- return;
+ struct inode *inode = d_backing_inode(dentry);
+ bool can_use = false;
- /* found the dentry for */
-found_dentry:
- kdebug("preemptive burial: OBJ%x [%s] %p",
- object->fscache.debug_id,
- object->fscache.state->name,
- dentry);
+ _enter(",%p", object);
- trace_cachefiles_mark_buried(object, dentry, why);
+ inode_lock(inode);
- if (fscache_object_is_live(&object->fscache)) {
- pr_err("\n");
- pr_err("Error: Can't preemptively bury live object\n");
- cachefiles_printk_object(object, NULL);
+ if (!(inode->i_flags & S_CACHE_FILE)) {
+ inode->i_flags |= S_CACHE_FILE;
+ trace_cachefiles_mark_active(object, dentry);
+ can_use = true;
} else {
- if (why != FSCACHE_OBJECT_IS_STALE)
- fscache_object_mark_killed(&object->fscache, why);
+ pr_notice("cachefiles: Inode already in use: %pd\n", dentry);
}
- write_unlock(&cache->active_lock);
- _leave(" [owner marked]");
+ inode_unlock(inode);
+ return can_use;
}
/*
- * record the fact that an object is now active
+ * Unmark a backing inode.
*/
-static int cachefiles_mark_object_active(struct cachefiles_cache *cache,
- struct cachefiles_object *object)
+void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry)
{
- struct cachefiles_object *xobject;
- struct rb_node **_p, *_parent = NULL;
- struct dentry *dentry;
-
- _enter(",%p", object);
-
-try_again:
- write_lock(&cache->active_lock);
-
- dentry = object->dentry;
- trace_cachefiles_mark_active(object, dentry);
-
- if (test_and_set_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags)) {
- pr_err("Error: Object already active\n");
- cachefiles_printk_object(object, NULL);
- BUG();
- }
-
- _p = &cache->active_nodes.rb_node;
- while (*_p) {
- _parent = *_p;
- xobject = rb_entry(_parent,
- struct cachefiles_object, active_node);
-
- ASSERT(xobject != object);
-
- if (xobject->dentry > dentry)
- _p = &(*_p)->rb_left;
- else if (xobject->dentry < dentry)
- _p = &(*_p)->rb_right;
- else
- goto wait_for_old_object;
- }
-
- rb_link_node(&object->active_node, _parent, _p);
- rb_insert_color(&object->active_node, &cache->active_nodes);
-
- write_unlock(&cache->active_lock);
- _leave(" = 0");
- return 0;
-
- /* an old object from a previous incarnation is hogging the slot - we
- * need to wait for it to be destroyed */
-wait_for_old_object:
- trace_cachefiles_wait_active(object, dentry, xobject);
- clear_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags);
-
- if (fscache_object_is_live(&xobject->fscache)) {
- pr_err("\n");
- pr_err("Error: Unexpected object collision\n");
- cachefiles_printk_object(object, xobject);
- }
- atomic_inc(&xobject->usage);
- write_unlock(&cache->active_lock);
-
- if (test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags)) {
- wait_queue_head_t *wq;
-
- signed long timeout = 60 * HZ;
- wait_queue_entry_t wait;
- bool requeue;
-
- /* if the object we're waiting for is queued for processing,
- * then just put ourselves on the queue behind it */
- if (work_pending(&xobject->fscache.work)) {
- _debug("queue OBJ%x behind OBJ%x immediately",
- object->fscache.debug_id,
- xobject->fscache.debug_id);
- goto requeue;
- }
-
- /* otherwise we sleep until either the object we're waiting for
- * is done, or the fscache_object is congested */
- wq = bit_waitqueue(&xobject->flags, CACHEFILES_OBJECT_ACTIVE);
- init_wait(&wait);
- requeue = false;
- do {
- prepare_to_wait(wq, &wait, TASK_UNINTERRUPTIBLE);
- if (!test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags))
- break;
-
- requeue = fscache_object_sleep_till_congested(&timeout);
- } while (timeout > 0 && !requeue);
- finish_wait(wq, &wait);
-
- if (requeue &&
- test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags)) {
- _debug("queue OBJ%x behind OBJ%x after wait",
- object->fscache.debug_id,
- xobject->fscache.debug_id);
- goto requeue;
- }
-
- if (timeout <= 0) {
- pr_err("\n");
- pr_err("Error: Overlong wait for old active object to go away\n");
- cachefiles_printk_object(object, xobject);
- goto requeue;
- }
- }
-
- ASSERT(!test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags));
-
- cache->cache.ops->put_object(&xobject->fscache,
- (enum fscache_obj_ref_trace)cachefiles_obj_put_wait_retry);
- goto try_again;
+ struct inode *inode = d_backing_inode(dentry);
-requeue:
- cache->cache.ops->put_object(&xobject->fscache,
- (enum fscache_obj_ref_trace)cachefiles_obj_put_wait_timeo);
- _leave(" = -ETIMEDOUT");
- return -ETIMEDOUT;
+ inode_lock(inode);
+ inode->i_flags &= ~S_CACHE_FILE;
+ inode_unlock(inode);
+ trace_cachefiles_mark_inactive(object, dentry, inode);
}
/*
* Mark an object as being inactive.
*/
-void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
- struct cachefiles_object *object,
- blkcnt_t i_blocks)
+static void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
+ struct cachefiles_object *object)
{
- struct dentry *dentry = object->dentry;
- struct inode *inode = d_backing_inode(dentry);
-
- trace_cachefiles_mark_inactive(object, dentry, inode);
-
- write_lock(&cache->active_lock);
- rb_erase(&object->active_node, &cache->active_nodes);
- clear_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags);
- write_unlock(&cache->active_lock);
-
- wake_up_bit(&object->flags, CACHEFILES_OBJECT_ACTIVE);
+ blkcnt_t i_blocks = d_backing_inode(object->dentry)->i_blocks;
/* This object can now be culled, so we need to let the daemon know
* that there is something it can remove if it needs to.
@@ -286,7 +86,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
struct cachefiles_object *object,
struct dentry *dir,
struct dentry *rep,
- bool preemptive,
enum fscache_why_object_killed why)
{
struct dentry *grave, *trap;
@@ -310,9 +109,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
} else {
trace_cachefiles_unlink(object, rep, why);
ret = vfs_unlink(d_inode(dir), rep, NULL);
-
- if (preemptive)
- cachefiles_mark_object_buried(cache, rep, why);
}
inode_unlock(d_inode(dir));
@@ -373,8 +169,7 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
return -ENOMEM;
}
- cachefiles_io_error(cache, "Lookup error %ld",
- PTR_ERR(grave));
+ cachefiles_io_error(cache, "Lookup error %ld", PTR_ERR(grave));
return -EIO;
}
@@ -416,9 +211,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
if (ret != 0 && ret != -ENOMEM)
cachefiles_io_error(cache,
"Rename failed with error %d", ret);
-
- if (preemptive)
- cachefiles_mark_object_buried(cache, rep, why);
}
unlock_rename(cache->graveyard, dir);
@@ -446,26 +238,18 @@ int cachefiles_delete_object(struct cachefiles_cache *cache,
inode_lock_nested(d_inode(dir), I_MUTEX_PARENT);
- if (test_bit(FSCACHE_OBJECT_KILLED_BY_CACHE, &object->fscache.flags)) {
- /* object allocation for the same key preemptively deleted this
- * object's file so that it could create its own file */
- _debug("object preemptively buried");
+ /* We need to check that our parent is _still_ our parent - it may have
+ * been renamed.
+ */
+ if (dir == object->dentry->d_parent) {
+ ret = cachefiles_bury_object(cache, object, dir, object->dentry,
+ FSCACHE_OBJECT_WAS_RETIRED);
+ } else {
+ /* It got moved, presumably by cachefilesd culling it, so it's
+ * no longer in the key path and we can ignore it.
+ */
inode_unlock(d_inode(dir));
ret = 0;
- } else {
- /* we need to check that our parent is _still_ our parent - it
- * may have been renamed */
- if (dir == object->dentry->d_parent) {
- ret = cachefiles_bury_object(cache, object, dir,
- object->dentry, false,
- FSCACHE_OBJECT_WAS_RETIRED);
- } else {
- /* it got moved, presumably by cachefilesd culling it,
- * so it's no longer in the key path and we can ignore
- * it */
- inode_unlock(d_inode(dir));
- ret = 0;
- }
}
dput(dir);
@@ -487,6 +271,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct path path;
unsigned long start;
const char *name;
+ bool marked = false;
int ret, nlen;
_enter("OBJ%x{%p},OBJ%x,%s,",
@@ -529,6 +314,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_hist(cachefiles_lookup_histogram, start);
if (IS_ERR(next)) {
trace_cachefiles_lookup(object, next, NULL);
+ ret = PTR_ERR(next);
goto lookup_error;
}
@@ -628,6 +414,13 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
/* we've found the object we were looking for */
object->dentry = next;
+ /* note that we're now using this object */
+ if (!cachefiles_mark_inode_in_use(object, object->dentry)) {
+ ret = -EBUSY;
+ goto check_error_unlock;
+ }
+ marked = true;
+
/* if we've found that the terminal object exists, then we need to
* check its attributes and delete it if it's out of date */
if (!object->new) {
@@ -640,13 +433,12 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
object->dentry = NULL;
ret = cachefiles_bury_object(cache, object, dir, next,
- true,
FSCACHE_OBJECT_IS_STALE);
dput(next);
next = NULL;
if (ret < 0)
- goto delete_error;
+ goto error_out2;
_debug("redo lookup");
fscache_object_retrying_stale(&object->fscache);
@@ -654,16 +446,10 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
}
}
- /* note that we're now using this object */
- ret = cachefiles_mark_object_active(cache, object);
-
inode_unlock(d_inode(dir));
dput(dir);
dir = NULL;
- if (ret == -ETIMEDOUT)
- goto mark_active_timed_out;
-
_debug("=== OBTAINED_OBJECT ===");
if (object->new) {
@@ -712,26 +498,19 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_io_error(cache, "Create/mkdir failed");
goto error;
-mark_active_timed_out:
- _debug("mark active timed out");
- goto release_dentry;
-
+check_error_unlock:
+ inode_unlock(d_inode(dir));
+ dput(dir);
check_error:
- _debug("check error %d", ret);
- cachefiles_mark_object_inactive(
- cache, object, d_backing_inode(object->dentry)->i_blocks);
-release_dentry:
+ if (marked)
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ cachefiles_mark_object_inactive(cache, object);
dput(object->dentry);
object->dentry = NULL;
goto error_out;
-delete_error:
- _debug("delete error %d", ret);
- goto error_out2;
-
lookup_error:
- _debug("lookup error %ld", PTR_ERR(next));
- ret = PTR_ERR(next);
+ _debug("lookup error %d", ret);
if (ret == -EIO)
cachefiles_io_error(cache, "Lookup failed");
next = NULL;
@@ -861,8 +640,6 @@ static struct dentry *cachefiles_check_active(struct cachefiles_cache *cache,
struct dentry *dir,
char *filename)
{
- struct cachefiles_object *object;
- struct rb_node *_n;
struct dentry *victim;
unsigned long start;
int ret;
@@ -892,34 +669,9 @@ static struct dentry *cachefiles_check_active(struct cachefiles_cache *cache,
return ERR_PTR(-ENOENT);
}
- /* check to see if we're using this object */
- read_lock(&cache->active_lock);
-
- _n = cache->active_nodes.rb_node;
-
- while (_n) {
- object = rb_entry(_n, struct cachefiles_object, active_node);
-
- if (object->dentry > victim)
- _n = _n->rb_left;
- else if (object->dentry < victim)
- _n = _n->rb_right;
- else
- goto object_in_use;
- }
-
- read_unlock(&cache->active_lock);
-
//_leave(" = %p", victim);
return victim;
-object_in_use:
- read_unlock(&cache->active_lock);
- inode_unlock(d_inode(dir));
- dput(victim);
- //_leave(" = -EBUSY [in use]");
- return ERR_PTR(-EBUSY);
-
lookup_error:
inode_unlock(d_inode(dir));
ret = PTR_ERR(victim);
@@ -948,6 +700,7 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
char *filename)
{
struct dentry *victim;
+ struct inode *inode;
int ret;
_enter(",%pd/,%s", dir, filename);
@@ -956,6 +709,19 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
if (IS_ERR(victim))
return PTR_ERR(victim);
+ /* check to see if someone is using this object */
+ inode = d_inode(victim);
+ inode_lock(inode);
+ if (inode->i_flags & S_CACHE_FILE) {
+ ret = -EBUSY;
+ } else {
+ inode->i_flags |= S_CACHE_FILE;
+ ret = 0;
+ }
+ inode_unlock(inode);
+ if (ret < 0)
+ goto error_unlock;
+
_debug("victim -> %p %s",
victim, d_backing_inode(victim) ? "positive" : "negative");
@@ -971,7 +737,7 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
/* actually remove the victim (drops the dir mutex) */
_debug("bury");
- ret = cachefiles_bury_object(cache, NULL, dir, victim, false,
+ ret = cachefiles_bury_object(cache, NULL, dir, victim,
FSCACHE_OBJECT_WAS_CULLED);
if (ret < 0)
goto error;
@@ -1008,6 +774,7 @@ int cachefiles_check_in_use(struct cachefiles_cache *cache, struct dentry *dir,
char *filename)
{
struct dentry *victim;
+ int ret = 0;
//_enter(",%pd/,%s",
// dir, filename);
@@ -1017,7 +784,9 @@ int cachefiles_check_in_use(struct cachefiles_cache *cache, struct dentry *dir,
return PTR_ERR(victim);
inode_unlock(d_inode(dir));
+ if (d_inode(victim)->i_flags & S_CACHE_FILE)
+ ret = -EBUSY;
dput(victim);
//_leave(" = 0");
- return 0;
+ return ret;
}
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 9a448fe9355d..c877035c2946 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -237,35 +237,6 @@ TRACE_EVENT(cachefiles_mark_active,
__entry->obj, __entry->de)
);
-TRACE_EVENT(cachefiles_wait_active,
- TP_PROTO(struct cachefiles_object *obj,
- struct dentry *de,
- struct cachefiles_object *xobj),
-
- TP_ARGS(obj, de, xobj),
-
- /* Note that obj may be NULL */
- TP_STRUCT__entry(
- __field(unsigned int, obj )
- __field(unsigned int, xobj )
- __field(struct dentry *, de )
- __field(u16, flags )
- __field(u16, fsc_flags )
- ),
-
- TP_fast_assign(
- __entry->obj = obj->fscache.debug_id;
- __entry->de = de;
- __entry->xobj = xobj->fscache.debug_id;
- __entry->flags = xobj->flags;
- __entry->fsc_flags = xobj->fscache.flags;
- ),
-
- TP_printk("o=%08x d=%p wo=%08x wf=%x wff=%x",
- __entry->obj, __entry->de, __entry->xobj,
- __entry->flags, __entry->fsc_flags)
- );
-
TRACE_EVENT(cachefiles_mark_inactive,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 20/61] cachefiles: Remove tree of active files and use S_CACHE_FILE inode flag
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove the tree of active dentries from the cachefiles_cache struct and
instead set a flag, S_CACHE_FILE, on the backing inode to indicate that
this file is in use by the kernel so as to ward off other kernel users.
This simplifies the code a lot and also prevents two overlain caches from
fighting with each other.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/daemon.c | 4
fs/cachefiles/interface.c | 20 --
fs/cachefiles/internal.h | 10 -
fs/cachefiles/namei.c | 375 +++++++------------------------------
include/trace/events/cachefiles.h | 29 ---
5 files changed, 78 insertions(+), 360 deletions(-)
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 752c1e43416f..8a937d6d5e22 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -102,8 +102,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
}
mutex_init(&cache->daemon_mutex);
- cache->active_nodes = RB_ROOT;
- rwlock_init(&cache->active_lock);
init_waitqueue_head(&cache->daemon_pollwq);
/* set default caching limits
@@ -138,8 +136,6 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
cachefiles_daemon_unbind(cache);
- ASSERT(!cache->active_nodes.rb_node);
-
/* clean up the control file interface */
cache->cachefilesd = NULL;
file->private_data = NULL;
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 99f42d216ef7..b868afb970ad 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -36,7 +36,6 @@ static struct fscache_object *cachefiles_alloc_object(
ASSERTCMP(object->backer, ==, NULL);
- BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
atomic_set(&object->usage, 1);
fscache_object_init(&object->fscache, cookie, &cache->cache);
@@ -74,7 +73,6 @@ static struct fscache_object *cachefiles_alloc_object(
nomem_key:
kfree(buffer);
nomem_buffer:
- BUG_ON(test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
kmem_cache_free(cachefiles_object_jar, object);
fscache_object_destroyed(&cache->cache);
nomem_object:
@@ -190,8 +188,6 @@ static void cachefiles_drop_object(struct fscache_object *_object)
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
- struct inode *inode;
- blkcnt_t i_blocks = 0;
ASSERT(_object);
@@ -218,10 +214,6 @@ static void cachefiles_drop_object(struct fscache_object *_object)
_object != cache->cache.fsdef
) {
_debug("- retire object OBJ%x", object->fscache.debug_id);
- inode = d_backing_inode(object->dentry);
- if (inode)
- i_blocks = inode->i_blocks;
-
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_delete_object(cache, object);
cachefiles_end_secure(cache, saved_cred);
@@ -231,14 +223,11 @@ static void cachefiles_drop_object(struct fscache_object *_object)
if (object->backer != object->dentry)
dput(object->backer);
object->backer = NULL;
- }
- /* note that the object is now inactive */
- if (test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags))
- cachefiles_mark_object_inactive(cache, object, i_blocks);
-
- dput(object->dentry);
- object->dentry = NULL;
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ dput(object->dentry);
+ object->dentry = NULL;
+ }
_leave("");
}
@@ -274,7 +263,6 @@ static void cachefiles_put_object(struct fscache_object *_object,
if (u == 0) {
_debug("- kill object OBJ%x", object->fscache.debug_id);
- ASSERT(!test_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags));
ASSERTCMP(object->fscache.parent, ==, NULL);
ASSERTCMP(object->backer, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index a5d48f271ce1..f8f308ce7385 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -38,12 +38,9 @@ struct cachefiles_object {
struct dentry *dentry; /* the file/dir representing this object */
struct dentry *backer; /* backing file */
loff_t i_size; /* object size */
- unsigned long flags;
-#define CACHEFILES_OBJECT_ACTIVE 0 /* T if marked active */
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
uint8_t new; /* T if object new */
- struct rb_node active_node; /* link in active tree (dentry is key) */
};
extern struct kmem_cache *cachefiles_object_jar;
@@ -59,8 +56,6 @@ struct cachefiles_cache {
const struct cred *cache_cred; /* security override for accessing cache */
struct mutex daemon_mutex; /* command serialisation mutex */
wait_queue_head_t daemon_pollwq; /* poll waitqueue for daemon */
- struct rb_root active_nodes; /* active nodes (can't be culled) */
- rwlock_t active_lock; /* lock for active_nodes */
atomic_t gravecounter; /* graveyard uniquifier */
atomic_t f_released; /* number of objects released lately */
atomic_long_t b_released; /* number of blocks released lately */
@@ -126,9 +121,8 @@ extern char *cachefiles_cook_key(const u8 *raw, int keylen, uint8_t type);
/*
* namei.c
*/
-extern void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
- struct cachefiles_object *object,
- blkcnt_t i_blocks);
+extern void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry);
extern int cachefiles_delete_object(struct cachefiles_cache *cache,
struct cachefiles_object *object);
extern int cachefiles_walk_to_object(struct cachefiles_object *parent,
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 924042e8cced..818d1bca1904 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -21,251 +21,51 @@
#define CACHEFILES_KEYBUF_SIZE 512
/*
- * dump debugging info about an object
+ * Mark the backing file as being a cache file if it's not already in use so.
*/
-static noinline
-void __cachefiles_printk_object(struct cachefiles_object *object,
- const char *prefix)
+static bool cachefiles_mark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry)
{
- struct fscache_cookie *cookie;
- const u8 *k;
- unsigned loop;
-
- pr_err("%sobject: OBJ%x\n", prefix, object->fscache.debug_id);
- pr_err("%sobjstate=%s fl=%lx wbusy=%x ev=%lx[%lx]\n",
- prefix, object->fscache.state->name,
- object->fscache.flags, work_busy(&object->fscache.work),
- object->fscache.events, object->fscache.event_mask);
- pr_err("%sops=%u\n",
- prefix, object->fscache.n_ops);
- pr_err("%sparent=%p\n",
- prefix, object->fscache.parent);
-
- spin_lock(&object->fscache.lock);
- cookie = object->fscache.cookie;
- if (cookie) {
- pr_err("%scookie=%p [pr=%p fl=%lx]\n",
- prefix,
- object->fscache.cookie,
- object->fscache.cookie->parent,
- object->fscache.cookie->flags);
- pr_err("%skey=[%u] '", prefix, cookie->key_len);
- k = (cookie->key_len <= sizeof(cookie->inline_key)) ?
- cookie->inline_key : cookie->key;
- for (loop = 0; loop < cookie->key_len; loop++)
- pr_cont("%02x", k[loop]);
- pr_cont("'\n");
- } else {
- pr_err("%scookie=NULL\n", prefix);
- }
- spin_unlock(&object->fscache.lock);
-}
-
-/*
- * dump debugging info about a pair of objects
- */
-static noinline void cachefiles_printk_object(struct cachefiles_object *object,
- struct cachefiles_object *xobject)
-{
- if (object)
- __cachefiles_printk_object(object, "");
- if (xobject)
- __cachefiles_printk_object(xobject, "x");
-}
-
-/*
- * mark the owner of a dentry, if there is one, to indicate that that dentry
- * has been preemptively deleted
- * - the caller must hold the i_mutex on the dentry's parent as required to
- * call vfs_unlink(), vfs_rmdir() or vfs_rename()
- */
-static void cachefiles_mark_object_buried(struct cachefiles_cache *cache,
- struct dentry *dentry,
- enum fscache_why_object_killed why)
-{
- struct cachefiles_object *object;
- struct rb_node *p;
-
- _enter(",'%pd'", dentry);
-
- write_lock(&cache->active_lock);
-
- p = cache->active_nodes.rb_node;
- while (p) {
- object = rb_entry(p, struct cachefiles_object, active_node);
- if (object->dentry > dentry)
- p = p->rb_left;
- else if (object->dentry < dentry)
- p = p->rb_right;
- else
- goto found_dentry;
- }
-
- write_unlock(&cache->active_lock);
- trace_cachefiles_mark_buried(NULL, dentry, why);
- _leave(" [no owner]");
- return;
+ struct inode *inode = d_backing_inode(dentry);
+ bool can_use = false;
- /* found the dentry for */
-found_dentry:
- kdebug("preemptive burial: OBJ%x [%s] %p",
- object->fscache.debug_id,
- object->fscache.state->name,
- dentry);
+ _enter(",%p", object);
- trace_cachefiles_mark_buried(object, dentry, why);
+ inode_lock(inode);
- if (fscache_object_is_live(&object->fscache)) {
- pr_err("\n");
- pr_err("Error: Can't preemptively bury live object\n");
- cachefiles_printk_object(object, NULL);
+ if (!(inode->i_flags & S_CACHE_FILE)) {
+ inode->i_flags |= S_CACHE_FILE;
+ trace_cachefiles_mark_active(object, dentry);
+ can_use = true;
} else {
- if (why != FSCACHE_OBJECT_IS_STALE)
- fscache_object_mark_killed(&object->fscache, why);
+ pr_notice("cachefiles: Inode already in use: %pd\n", dentry);
}
- write_unlock(&cache->active_lock);
- _leave(" [owner marked]");
+ inode_unlock(inode);
+ return can_use;
}
/*
- * record the fact that an object is now active
+ * Unmark a backing inode.
*/
-static int cachefiles_mark_object_active(struct cachefiles_cache *cache,
- struct cachefiles_object *object)
+void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
+ struct dentry *dentry)
{
- struct cachefiles_object *xobject;
- struct rb_node **_p, *_parent = NULL;
- struct dentry *dentry;
-
- _enter(",%p", object);
-
-try_again:
- write_lock(&cache->active_lock);
-
- dentry = object->dentry;
- trace_cachefiles_mark_active(object, dentry);
-
- if (test_and_set_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags)) {
- pr_err("Error: Object already active\n");
- cachefiles_printk_object(object, NULL);
- BUG();
- }
-
- _p = &cache->active_nodes.rb_node;
- while (*_p) {
- _parent = *_p;
- xobject = rb_entry(_parent,
- struct cachefiles_object, active_node);
-
- ASSERT(xobject != object);
-
- if (xobject->dentry > dentry)
- _p = &(*_p)->rb_left;
- else if (xobject->dentry < dentry)
- _p = &(*_p)->rb_right;
- else
- goto wait_for_old_object;
- }
-
- rb_link_node(&object->active_node, _parent, _p);
- rb_insert_color(&object->active_node, &cache->active_nodes);
-
- write_unlock(&cache->active_lock);
- _leave(" = 0");
- return 0;
-
- /* an old object from a previous incarnation is hogging the slot - we
- * need to wait for it to be destroyed */
-wait_for_old_object:
- trace_cachefiles_wait_active(object, dentry, xobject);
- clear_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags);
-
- if (fscache_object_is_live(&xobject->fscache)) {
- pr_err("\n");
- pr_err("Error: Unexpected object collision\n");
- cachefiles_printk_object(object, xobject);
- }
- atomic_inc(&xobject->usage);
- write_unlock(&cache->active_lock);
-
- if (test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags)) {
- wait_queue_head_t *wq;
-
- signed long timeout = 60 * HZ;
- wait_queue_entry_t wait;
- bool requeue;
-
- /* if the object we're waiting for is queued for processing,
- * then just put ourselves on the queue behind it */
- if (work_pending(&xobject->fscache.work)) {
- _debug("queue OBJ%x behind OBJ%x immediately",
- object->fscache.debug_id,
- xobject->fscache.debug_id);
- goto requeue;
- }
-
- /* otherwise we sleep until either the object we're waiting for
- * is done, or the fscache_object is congested */
- wq = bit_waitqueue(&xobject->flags, CACHEFILES_OBJECT_ACTIVE);
- init_wait(&wait);
- requeue = false;
- do {
- prepare_to_wait(wq, &wait, TASK_UNINTERRUPTIBLE);
- if (!test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags))
- break;
-
- requeue = fscache_object_sleep_till_congested(&timeout);
- } while (timeout > 0 && !requeue);
- finish_wait(wq, &wait);
-
- if (requeue &&
- test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags)) {
- _debug("queue OBJ%x behind OBJ%x after wait",
- object->fscache.debug_id,
- xobject->fscache.debug_id);
- goto requeue;
- }
-
- if (timeout <= 0) {
- pr_err("\n");
- pr_err("Error: Overlong wait for old active object to go away\n");
- cachefiles_printk_object(object, xobject);
- goto requeue;
- }
- }
-
- ASSERT(!test_bit(CACHEFILES_OBJECT_ACTIVE, &xobject->flags));
-
- cache->cache.ops->put_object(&xobject->fscache,
- (enum fscache_obj_ref_trace)cachefiles_obj_put_wait_retry);
- goto try_again;
+ struct inode *inode = d_backing_inode(dentry);
-requeue:
- cache->cache.ops->put_object(&xobject->fscache,
- (enum fscache_obj_ref_trace)cachefiles_obj_put_wait_timeo);
- _leave(" = -ETIMEDOUT");
- return -ETIMEDOUT;
+ inode_lock(inode);
+ inode->i_flags &= ~S_CACHE_FILE;
+ inode_unlock(inode);
+ trace_cachefiles_mark_inactive(object, dentry, inode);
}
/*
* Mark an object as being inactive.
*/
-void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
- struct cachefiles_object *object,
- blkcnt_t i_blocks)
+static void cachefiles_mark_object_inactive(struct cachefiles_cache *cache,
+ struct cachefiles_object *object)
{
- struct dentry *dentry = object->dentry;
- struct inode *inode = d_backing_inode(dentry);
-
- trace_cachefiles_mark_inactive(object, dentry, inode);
-
- write_lock(&cache->active_lock);
- rb_erase(&object->active_node, &cache->active_nodes);
- clear_bit(CACHEFILES_OBJECT_ACTIVE, &object->flags);
- write_unlock(&cache->active_lock);
-
- wake_up_bit(&object->flags, CACHEFILES_OBJECT_ACTIVE);
+ blkcnt_t i_blocks = d_backing_inode(object->dentry)->i_blocks;
/* This object can now be culled, so we need to let the daemon know
* that there is something it can remove if it needs to.
@@ -286,7 +86,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
struct cachefiles_object *object,
struct dentry *dir,
struct dentry *rep,
- bool preemptive,
enum fscache_why_object_killed why)
{
struct dentry *grave, *trap;
@@ -310,9 +109,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
} else {
trace_cachefiles_unlink(object, rep, why);
ret = vfs_unlink(d_inode(dir), rep, NULL);
-
- if (preemptive)
- cachefiles_mark_object_buried(cache, rep, why);
}
inode_unlock(d_inode(dir));
@@ -373,8 +169,7 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
return -ENOMEM;
}
- cachefiles_io_error(cache, "Lookup error %ld",
- PTR_ERR(grave));
+ cachefiles_io_error(cache, "Lookup error %ld", PTR_ERR(grave));
return -EIO;
}
@@ -416,9 +211,6 @@ static int cachefiles_bury_object(struct cachefiles_cache *cache,
if (ret != 0 && ret != -ENOMEM)
cachefiles_io_error(cache,
"Rename failed with error %d", ret);
-
- if (preemptive)
- cachefiles_mark_object_buried(cache, rep, why);
}
unlock_rename(cache->graveyard, dir);
@@ -446,26 +238,18 @@ int cachefiles_delete_object(struct cachefiles_cache *cache,
inode_lock_nested(d_inode(dir), I_MUTEX_PARENT);
- if (test_bit(FSCACHE_OBJECT_KILLED_BY_CACHE, &object->fscache.flags)) {
- /* object allocation for the same key preemptively deleted this
- * object's file so that it could create its own file */
- _debug("object preemptively buried");
+ /* We need to check that our parent is _still_ our parent - it may have
+ * been renamed.
+ */
+ if (dir == object->dentry->d_parent) {
+ ret = cachefiles_bury_object(cache, object, dir, object->dentry,
+ FSCACHE_OBJECT_WAS_RETIRED);
+ } else {
+ /* It got moved, presumably by cachefilesd culling it, so it's
+ * no longer in the key path and we can ignore it.
+ */
inode_unlock(d_inode(dir));
ret = 0;
- } else {
- /* we need to check that our parent is _still_ our parent - it
- * may have been renamed */
- if (dir == object->dentry->d_parent) {
- ret = cachefiles_bury_object(cache, object, dir,
- object->dentry, false,
- FSCACHE_OBJECT_WAS_RETIRED);
- } else {
- /* it got moved, presumably by cachefilesd culling it,
- * so it's no longer in the key path and we can ignore
- * it */
- inode_unlock(d_inode(dir));
- ret = 0;
- }
}
dput(dir);
@@ -487,6 +271,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct path path;
unsigned long start;
const char *name;
+ bool marked = false;
int ret, nlen;
_enter("OBJ%x{%p},OBJ%x,%s,",
@@ -529,6 +314,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_hist(cachefiles_lookup_histogram, start);
if (IS_ERR(next)) {
trace_cachefiles_lookup(object, next, NULL);
+ ret = PTR_ERR(next);
goto lookup_error;
}
@@ -628,6 +414,13 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
/* we've found the object we were looking for */
object->dentry = next;
+ /* note that we're now using this object */
+ if (!cachefiles_mark_inode_in_use(object, object->dentry)) {
+ ret = -EBUSY;
+ goto check_error_unlock;
+ }
+ marked = true;
+
/* if we've found that the terminal object exists, then we need to
* check its attributes and delete it if it's out of date */
if (!object->new) {
@@ -640,13 +433,12 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
object->dentry = NULL;
ret = cachefiles_bury_object(cache, object, dir, next,
- true,
FSCACHE_OBJECT_IS_STALE);
dput(next);
next = NULL;
if (ret < 0)
- goto delete_error;
+ goto error_out2;
_debug("redo lookup");
fscache_object_retrying_stale(&object->fscache);
@@ -654,16 +446,10 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
}
}
- /* note that we're now using this object */
- ret = cachefiles_mark_object_active(cache, object);
-
inode_unlock(d_inode(dir));
dput(dir);
dir = NULL;
- if (ret == -ETIMEDOUT)
- goto mark_active_timed_out;
-
_debug("=== OBTAINED_OBJECT ===");
if (object->new) {
@@ -712,26 +498,19 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_io_error(cache, "Create/mkdir failed");
goto error;
-mark_active_timed_out:
- _debug("mark active timed out");
- goto release_dentry;
-
+check_error_unlock:
+ inode_unlock(d_inode(dir));
+ dput(dir);
check_error:
- _debug("check error %d", ret);
- cachefiles_mark_object_inactive(
- cache, object, d_backing_inode(object->dentry)->i_blocks);
-release_dentry:
+ if (marked)
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ cachefiles_mark_object_inactive(cache, object);
dput(object->dentry);
object->dentry = NULL;
goto error_out;
-delete_error:
- _debug("delete error %d", ret);
- goto error_out2;
-
lookup_error:
- _debug("lookup error %ld", PTR_ERR(next));
- ret = PTR_ERR(next);
+ _debug("lookup error %d", ret);
if (ret == -EIO)
cachefiles_io_error(cache, "Lookup failed");
next = NULL;
@@ -861,8 +640,6 @@ static struct dentry *cachefiles_check_active(struct cachefiles_cache *cache,
struct dentry *dir,
char *filename)
{
- struct cachefiles_object *object;
- struct rb_node *_n;
struct dentry *victim;
unsigned long start;
int ret;
@@ -892,34 +669,9 @@ static struct dentry *cachefiles_check_active(struct cachefiles_cache *cache,
return ERR_PTR(-ENOENT);
}
- /* check to see if we're using this object */
- read_lock(&cache->active_lock);
-
- _n = cache->active_nodes.rb_node;
-
- while (_n) {
- object = rb_entry(_n, struct cachefiles_object, active_node);
-
- if (object->dentry > victim)
- _n = _n->rb_left;
- else if (object->dentry < victim)
- _n = _n->rb_right;
- else
- goto object_in_use;
- }
-
- read_unlock(&cache->active_lock);
-
//_leave(" = %p", victim);
return victim;
-object_in_use:
- read_unlock(&cache->active_lock);
- inode_unlock(d_inode(dir));
- dput(victim);
- //_leave(" = -EBUSY [in use]");
- return ERR_PTR(-EBUSY);
-
lookup_error:
inode_unlock(d_inode(dir));
ret = PTR_ERR(victim);
@@ -948,6 +700,7 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
char *filename)
{
struct dentry *victim;
+ struct inode *inode;
int ret;
_enter(",%pd/,%s", dir, filename);
@@ -956,6 +709,19 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
if (IS_ERR(victim))
return PTR_ERR(victim);
+ /* check to see if someone is using this object */
+ inode = d_inode(victim);
+ inode_lock(inode);
+ if (inode->i_flags & S_CACHE_FILE) {
+ ret = -EBUSY;
+ } else {
+ inode->i_flags |= S_CACHE_FILE;
+ ret = 0;
+ }
+ inode_unlock(inode);
+ if (ret < 0)
+ goto error_unlock;
+
_debug("victim -> %p %s",
victim, d_backing_inode(victim) ? "positive" : "negative");
@@ -971,7 +737,7 @@ int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
/* actually remove the victim (drops the dir mutex) */
_debug("bury");
- ret = cachefiles_bury_object(cache, NULL, dir, victim, false,
+ ret = cachefiles_bury_object(cache, NULL, dir, victim,
FSCACHE_OBJECT_WAS_CULLED);
if (ret < 0)
goto error;
@@ -1008,6 +774,7 @@ int cachefiles_check_in_use(struct cachefiles_cache *cache, struct dentry *dir,
char *filename)
{
struct dentry *victim;
+ int ret = 0;
//_enter(",%pd/,%s",
// dir, filename);
@@ -1017,7 +784,9 @@ int cachefiles_check_in_use(struct cachefiles_cache *cache, struct dentry *dir,
return PTR_ERR(victim);
inode_unlock(d_inode(dir));
+ if (d_inode(victim)->i_flags & S_CACHE_FILE)
+ ret = -EBUSY;
dput(victim);
//_leave(" = 0");
- return 0;
+ return ret;
}
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 9a448fe9355d..c877035c2946 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -237,35 +237,6 @@ TRACE_EVENT(cachefiles_mark_active,
__entry->obj, __entry->de)
);
-TRACE_EVENT(cachefiles_wait_active,
- TP_PROTO(struct cachefiles_object *obj,
- struct dentry *de,
- struct cachefiles_object *xobj),
-
- TP_ARGS(obj, de, xobj),
-
- /* Note that obj may be NULL */
- TP_STRUCT__entry(
- __field(unsigned int, obj )
- __field(unsigned int, xobj )
- __field(struct dentry *, de )
- __field(u16, flags )
- __field(u16, fsc_flags )
- ),
-
- TP_fast_assign(
- __entry->obj = obj->fscache.debug_id;
- __entry->de = de;
- __entry->xobj = xobj->fscache.debug_id;
- __entry->flags = xobj->flags;
- __entry->fsc_flags = xobj->fscache.flags;
- ),
-
- TP_printk("o=%08x d=%p wo=%08x wf=%x wff=%x",
- __entry->obj, __entry->de, __entry->xobj,
- __entry->flags, __entry->fsc_flags)
- );
-
TRACE_EVENT(cachefiles_mark_inactive,
TP_PROTO(struct cachefiles_object *obj,
struct dentry *de,
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 21/61] fscache: Provide a simple thread pool for running ops asynchronously
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Provide a simple thread pool that can be used to run cookie management
operations in the background and a dispatcher infrastructure to punt
operations to the pool if threads are available or to just run the
operation in the calling thread if not.
A future patch will replace all the object state machine stuff with whole
routines that do all the work in one go without trying to interleave bits
from various objects.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/Makefile | 1
fs/fscache/dispatcher.c | 144 ++++++++++++++++++++++++++++++++++++++++
fs/fscache/internal.h | 8 ++
fs/fscache/main.c | 7 ++
include/trace/events/fscache.h | 6 +-
5 files changed, 165 insertions(+), 1 deletion(-)
create mode 100644 fs/fscache/dispatcher.c
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index ac3fcd909fff..7b10c6aad157 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -6,6 +6,7 @@
fscache-y := \
cache.o \
cookie.o \
+ dispatcher.o \
fsdef.o \
main.o \
netfs.o \
diff --git a/fs/fscache/dispatcher.c b/fs/fscache/dispatcher.c
new file mode 100644
index 000000000000..fba71b99c951
--- /dev/null
+++ b/fs/fscache/dispatcher.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Object dispatcher
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/kthread.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+#include <linux/freezer.h>
+#include "internal.h"
+
+#define FSCACHE_DISPATCHER_POOL_SIZE 8
+
+static LIST_HEAD(fscache_pending_work);
+static DEFINE_SPINLOCK(fscache_work_lock);
+static DECLARE_WAIT_QUEUE_HEAD(fscache_dispatcher_pool);
+static struct completion fscache_dispatcher_pool_done[FSCACHE_DISPATCHER_POOL_SIZE];
+static bool fscache_dispatcher_stop;
+
+struct fscache_work {
+ struct list_head link;
+ struct fscache_cookie *cookie;
+ struct fscache_object *object;
+ int param;
+ void (*func)(struct fscache_cookie *, struct fscache_object *, int);
+};
+
+/*
+ * Attempt to queue some work to do. If there's too much asynchronous work
+ * already queued, we'll do it here in this thread instead.
+ */
+void fscache_dispatch(struct fscache_cookie *cookie,
+ struct fscache_object *object,
+ int param,
+ void (*func)(struct fscache_cookie *,
+ struct fscache_object *, int))
+{
+ struct fscache_work *work;
+ bool queued = false;
+
+ work = kzalloc(sizeof(struct fscache_work), GFP_KERNEL);
+ if (work) {
+ work->cookie = cookie;
+ work->object = object;
+ work->param = param;
+ work->func = func;
+
+ spin_lock(&fscache_work_lock);
+ if (waitqueue_active(&fscache_dispatcher_pool) ||
+ list_empty(&fscache_pending_work)) {
+ fscache_cookie_get(cookie, fscache_cookie_get_work);
+ list_add_tail(&work->link, &fscache_pending_work);
+ wake_up(&fscache_dispatcher_pool);
+ queued = true;
+ }
+ spin_unlock(&fscache_work_lock);
+ }
+
+ if (!queued) {
+ kfree(work);
+ func(cookie, object, param);
+ }
+}
+
+/*
+ * A dispatcher thread.
+ */
+static int fscache_dispatcher(void *data)
+{
+ struct completion *done = data;
+
+ for (;;) {
+ if (!list_empty(&fscache_pending_work)) {
+ struct fscache_work *work = NULL;
+
+ spin_lock(&fscache_work_lock);
+ if (!list_empty(&fscache_pending_work)) {
+ work = list_entry(fscache_pending_work.next,
+ struct fscache_work, link);
+ list_del_init(&work->link);
+ }
+ spin_unlock(&fscache_work_lock);
+
+ if (work) {
+ work->func(work->cookie, work->object, work->param);
+ fscache_cookie_put(work->cookie, fscache_cookie_put_work);
+ kfree(work);
+ }
+ continue;
+ } else if (fscache_dispatcher_stop) {
+ break;
+ }
+
+ wait_event_freezable(fscache_dispatcher_pool,
+ (fscache_dispatcher_stop ||
+ !list_empty(&fscache_pending_work)));
+ }
+
+ complete_and_exit(done, 0);
+}
+
+/*
+ * Start up the dispatcher threads.
+ */
+int fscache_init_dispatchers(void)
+{
+ struct task_struct *t;
+ int i;
+
+ for (i = 0; i < FSCACHE_DISPATCHER_POOL_SIZE; i++) {
+ t = kthread_create(fscache_dispatcher,
+ &fscache_dispatcher_pool_done[i],
+ "kfsc/%d", i);
+ if (IS_ERR(t))
+ goto failed;
+ wake_up_process(t);
+ }
+
+ return 0;
+
+failed:
+ fscache_dispatcher_stop = true;
+ wake_up_all(&fscache_dispatcher_pool);
+ for (i--; i >= 0; i--)
+ wait_for_completion(&fscache_dispatcher_pool_done[i]);
+ return PTR_ERR(t);
+}
+
+/*
+ * Kill off the dispatcher threads.
+ */
+void fscache_kill_dispatchers(void)
+{
+ int i;
+
+ fscache_dispatcher_stop = true;
+ wake_up_all(&fscache_dispatcher_pool);
+
+ for (i = 0; i < FSCACHE_DISPATCHER_POOL_SIZE; i++)
+ wait_for_completion(&fscache_dispatcher_pool_done[i]);
+}
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index bc5539d2157b..2100e2222884 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -75,6 +75,14 @@ extern struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *);
extern void fscache_cookie_put(struct fscache_cookie *,
enum fscache_cookie_trace);
+/*
+ * dispatcher.c
+ */
+extern void fscache_dispatch(struct fscache_cookie *, struct fscache_object *, int,
+ void (*func)(struct fscache_cookie *, struct fscache_object *, int));
+extern int fscache_init_dispatchers(void);
+extern void fscache_kill_dispatchers(void);
+
/*
* fsdef.c
*/
diff --git a/fs/fscache/main.c b/fs/fscache/main.c
index 59c2494efda3..6f225ae0fd99 100644
--- a/fs/fscache/main.c
+++ b/fs/fscache/main.c
@@ -126,6 +126,10 @@ static int __init fscache_init(void)
for_each_possible_cpu(cpu)
init_waitqueue_head(&per_cpu(fscache_object_cong_wait, cpu));
+ ret = fscache_init_dispatchers();
+ if (ret < 0)
+ goto error_dispatchers;
+
ret = fscache_proc_init();
if (ret < 0)
goto error_proc;
@@ -160,6 +164,8 @@ static int __init fscache_init(void)
unregister_sysctl_table(fscache_sysctl_header);
error_sysctl:
#endif
+ fscache_kill_dispatchers();
+error_dispatchers:
fscache_proc_cleanup();
error_proc:
destroy_workqueue(fscache_op_wq);
@@ -184,6 +190,7 @@ static void __exit fscache_exit(void)
unregister_sysctl_table(fscache_sysctl_header);
#endif
fscache_proc_cleanup();
+ fscache_kill_dispatchers();
destroy_workqueue(fscache_op_wq);
destroy_workqueue(fscache_object_wq);
pr_notice("Unloaded\n");
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 08d7de72409d..fb3fdf2921ee 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -26,11 +26,13 @@ enum fscache_cookie_trace {
fscache_cookie_get_attach_object,
fscache_cookie_get_reacquire,
fscache_cookie_get_register_netfs,
+ fscache_cookie_get_work,
fscache_cookie_put_acquire_nobufs,
fscache_cookie_put_dup_netfs,
fscache_cookie_put_relinquish,
fscache_cookie_put_object,
fscache_cookie_put_parent,
+ fscache_cookie_put_work,
};
#endif
@@ -45,11 +47,13 @@ enum fscache_cookie_trace {
EM(fscache_cookie_get_attach_object, "GET obj") \
EM(fscache_cookie_get_reacquire, "GET raq") \
EM(fscache_cookie_get_register_netfs, "GET net") \
+ EM(fscache_cookie_get_work, "GET wrk") \
EM(fscache_cookie_put_acquire_nobufs, "PUT nbf") \
EM(fscache_cookie_put_dup_netfs, "PUT dnt") \
EM(fscache_cookie_put_relinquish, "PUT rlq") \
EM(fscache_cookie_put_object, "PUT obj") \
- E_(fscache_cookie_put_parent, "PUT prn")
+ EM(fscache_cookie_put_parent, "PUT prn") \
+ E_(fscache_cookie_put_work, "PUT wrk")
/*
* Export enum symbols via userspace.
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 21/61] fscache: Provide a simple thread pool for running ops asynchronously
@ 2020-05-04 17:10 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Provide a simple thread pool that can be used to run cookie management
operations in the background and a dispatcher infrastructure to punt
operations to the pool if threads are available or to just run the
operation in the calling thread if not.
A future patch will replace all the object state machine stuff with whole
routines that do all the work in one go without trying to interleave bits
from various objects.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/fscache/Makefile | 1
fs/fscache/dispatcher.c | 144 ++++++++++++++++++++++++++++++++++++++++
fs/fscache/internal.h | 8 ++
fs/fscache/main.c | 7 ++
include/trace/events/fscache.h | 6 +-
5 files changed, 165 insertions(+), 1 deletion(-)
create mode 100644 fs/fscache/dispatcher.c
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index ac3fcd909fff..7b10c6aad157 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -6,6 +6,7 @@
fscache-y := \
cache.o \
cookie.o \
+ dispatcher.o \
fsdef.o \
main.o \
netfs.o \
diff --git a/fs/fscache/dispatcher.c b/fs/fscache/dispatcher.c
new file mode 100644
index 000000000000..fba71b99c951
--- /dev/null
+++ b/fs/fscache/dispatcher.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Object dispatcher
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/kthread.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+#include <linux/freezer.h>
+#include "internal.h"
+
+#define FSCACHE_DISPATCHER_POOL_SIZE 8
+
+static LIST_HEAD(fscache_pending_work);
+static DEFINE_SPINLOCK(fscache_work_lock);
+static DECLARE_WAIT_QUEUE_HEAD(fscache_dispatcher_pool);
+static struct completion fscache_dispatcher_pool_done[FSCACHE_DISPATCHER_POOL_SIZE];
+static bool fscache_dispatcher_stop;
+
+struct fscache_work {
+ struct list_head link;
+ struct fscache_cookie *cookie;
+ struct fscache_object *object;
+ int param;
+ void (*func)(struct fscache_cookie *, struct fscache_object *, int);
+};
+
+/*
+ * Attempt to queue some work to do. If there's too much asynchronous work
+ * already queued, we'll do it here in this thread instead.
+ */
+void fscache_dispatch(struct fscache_cookie *cookie,
+ struct fscache_object *object,
+ int param,
+ void (*func)(struct fscache_cookie *,
+ struct fscache_object *, int))
+{
+ struct fscache_work *work;
+ bool queued = false;
+
+ work = kzalloc(sizeof(struct fscache_work), GFP_KERNEL);
+ if (work) {
+ work->cookie = cookie;
+ work->object = object;
+ work->param = param;
+ work->func = func;
+
+ spin_lock(&fscache_work_lock);
+ if (waitqueue_active(&fscache_dispatcher_pool) ||
+ list_empty(&fscache_pending_work)) {
+ fscache_cookie_get(cookie, fscache_cookie_get_work);
+ list_add_tail(&work->link, &fscache_pending_work);
+ wake_up(&fscache_dispatcher_pool);
+ queued = true;
+ }
+ spin_unlock(&fscache_work_lock);
+ }
+
+ if (!queued) {
+ kfree(work);
+ func(cookie, object, param);
+ }
+}
+
+/*
+ * A dispatcher thread.
+ */
+static int fscache_dispatcher(void *data)
+{
+ struct completion *done = data;
+
+ for (;;) {
+ if (!list_empty(&fscache_pending_work)) {
+ struct fscache_work *work = NULL;
+
+ spin_lock(&fscache_work_lock);
+ if (!list_empty(&fscache_pending_work)) {
+ work = list_entry(fscache_pending_work.next,
+ struct fscache_work, link);
+ list_del_init(&work->link);
+ }
+ spin_unlock(&fscache_work_lock);
+
+ if (work) {
+ work->func(work->cookie, work->object, work->param);
+ fscache_cookie_put(work->cookie, fscache_cookie_put_work);
+ kfree(work);
+ }
+ continue;
+ } else if (fscache_dispatcher_stop) {
+ break;
+ }
+
+ wait_event_freezable(fscache_dispatcher_pool,
+ (fscache_dispatcher_stop ||
+ !list_empty(&fscache_pending_work)));
+ }
+
+ complete_and_exit(done, 0);
+}
+
+/*
+ * Start up the dispatcher threads.
+ */
+int fscache_init_dispatchers(void)
+{
+ struct task_struct *t;
+ int i;
+
+ for (i = 0; i < FSCACHE_DISPATCHER_POOL_SIZE; i++) {
+ t = kthread_create(fscache_dispatcher,
+ &fscache_dispatcher_pool_done[i],
+ "kfsc/%d", i);
+ if (IS_ERR(t))
+ goto failed;
+ wake_up_process(t);
+ }
+
+ return 0;
+
+failed:
+ fscache_dispatcher_stop = true;
+ wake_up_all(&fscache_dispatcher_pool);
+ for (i--; i >= 0; i--)
+ wait_for_completion(&fscache_dispatcher_pool_done[i]);
+ return PTR_ERR(t);
+}
+
+/*
+ * Kill off the dispatcher threads.
+ */
+void fscache_kill_dispatchers(void)
+{
+ int i;
+
+ fscache_dispatcher_stop = true;
+ wake_up_all(&fscache_dispatcher_pool);
+
+ for (i = 0; i < FSCACHE_DISPATCHER_POOL_SIZE; i++)
+ wait_for_completion(&fscache_dispatcher_pool_done[i]);
+}
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index bc5539d2157b..2100e2222884 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -75,6 +75,14 @@ extern struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *);
extern void fscache_cookie_put(struct fscache_cookie *,
enum fscache_cookie_trace);
+/*
+ * dispatcher.c
+ */
+extern void fscache_dispatch(struct fscache_cookie *, struct fscache_object *, int,
+ void (*func)(struct fscache_cookie *, struct fscache_object *, int));
+extern int fscache_init_dispatchers(void);
+extern void fscache_kill_dispatchers(void);
+
/*
* fsdef.c
*/
diff --git a/fs/fscache/main.c b/fs/fscache/main.c
index 59c2494efda3..6f225ae0fd99 100644
--- a/fs/fscache/main.c
+++ b/fs/fscache/main.c
@@ -126,6 +126,10 @@ static int __init fscache_init(void)
for_each_possible_cpu(cpu)
init_waitqueue_head(&per_cpu(fscache_object_cong_wait, cpu));
+ ret = fscache_init_dispatchers();
+ if (ret < 0)
+ goto error_dispatchers;
+
ret = fscache_proc_init();
if (ret < 0)
goto error_proc;
@@ -160,6 +164,8 @@ static int __init fscache_init(void)
unregister_sysctl_table(fscache_sysctl_header);
error_sysctl:
#endif
+ fscache_kill_dispatchers();
+error_dispatchers:
fscache_proc_cleanup();
error_proc:
destroy_workqueue(fscache_op_wq);
@@ -184,6 +190,7 @@ static void __exit fscache_exit(void)
unregister_sysctl_table(fscache_sysctl_header);
#endif
fscache_proc_cleanup();
+ fscache_kill_dispatchers();
destroy_workqueue(fscache_op_wq);
destroy_workqueue(fscache_object_wq);
pr_notice("Unloaded\n");
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 08d7de72409d..fb3fdf2921ee 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -26,11 +26,13 @@ enum fscache_cookie_trace {
fscache_cookie_get_attach_object,
fscache_cookie_get_reacquire,
fscache_cookie_get_register_netfs,
+ fscache_cookie_get_work,
fscache_cookie_put_acquire_nobufs,
fscache_cookie_put_dup_netfs,
fscache_cookie_put_relinquish,
fscache_cookie_put_object,
fscache_cookie_put_parent,
+ fscache_cookie_put_work,
};
#endif
@@ -45,11 +47,13 @@ enum fscache_cookie_trace {
EM(fscache_cookie_get_attach_object, "GET obj") \
EM(fscache_cookie_get_reacquire, "GET raq") \
EM(fscache_cookie_get_register_netfs, "GET net") \
+ EM(fscache_cookie_get_work, "GET wrk") \
EM(fscache_cookie_put_acquire_nobufs, "PUT nbf") \
EM(fscache_cookie_put_dup_netfs, "PUT dnt") \
EM(fscache_cookie_put_relinquish, "PUT rlq") \
EM(fscache_cookie_put_object, "PUT obj") \
- E_(fscache_cookie_put_parent, "PUT prn")
+ EM(fscache_cookie_put_parent, "PUT prn") \
+ E_(fscache_cookie_put_work, "PUT wrk")
/*
* Export enum symbols via userspace.
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 22/61] fscache: Replace the object management state machine
2020-05-04 17:07 ` David Howells
` (21 preceding siblings ...)
(?)
@ 2020-05-04 17:10 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Replace the object management state machine with something a lot simpler.
The entire process of setting up or tearing down a cookie is done in one
go, and the dispatcher either punts it to a worker thread, or if all the
worker threads are all busy, does it in the current thread.
fscache_enable_cookie() and fscache_disable_cookie() are replaced by 'use'
and 'unuse' routines to which the mode of access (readonly or writable) is
declared - these then impose the policy of what to do with the backing
object.
The policy for handling local writes is declared to
fscache_acquire_cookie() using FSACHE_ADV_WRITE_*CACHE flags. This only
allows for the possibility of suspending caching whilst a file is open for
writing; policies such as write-through and write-back have to be handled
at the netfs level.
At some point in the future, object records that aren't in use will get put
on an LRU and discarded under memory pressure or if they haven't been used
for a while.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/cell.c | 4
fs/afs/file.c | 15 +
fs/afs/inode.c | 13
fs/afs/volume.c | 4
fs/cachefiles/bind.c | 4
fs/cachefiles/interface.c | 156 +++---
fs/cachefiles/internal.h | 12
fs/cachefiles/main.c | 11
fs/cachefiles/namei.c | 58 +-
fs/cachefiles/xattr.c | 1
fs/fscache/Makefile | 3
fs/fscache/cache.c | 107 ++--
fs/fscache/cookie.c | 485 +++++-------------
fs/fscache/fsdef.c | 1
fs/fscache/histogram.c | 2
fs/fscache/internal.h | 53 +-
fs/fscache/main.c | 29 -
fs/fscache/netfs.c | 4
fs/fscache/obj.c | 349 +++++++++++++
fs/fscache/object-list.c | 58 --
fs/fscache/object.c | 982 -------------------------------------
fs/fscache/object_bits.c | 120 +++++
fs/fscache/proc.c | 6
fs/fscache/stats.c | 2
include/linux/fscache-cache.h | 208 ++------
include/linux/fscache.h | 173 +++----
include/trace/events/cachefiles.h | 17 -
include/trace/events/fscache.h | 133 +----
28 files changed, 1015 insertions(+), 1995 deletions(-)
create mode 100644 fs/fscache/obj.c
delete mode 100644 fs/fscache/object.c
create mode 100644 fs/fscache/object_bits.c
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index 31877638f673..40491edefd69 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -598,7 +598,7 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
NULL,
cell->name, strlen(cell->name),
NULL, 0,
- 0, true);
+ 0);
#endif
ret = afs_proc_cell_setup(cell);
if (ret < 0)
@@ -637,7 +637,7 @@ static void afs_deactivate_cell(struct afs_net *net, struct afs_cell *cell)
mutex_unlock(&net->proc_cells_lock);
#ifdef CONFIG_AFS_FSCACHE
- fscache_relinquish_cookie(cell->cache, NULL, false);
+ fscache_relinquish_cookie(cell->cache, false);
cell->cache = NULL;
#endif
diff --git a/fs/afs/file.c b/fs/afs/file.c
index f0cca37f2da8..39db1e7b186d 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -145,7 +145,9 @@ int afs_open(struct inode *inode, struct file *file)
if (file->f_flags & O_TRUNC)
set_bit(AFS_VNODE_NEW_CONTENT, &vnode->flags);
-
+
+ fscache_use_cookie(afs_vnode_cache(vnode), file->f_mode & FMODE_WRITE);
+
file->private_data = af;
_leave(" = 0");
return 0;
@@ -164,8 +166,10 @@ int afs_open(struct inode *inode, struct file *file)
*/
int afs_release(struct inode *inode, struct file *file)
{
+ struct afs_vnode_cache_aux aux;
struct afs_vnode *vnode = AFS_FS_I(inode);
struct afs_file *af = file->private_data;
+ loff_t i_size;
int ret = 0;
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
@@ -176,6 +180,15 @@ int afs_release(struct inode *inode, struct file *file)
file->private_data = NULL;
if (af->wb)
afs_put_wb_key(af->wb);
+
+ if ((file->f_mode & FMODE_WRITE)) {
+ i_size = i_size_read(&vnode->vfs_inode);
+ aux.data_version = vnode->status.data_version;
+ fscache_unuse_cookie(afs_vnode_cache(vnode), &aux, &i_size);
+ } else {
+ fscache_unuse_cookie(afs_vnode_cache(vnode), NULL, NULL);
+ }
+
key_put(af->key);
kfree(af);
afs_prune_wb_keys(vnode);
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index cf843af5f65e..9485a5afa13d 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -498,7 +498,7 @@ static void afs_get_inode_cache(struct afs_vnode *vnode)
NULL,
&key, sizeof(key),
&aux, sizeof(aux),
- vnode->status.size, true);
+ vnode->status.size);
#endif
}
@@ -793,14 +793,9 @@ void afs_evict_inode(struct inode *inode)
}
#ifdef CONFIG_AFS_FSCACHE
- {
- struct afs_vnode_cache_aux aux;
-
- aux.data_version = vnode->status.data_version;
- fscache_relinquish_cookie(vnode->cache, &aux,
- test_bit(AFS_VNODE_DELETED, &vnode->flags));
- vnode->cache = NULL;
- }
+ fscache_relinquish_cookie(vnode->cache,
+ test_bit(AFS_VNODE_DELETED, &vnode->flags));
+ vnode->cache = NULL;
#endif
afs_prune_wb_keys(vnode);
diff --git a/fs/afs/volume.c b/fs/afs/volume.c
index 5fefb0ba85bf..83ea0fa45d80 100644
--- a/fs/afs/volume.c
+++ b/fs/afs/volume.c
@@ -187,7 +187,7 @@ void afs_activate_volume(struct afs_volume *volume)
"AFS.vol",
0, NULL,
&volume->vid, sizeof(volume->vid),
- NULL, 0, 0, true);
+ NULL, 0, 0);
#endif
write_lock(&volume->cell->proc_lock);
@@ -207,7 +207,7 @@ void afs_deactivate_volume(struct afs_volume *volume)
write_unlock(&volume->cell->proc_lock);
#ifdef CONFIG_AFS_FSCACHE
- fscache_relinquish_cookie(volume->cache, NULL,
+ fscache_relinquish_cookie(volume->cache,
test_bit(AFS_VOLUME_DELETED, &volume->flags));
volume->cache = NULL;
#endif
diff --git a/fs/cachefiles/bind.c b/fs/cachefiles/bind.c
index dfb14dbddf51..4c59e1ef4500 100644
--- a/fs/cachefiles/bind.c
+++ b/fs/cachefiles/bind.c
@@ -99,12 +99,10 @@ static int cachefiles_daemon_add_cache(struct cachefiles_cache *cache)
/* allocate the root index object */
ret = -ENOMEM;
- fsdef = kmem_cache_alloc(cachefiles_object_jar, GFP_KERNEL);
+ fsdef = kmem_cache_zalloc(cachefiles_object_jar, GFP_KERNEL);
if (!fsdef)
goto error_root_object;
- ASSERTCMP(fsdef->backer, ==, NULL);
-
atomic_set(&fsdef->usage, 1);
fsdef->type = FSCACHE_COOKIE_TYPE_INDEX;
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index b868afb970ad..daa4f316d104 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -8,39 +8,56 @@
#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/xattr.h>
+#include <linux/file.h>
#include "internal.h"
static int cachefiles_attr_changed(struct cachefiles_object *object);
+static void cachefiles_put_object(struct fscache_object *_object,
+ enum fscache_obj_ref_trace why);
/*
- * allocate an object record for a cookie lookup and prepare the lookup data
+ * Allocate an object record for a cookie lookup and prepare the lookup data.
+ * Eats the caller's ref on parent.
*/
-static struct fscache_object *cachefiles_alloc_object(
- struct fscache_cache *_cache,
- struct fscache_cookie *cookie)
+static
+struct fscache_object *cachefiles_alloc_object(struct fscache_cookie *cookie,
+ struct fscache_cache *_cache,
+ struct fscache_object *parent)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
- unsigned keylen;
- void *buffer, *p;
- char *key;
cache = container_of(_cache, struct cachefiles_cache, cache);
_enter("{%s},%p,", cache->cache.identifier, cookie);
- /* create a new object record and a temporary leaf image */
- object = kmem_cache_alloc(cachefiles_object_jar, cachefiles_gfp);
- if (!object)
- goto nomem_object;
-
- ASSERTCMP(object->backer, ==, NULL);
-
- atomic_set(&object->usage, 1);
+ object = kmem_cache_zalloc(cachefiles_object_jar, cachefiles_gfp);
+ if (!object) {
+ cachefiles_put_object(parent, fscache_obj_put_alloc_fail);
+ return NULL;
+ }
fscache_object_init(&object->fscache, cookie, &cache->cache);
+ object->fscache.parent = parent;
+ object->fscache.stage = FSCACHE_OBJECT_STAGE_LOOKING_UP;
+ atomic_set(&object->usage, 1);
object->type = cookie->type;
+ trace_cachefiles_ref(object, cookie,
+ (enum cachefiles_obj_ref_trace)fscache_obj_new, 1);
+ return &object->fscache;
+}
+
+/*
+ * Prepare data for use in lookup. This involves cooking the binary key into
+ * something that can be used as a filename.
+ */
+static void *cachefiles_prepare_lookup_data(struct fscache_object *object)
+{
+ struct fscache_cookie *cookie = object->cookie;
+ unsigned keylen;
+ void *buffer, *p;
+ char *key;
/* get hold of the raw key
* - stick the length on the front and leave space on the back for the
@@ -48,7 +65,7 @@ static struct fscache_object *cachefiles_alloc_object(
*/
buffer = kmalloc((2 + 512) + 3, cachefiles_gfp);
if (!buffer)
- goto nomem_buffer;
+ goto nomem;
keylen = cookie->key_len;
p = fscache_get_key(cookie);
@@ -60,36 +77,29 @@ static struct fscache_object *cachefiles_alloc_object(
((char *)buffer)[keylen + 4] = 0;
/* turn the raw key into something that can work with as a filename */
- key = cachefiles_cook_key(buffer, keylen + 2, object->type);
+ key = cachefiles_cook_key(buffer, keylen + 2, cookie->type);
kfree(buffer);
if (!key)
- goto nomem_key;
-
- object->lookup_key = key;
+ goto nomem;
- _leave(" = %p [%s]", &object->fscache, key);
- return &object->fscache;
+ _leave(" = %s", key);
+ return key;
-nomem_key:
- kfree(buffer);
-nomem_buffer:
- kmem_cache_free(cachefiles_object_jar, object);
- fscache_object_destroyed(&cache->cache);
-nomem_object:
- _leave(" = -ENOMEM");
+nomem:
return ERR_PTR(-ENOMEM);
}
/*
- * attempt to look up the nominated node in this cache
- * - return -ETIMEDOUT to be scheduled again
+ * Attempt to look up the nominated node in this cache
*/
-static int cachefiles_lookup_object(struct fscache_object *_object)
+static bool cachefiles_lookup_object(struct fscache_object *_object,
+ void *lookup_data)
{
struct cachefiles_object *parent, *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
- int ret;
+ char *lookup_key = lookup_data;
+ bool success;
_enter("{OBJ%x}", _object->debug_id);
@@ -98,47 +108,30 @@ static int cachefiles_lookup_object(struct fscache_object *_object)
struct cachefiles_object, fscache);
object = container_of(_object, struct cachefiles_object, fscache);
- ASSERTCMP(object->lookup_key, !=, NULL);
+ ASSERTCMP(lookup_key, !=, NULL);
/* look up the key, creating any missing bits */
cachefiles_begin_secure(cache, &saved_cred);
- ret = cachefiles_walk_to_object(parent, object, object->lookup_key);
+ success = cachefiles_walk_to_object(parent, object, lookup_key);
cachefiles_end_secure(cache, saved_cred);
/* polish off by setting the attributes of non-index files */
- if (ret == 0 &&
+ if (success &&
object->fscache.cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
cachefiles_attr_changed(object);
- if (ret < 0 && ret != -ETIMEDOUT) {
- if (ret != -ENOBUFS)
- pr_warn("Lookup failed error %d\n", ret);
- fscache_object_lookup_error(&object->fscache);
- }
-
- _leave(" [%d]", ret);
- return ret;
+ _leave(" [%d]", success);
+ return success;
}
-/*
- * indication of lookup completion
- */
-static void cachefiles_lookup_complete(struct fscache_object *_object)
+static void cachefiles_free_lookup_data(struct fscache_object *object, void *lookup_data)
{
- struct cachefiles_object *object;
-
- object = container_of(_object, struct cachefiles_object, fscache);
-
- _enter("{OBJ%x}", object->fscache.debug_id);
-
- kfree(object->lookup_key);
- object->lookup_key = NULL;
+ kfree(lookup_data);
}
/*
* increment the usage count on an inode object (may fail if unmounting)
*/
-static
struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
enum fscache_obj_ref_trace why)
{
@@ -166,6 +159,8 @@ static void cachefiles_update_object(struct fscache_object *_object)
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
+ loff_t object_size;
+ int ret;
_enter("{OBJ%x}", _object->debug_id);
@@ -174,7 +169,25 @@ static void cachefiles_update_object(struct fscache_object *_object)
cache);
cachefiles_begin_secure(cache, &saved_cred);
+
+ object_size = object->fscache.cookie->object_size;
+ if (i_size_read(d_inode(object->backer)) > object_size) {
+ struct path path = {
+ .mnt = cache->mnt,
+ .dentry = object->backer
+ };
+ _debug("trunc %llx -> %llx", i_size_read(d_inode(object->backer)), object_size);
+ ret = vfs_truncate(&path, object_size);
+ if (ret < 0) {
+ cachefiles_io_error_obj(object, "Trunc-to-size failed");
+ cachefiles_remove_object_xattr(cache, object->backer);
+ goto out;
+ }
+ }
+
cachefiles_set_object_xattr(object, XATTR_REPLACE);
+
+out:
cachefiles_end_secure(cache, saved_cred);
_leave("");
}
@@ -183,7 +196,8 @@ static void cachefiles_update_object(struct fscache_object *_object)
* discard the resources pinned by an object and effect retirement if
* requested
*/
-static void cachefiles_drop_object(struct fscache_object *_object)
+static void cachefiles_drop_object(struct fscache_object *_object,
+ bool invalidate)
{
struct cachefiles_object *object;
struct cachefiles_cache *cache;
@@ -209,17 +223,18 @@ static void cachefiles_drop_object(struct fscache_object *_object)
* before we set it up.
*/
if (object->dentry) {
- /* delete retired objects */
- if (test_bit(FSCACHE_OBJECT_RETIRED, &object->fscache.flags) &&
- _object != cache->cache.fsdef
- ) {
- _debug("- retire object OBJ%x", object->fscache.debug_id);
+ if (invalidate && _object != cache->cache.fsdef) {
+ _debug("- inval object OBJ%x", object->fscache.debug_id);
cachefiles_begin_secure(cache, &saved_cred);
cachefiles_delete_object(cache, object);
cachefiles_end_secure(cache, saved_cred);
}
/* close the filesystem stuff attached to the object */
+ if (object->backing_file)
+ fput(object->backing_file);
+ object->backing_file = NULL;
+
if (object->backer != object->dentry)
dput(object->backer);
object->backer = NULL;
@@ -266,11 +281,8 @@ static void cachefiles_put_object(struct fscache_object *_object,
ASSERTCMP(object->fscache.parent, ==, NULL);
ASSERTCMP(object->backer, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
- ASSERTCMP(object->fscache.n_ops, ==, 0);
ASSERTCMP(object->fscache.n_children, ==, 0);
- kfree(object->lookup_key);
-
cache = object->fscache.cache;
fscache_object_destroy(&object->fscache);
kmem_cache_free(cachefiles_object_jar, object);
@@ -416,15 +428,25 @@ static void cachefiles_invalidate_object(struct fscache_object *_object)
_leave("");
}
+static unsigned int cachefiles_get_object_usage(const struct fscache_object *_object)
+{
+ struct cachefiles_object *object;
+
+ object = container_of(_object, struct cachefiles_object, fscache);
+ return atomic_read(&object->usage);
+}
+
const struct fscache_cache_ops cachefiles_cache_ops = {
.name = "cachefiles",
.alloc_object = cachefiles_alloc_object,
+ .prepare_lookup_data = cachefiles_prepare_lookup_data,
.lookup_object = cachefiles_lookup_object,
- .lookup_complete = cachefiles_lookup_complete,
+ .free_lookup_data = cachefiles_free_lookup_data,
.grab_object = cachefiles_grab_object,
.update_object = cachefiles_update_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
.put_object = cachefiles_put_object,
+ .get_object_usage = cachefiles_get_object_usage,
.sync_cache = cachefiles_sync_cache,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index f8f308ce7385..b89f76a03546 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -34,13 +34,13 @@ extern unsigned cachefiles_debug;
*/
struct cachefiles_object {
struct fscache_object fscache; /* fscache handle */
- char *lookup_key; /* key to look up */
struct dentry *dentry; /* the file/dir representing this object */
struct dentry *backer; /* backing file */
+ struct file *backing_file; /* File open on backing storage */
loff_t i_size; /* object size */
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
- uint8_t new; /* T if object new */
+ bool new; /* T if object new */
};
extern struct kmem_cache *cachefiles_object_jar;
@@ -112,6 +112,8 @@ extern int cachefiles_has_space(struct cachefiles_cache *cache,
* interface.c
*/
extern const struct fscache_cache_ops cachefiles_cache_ops;
+extern struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
+ enum fscache_obj_ref_trace why);
/*
* key.c
@@ -125,9 +127,9 @@ extern void cachefiles_unmark_inode_in_use(struct cachefiles_object *object,
struct dentry *dentry);
extern int cachefiles_delete_object(struct cachefiles_cache *cache,
struct cachefiles_object *object);
-extern int cachefiles_walk_to_object(struct cachefiles_object *parent,
- struct cachefiles_object *object,
- const char *key);
+extern bool cachefiles_walk_to_object(struct cachefiles_object *parent,
+ struct cachefiles_object *object,
+ const char *key);
extern struct dentry *cachefiles_get_directory(struct cachefiles_cache *cache,
struct dentry *dir,
const char *name);
diff --git a/fs/cachefiles/main.c b/fs/cachefiles/main.c
index 3f0101a74809..3373f3eddbf8 100644
--- a/fs/cachefiles/main.c
+++ b/fs/cachefiles/main.c
@@ -37,13 +37,6 @@ static struct miscdevice cachefiles_dev = {
.fops = &cachefiles_daemon_fops,
};
-static void cachefiles_object_init_once(void *_object)
-{
- struct cachefiles_object *object = _object;
-
- memset(object, 0, sizeof(*object));
-}
-
/*
* initialise the fs caching module
*/
@@ -60,9 +53,7 @@ static int __init cachefiles_init(void)
cachefiles_object_jar =
kmem_cache_create("cachefiles_object_jar",
sizeof(struct cachefiles_object),
- 0,
- SLAB_HWCACHE_ALIGN,
- cachefiles_object_init_once);
+ 0, SLAB_HWCACHE_ALIGN, NULL);
if (!cachefiles_object_jar) {
pr_notice("Failed to allocate an object jar\n");
goto error_object_jar;
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 818d1bca1904..ad7edd41ce27 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -18,8 +18,6 @@
#include <linux/slab.h>
#include "internal.h"
-#define CACHEFILES_KEYBUF_SIZE 512
-
/*
* Mark the backing file as being a cache file if it's not already in use so.
*/
@@ -257,13 +255,24 @@ int cachefiles_delete_object(struct cachefiles_cache *cache,
return ret;
}
+/*
+ * Handle negative lookup. We create a temporary file for a datafile to use
+ * which we will link into place later.
+ */
+static void cachefiles_negative_lookup(struct cachefiles_object *parent,
+ struct cachefiles_object *object)
+{
+ // TODO: Use vfs_tmpfile() for datafiles
+ //fscache_object_lookup_negative(&object->fscache);
+}
+
/*
* walk from the parent object to the child object through the backing
* filesystem, creating directories as we go
*/
-int cachefiles_walk_to_object(struct cachefiles_object *parent,
- struct cachefiles_object *object,
- const char *key)
+bool cachefiles_walk_to_object(struct cachefiles_object *parent,
+ struct cachefiles_object *object,
+ const char *key)
{
struct cachefiles_cache *cache;
struct dentry *dir, *next = NULL;
@@ -271,7 +280,7 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
struct path path;
unsigned long start;
const char *name;
- bool marked = false;
+ bool marked = false, negated = false;
int ret, nlen;
_enter("OBJ%x{%p},OBJ%x,%s,",
@@ -328,8 +337,10 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
/* if this element of the path doesn't exist, then the lookup phase
* failed, and we can release any readers in the certain knowledge that
* there's nothing for them to actually read */
- if (d_is_negative(next))
- fscache_object_lookup_negative(&object->fscache);
+ if (d_is_negative(next) && !negated) {
+ cachefiles_negative_lookup(object, parent);
+ negated = true;
+ }
/* we need to create the object if it's negative */
if (key || object->type == FSCACHE_COOKIE_TYPE_INDEX) {
@@ -444,6 +455,9 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
fscache_object_retrying_stale(&object->fscache);
goto lookup_again;
}
+
+ if (ret < 0)
+ goto check_error_unlock;
}
inode_unlock(d_inode(dir));
@@ -469,14 +483,10 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
/* open a file interface onto a data file */
if (object->type != FSCACHE_COOKIE_TYPE_INDEX) {
if (d_is_reg(object->dentry)) {
- const struct address_space_operations *aops;
-
- ret = -EPERM;
- aops = d_backing_inode(object->dentry)->i_mapping->a_ops;
- if (!aops->bmap)
- goto check_error;
- if (object->dentry->d_sb->s_blocksize > PAGE_SIZE)
+ if (object->dentry->d_sb->s_blocksize > PAGE_SIZE) {
+ pr_warn("cachefiles: Block size too large\n");
goto check_error;
+ }
object->backer = object->dentry;
} else {
@@ -484,11 +494,15 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
}
}
- object->new = 0;
- fscache_obtained_object(&object->fscache);
+ if (object->new)
+ object->fscache.stage = FSCACHE_OBJECT_STAGE_LIVE_EMPTY;
+ else
+ object->fscache.stage = FSCACHE_OBJECT_STAGE_LIVE;
+ wake_up_var(&object->fscache.stage);
- _leave(" = 0 [%lu]", d_backing_inode(object->dentry)->i_ino);
- return 0;
+ object->new = false;
+ _leave(" = t [%lu]", d_backing_inode(object->dentry)->i_ino);
+ return true;
no_space_error:
fscache_object_mark_killed(&object->fscache, FSCACHE_OBJECT_NO_SPACE);
@@ -520,8 +534,10 @@ int cachefiles_walk_to_object(struct cachefiles_object *parent,
error_out2:
dput(dir);
error_out:
- _leave(" = error %d", -ret);
- return ret;
+ object->fscache.stage = FSCACHE_OBJECT_STAGE_DEAD;
+ wake_up_var(&object->fscache.stage);
+ _leave(" = f");
+ return false;
}
/*
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index a4f1eddebe6f..5b2f6da91cc8 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -122,7 +122,6 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
- clear_bit(FSCACHE_COOKIE_AUX_UPDATED, &object->fscache.cookie->flags);
ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
buf, sizeof(struct cachefiles_xattr) + len,
xattr_flags);
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 7b10c6aad157..396e1b5fdc28 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -10,7 +10,8 @@ fscache-y := \
fsdef.o \
main.o \
netfs.o \
- object.o
+ obj.o \
+ object_bits.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/cache.c b/fs/fscache/cache.c
index bea53cd92a45..8240dea803d2 100644
--- a/fs/fscache/cache.c
+++ b/fs/fscache/cache.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* FS-Cache cache handling
*
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2007, 2019 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
@@ -93,6 +93,7 @@ struct fscache_cache *fscache_select_cache_for_object(
struct fscache_cookie *cookie)
{
struct fscache_cache_tag *tag;
+ struct fscache_cookie *parent = cookie->parent;
struct fscache_object *object;
struct fscache_cache *cache;
@@ -104,33 +105,32 @@ struct fscache_cache *fscache_select_cache_for_object(
}
/* we check the parent to determine the cache to use */
- spin_lock(&cookie->lock);
+ spin_lock(&parent->lock);
/* the first in the parent's backing list should be the preferred
* cache */
- if (!hlist_empty(&cookie->backing_objects)) {
- object = hlist_entry(cookie->backing_objects.first,
+ if (!hlist_empty(&parent->backing_objects)) {
+ object = hlist_entry(parent->backing_objects.first,
struct fscache_object, cookie_link);
cache = object->cache;
- if (fscache_object_is_dying(object) ||
- test_bit(FSCACHE_IOERROR, &cache->flags))
+ if (test_bit(FSCACHE_IOERROR, &cache->flags))
cache = NULL;
- spin_unlock(&cookie->lock);
+ spin_unlock(&parent->lock);
_leave(" = %p [parent]", cache);
return cache;
}
/* the parent is unbacked */
- if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX) {
+ if (parent->type != FSCACHE_COOKIE_TYPE_INDEX) {
/* cookie not an index and is unbacked */
- spin_unlock(&cookie->lock);
- _leave(" = NULL [cookie ub,ni]");
+ spin_unlock(&parent->lock);
+ _leave(" = NULL [parent ub,ni]");
return NULL;
}
- spin_unlock(&cookie->lock);
+ spin_unlock(&parent->lock);
tag = cookie->preferred_cache;
if (!tag)
@@ -210,10 +210,7 @@ int fscache_add_cache(struct fscache_cache *cache,
BUG_ON(!ifsdef);
cache->flags = 0;
- ifsdef->event_mask =
- ((1 << NR_FSCACHE_OBJECT_EVENTS) - 1) &
- ~(1 << FSCACHE_OBJECT_EV_CLEARED);
- __set_bit(FSCACHE_OBJECT_IS_AVAILABLE, &ifsdef->flags);
+ ifsdef->stage = FSCACHE_OBJECT_STAGE_LIVE;
if (!tagname)
tagname = cache->identifier;
@@ -308,33 +305,57 @@ void fscache_io_error(struct fscache_cache *cache)
EXPORT_SYMBOL(fscache_io_error);
/*
- * request withdrawal of all the objects in a cache
- * - all the objects being withdrawn are moved onto the supplied list
+ * Withdraw an object.
*/
-static void fscache_withdraw_all_objects(struct fscache_cache *cache,
- struct list_head *dying_objects)
+static void fscache_withdraw_object(struct fscache_cookie *cookie,
+ struct fscache_object *object,
+ int param)
{
- struct fscache_object *object;
+ _enter("c=%08x o=%08x", cookie ? cookie->debug_id : 0, object->debug_id);
- while (!list_empty(&cache->object_list)) {
- spin_lock(&cache->object_list_lock);
+ _debug("WITHDRAW %x", object->debug_id);
- if (!list_empty(&cache->object_list)) {
- object = list_entry(cache->object_list.next,
- struct fscache_object, cache_link);
- list_move_tail(&object->cache_link, dying_objects);
+ spin_lock(&cookie->lock);
+ cookie->stage = FSCACHE_COOKIE_STAGE_DEAD;
+ hlist_del_init(&object->cookie_link);
+ spin_unlock(&cookie->lock);
- _debug("withdraw %p", object->cookie);
+ fscache_drop_object(cookie, object, param);
+ object->cache->ops->put_object(object, fscache_obj_put_withdraw);
+}
- /* This must be done under object_list_lock to prevent
- * a race with fscache_drop_object().
- */
- fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
- }
+/*
+ * Request withdrawal of all the objects in a cache.
+ */
+static void fscache_withdraw_all_objects(struct fscache_cache *cache)
+{
+ struct fscache_object *object;
+ _enter("");
+
+ spin_lock(&cache->object_list_lock);
+ while (!list_empty(&cache->object_list)) {
+ /* Go through the list backwards so that we do children before
+ * their parents.
+ */
+ object = list_entry(cache->object_list.prev,
+ struct fscache_object, cache_link);
+ list_del_init(&object->cache_link);
+ cache->ops->grab_object(object, fscache_obj_get_withdraw);
spin_unlock(&cache->object_list_lock);
+
+ _debug("o=%08x n=%u", object->debug_id, object->n_children);
+ wait_var_event(&object->n_children, object->n_children == 0);
+
+ fscache_dispatch(object->cookie, object, 0,
+ fscache_withdraw_object);
+
cond_resched();
+ spin_lock(&cache->object_list_lock);
}
+ spin_unlock(&cache->object_list_lock);
+
+ _leave("");
}
/**
@@ -349,12 +370,10 @@ static void fscache_withdraw_all_objects(struct fscache_cache *cache,
*/
void fscache_withdraw_cache(struct fscache_cache *cache)
{
- LIST_HEAD(dying_objects);
-
_enter("");
- pr_notice("Withdrawing cache \"%s\"\n",
- cache->tag->name);
+ pr_notice("Withdrawing cache \"%s\" (%u objs)\n",
+ cache->tag->name, atomic_read(&cache->object_count));
/* make the cache unavailable for cookie acquisition */
if (test_and_set_bit(FSCACHE_CACHE_WITHDRAWN, &cache->flags))
@@ -365,29 +384,27 @@ void fscache_withdraw_cache(struct fscache_cache *cache)
cache->tag->cache = NULL;
up_write(&fscache_addremove_sem);
- /* make sure all pages pinned by operations on behalf of the netfs are
- * written to disk */
- fscache_stat(&fscache_n_cop_sync_cache);
- cache->ops->sync_cache(cache);
- fscache_stat_d(&fscache_n_cop_sync_cache);
-
/* we now have to destroy all the active objects pertaining to this
* cache - which we do by passing them off to thread pool to be
* disposed of */
_debug("destroy");
- fscache_withdraw_all_objects(cache, &dying_objects);
+ fscache_withdraw_all_objects(cache);
+
+ /* make sure all outstanding data is written to disk */
+ fscache_stat(&fscache_n_cop_sync_cache);
+ cache->ops->sync_cache(cache);
+ fscache_stat_d(&fscache_n_cop_sync_cache);
/* wait for all extant objects to finish their outstanding operations
* and go away */
- _debug("wait for finish");
+ _debug("wait for finish %u", atomic_read(&cache->object_count));
wait_event(fscache_cache_cleared_wq,
atomic_read(&cache->object_count) == 0);
_debug("wait for clearance");
wait_event(fscache_cache_cleared_wq,
list_empty(&cache->object_list));
_debug("cleared");
- ASSERT(list_empty(&dying_objects));
kobject_put(cache->kobj);
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 3b450c783f03..3a1b71e09c6b 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/* netfs cookie management
*
- * Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2004-2007, 2019 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*
* See Documentation/filesystems/caching/netfs-api.txt for more information on
@@ -15,19 +15,11 @@
struct kmem_cache *fscache_cookie_jar;
-static atomic_t fscache_object_debug_id = ATOMIC_INIT(0);
-
#define fscache_cookie_hash_shift 15
static struct hlist_bl_head fscache_cookie_hash[1 << fscache_cookie_hash_shift];
static LIST_HEAD(fscache_cookies);
static DEFINE_RWLOCK(fscache_cookies_lock);
-static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie);
-static int fscache_alloc_object(struct fscache_cache *cache,
- struct fscache_cookie *cookie);
-static int fscache_attach_object(struct fscache_cookie *cookie,
- struct fscache_object *object);
-
static void fscache_print_cookie(struct fscache_cookie *cookie, char prefix)
{
struct fscache_object *object;
@@ -162,8 +154,8 @@ struct fscache_cookie *fscache_alloc_cookie(
if (!cookie)
return NULL;
- cookie->type = type;
- cookie->advice = advice;
+ cookie->type = type;
+ cookie->advice = advice;
cookie->key_len = index_key_len;
cookie->aux_len = aux_data_len;
cookie->object_size = object_size;
@@ -180,19 +172,21 @@ struct fscache_cookie *fscache_alloc_cookie(
goto nomem;
}
+ cookie->parent = parent;
atomic_set(&cookie->usage, 1);
atomic_set(&cookie->n_children, 0);
+ atomic_set(&cookie->n_ops, 1);
cookie->debug_id = atomic_inc_return(&fscache_cookie_debug_id);
+ cookie->stage = FSCACHE_COOKIE_STAGE_QUIESCENT;
+
/* We keep the active count elevated until relinquishment to prevent an
* attempt to wake up every time the object operations queue quiesces.
*/
atomic_set(&cookie->n_active, 1);
- cookie->parent = parent;
cookie->preferred_cache = fscache_get_cache_tag(preferred_cache);
-
- cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
+
spin_lock_init(&cookie->lock);
INIT_HLIST_HEAD(&cookie->backing_objects);
@@ -270,13 +264,12 @@ struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cache_tag *preferred_cache,
const void *index_key, size_t index_key_len,
const void *aux_data, size_t aux_data_len,
- loff_t object_size,
- bool enable)
+ loff_t object_size)
{
struct fscache_cookie *candidate, *cookie;
- _enter("{%s},{%s},%u",
- parent ? parent->type_name : "<no-parent>", type_name, enable);
+ _enter("{%s},{%s}",
+ parent ? parent->type_name : "<no-parent>", type_name);
if (!index_key || !index_key_len || index_key_len > 255 || aux_data_len > 255)
return NULL;
@@ -308,6 +301,7 @@ struct fscache_cookie *__fscache_acquire_cookie(
_leave(" [ENOMEM]");
return NULL;
}
+ trace_fscache_cookie(candidate, fscache_cookie_new_acquire, 1);
cookie = fscache_hash_cookie(candidate);
if (!cookie) {
@@ -331,28 +325,8 @@ struct fscache_cookie *__fscache_acquire_cookie(
}
trace_fscache_acquire(cookie);
-
- if (enable) {
- /* if the object is an index then we need do nothing more here
- * - we create indices on disk when we need them as an index
- * may exist in multiple caches */
- if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX) {
- if (fscache_acquire_non_index_cookie(cookie) == 0) {
- set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
- } else {
- atomic_dec(&parent->n_children);
- fscache_cookie_put(cookie,
- fscache_cookie_put_acquire_nobufs);
- fscache_stat(&fscache_n_acquires_nobufs);
- _leave(" = NULL");
- return NULL;
- }
- } else {
- set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
- }
- }
-
fscache_stat(&fscache_n_acquires_ok);
+ _leave(" = c=%08x", cookie->debug_id);
out:
fscache_free_cookie(candidate);
@@ -361,211 +335,80 @@ struct fscache_cookie *__fscache_acquire_cookie(
EXPORT_SYMBOL(__fscache_acquire_cookie);
/*
- * Enable a cookie to permit it to accept new operations.
+ * Start using the cookie for I/O. This prevents the backing object from being
+ * reaped by VM pressure.
*/
-void __fscache_enable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- loff_t object_size,
- bool (*can_enable)(void *data),
- void *data)
+void __fscache_use_cookie(struct fscache_cookie *cookie, bool will_modify)
{
- _enter("%p", cookie);
+ enum fscache_cookie_stage stage;
- trace_fscache_enable(cookie);
+ _enter("c=%08x", cookie->debug_id);
- wait_on_bit_lock(&cookie->flags, FSCACHE_COOKIE_ENABLEMENT_LOCK,
- TASK_UNINTERRUPTIBLE);
-
- cookie->object_size = object_size;
- fscache_update_aux(cookie, aux_data);
-
- if (test_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags))
- goto out_unlock;
-
- if (can_enable && !can_enable(data)) {
- /* The netfs decided it didn't want to enable after all */
- } else if (cookie->type != FSCACHE_COOKIE_TYPE_INDEX) {
- /* Wait for outstanding disablement to complete */
- __fscache_wait_on_invalidate(cookie);
-
- if (fscache_acquire_non_index_cookie(cookie) == 0)
- set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
- } else {
- set_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
- }
-
-out_unlock:
- clear_bit_unlock(FSCACHE_COOKIE_ENABLEMENT_LOCK, &cookie->flags);
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_ENABLEMENT_LOCK);
-}
-EXPORT_SYMBOL(__fscache_enable_cookie);
+ ASSERT(cookie->type != FSCACHE_COOKIE_TYPE_INDEX);
-/*
- * acquire a non-index cookie
- * - this must make sure the index chain is instantiated and instantiate the
- * object representation too
- */
-static int fscache_acquire_non_index_cookie(struct fscache_cookie *cookie)
-{
- struct fscache_object *object;
- struct fscache_cache *cache;
- int ret;
-
- _enter("");
-
- set_bit(FSCACHE_COOKIE_UNAVAILABLE, &cookie->flags);
-
- /* now we need to see whether the backing objects for this cookie yet
- * exist, if not there'll be nothing to search */
- down_read(&fscache_addremove_sem);
-
- if (list_empty(&fscache_cache_list)) {
- up_read(&fscache_addremove_sem);
- _leave(" = 0 [no caches]");
- return 0;
- }
-
- /* select a cache in which to store the object */
- cache = fscache_select_cache_for_object(cookie->parent);
- if (!cache) {
- up_read(&fscache_addremove_sem);
- fscache_stat(&fscache_n_acquires_no_cache);
- _leave(" = -ENOMEDIUM [no cache]");
- return -ENOMEDIUM;
- }
-
- _debug("cache %s", cache->tag->name);
-
- set_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags);
+ spin_lock(&cookie->lock);
- /* ask the cache to allocate objects for this cookie and its parent
- * chain */
- ret = fscache_alloc_object(cache, cookie);
- if (ret < 0) {
- up_read(&fscache_addremove_sem);
- _leave(" = %d", ret);
- return ret;
- }
+ atomic_inc(&cookie->n_active);
- spin_lock(&cookie->lock);
- if (hlist_empty(&cookie->backing_objects)) {
+again:
+ stage = cookie->stage;
+ switch (stage) {
+ case FSCACHE_COOKIE_STAGE_QUIESCENT:
+ cookie->stage = FSCACHE_COOKIE_STAGE_INITIALISING;
spin_unlock(&cookie->lock);
- goto unavailable;
- }
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object, cookie_link);
+ /* The lookup job holds its own active increment */
+ atomic_inc(&cookie->n_active);
+ fscache_dispatch(cookie, NULL, 0, fscache_lookup_object);
+ break;
- /* initiate the process of looking up all the objects in the chain
- * (done by fscache_initialise_object()) */
- fscache_raise_event(object, FSCACHE_OBJECT_EV_NEW_CHILD);
+ case FSCACHE_COOKIE_STAGE_INITIALISING:
+ case FSCACHE_COOKIE_STAGE_LOOKING_UP:
+ spin_unlock(&cookie->lock);
+ wait_var_event(&cookie->stage, cookie->stage != stage);
+ spin_lock(&cookie->lock);
+ goto again;
- spin_unlock(&cookie->lock);
+ case FSCACHE_COOKIE_STAGE_NO_DATA_YET:
+ case FSCACHE_COOKIE_STAGE_ACTIVE:
+ case FSCACHE_COOKIE_STAGE_INVALIDATING:
+ // TODO: Handle will_modify
+ spin_unlock(&cookie->lock);
+ break;
- /* we may be required to wait for lookup to complete at this point */
- if (!fscache_defer_lookup) {
- _debug("non-deferred lookup %p", &cookie->flags);
- wait_on_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP,
- TASK_UNINTERRUPTIBLE);
- _debug("complete");
- if (test_bit(FSCACHE_COOKIE_UNAVAILABLE, &cookie->flags))
- goto unavailable;
+ case FSCACHE_COOKIE_STAGE_DEAD:
+ spin_unlock(&cookie->lock);
+ kdebug("dead");
+ break;
}
- up_read(&fscache_addremove_sem);
- _leave(" = 0 [deferred]");
- return 0;
-
-unavailable:
- up_read(&fscache_addremove_sem);
- _leave(" = -ENOBUFS");
- return -ENOBUFS;
+ _leave("");
}
+EXPORT_SYMBOL(__fscache_use_cookie);
/*
- * recursively allocate cache object records for a cookie/cache combination
- * - caller must be holding the addremove sem
+ * Stop using the cookie for I/O.
*/
-static int fscache_alloc_object(struct fscache_cache *cache,
- struct fscache_cookie *cookie)
+void __fscache_unuse_cookie(struct fscache_cookie *cookie,
+ const void *aux_data, const loff_t *object_size)
{
- struct fscache_object *object;
- int ret;
-
- _enter("%p,%p{%s}", cache, cookie, cookie->type_name);
-
- spin_lock(&cookie->lock);
- hlist_for_each_entry(object, &cookie->backing_objects,
- cookie_link) {
- if (object->cache == cache)
- goto object_already_extant;
- }
- spin_unlock(&cookie->lock);
-
- /* ask the cache to allocate an object (we may end up with duplicate
- * objects at this stage, but we sort that out later) */
- fscache_stat(&fscache_n_cop_alloc_object);
- object = cache->ops->alloc_object(cache, cookie);
- fscache_stat_d(&fscache_n_cop_alloc_object);
- if (IS_ERR(object)) {
- fscache_stat(&fscache_n_object_no_alloc);
- ret = PTR_ERR(object);
- goto error;
- }
-
- ASSERTCMP(object->cookie, ==, cookie);
- fscache_stat(&fscache_n_object_alloc);
-
- object->debug_id = atomic_inc_return(&fscache_object_debug_id);
-
- _debug("ALLOC OBJ%x: %s {%lx}",
- object->debug_id, cookie->type_name, object->events);
-
- ret = fscache_alloc_object(cache, cookie->parent);
- if (ret < 0)
- goto error_put;
-
- /* only attach if we managed to allocate all we needed, otherwise
- * discard the object we just allocated and instead use the one
- * attached to the cookie */
- if (fscache_attach_object(cookie, object) < 0) {
- fscache_stat(&fscache_n_cop_put_object);
- cache->ops->put_object(object, fscache_obj_put_attach_fail);
- fscache_stat_d(&fscache_n_cop_put_object);
- }
-
- _leave(" = 0");
- return 0;
-
-object_already_extant:
- ret = -ENOBUFS;
- if (fscache_object_is_dying(object) ||
- fscache_cache_is_broken(object)) {
- spin_unlock(&cookie->lock);
- goto error;
- }
- spin_unlock(&cookie->lock);
- _leave(" = 0 [found]");
- return 0;
-
-error_put:
- fscache_stat(&fscache_n_cop_put_object);
- cache->ops->put_object(object, fscache_obj_put_alloc_fail);
- fscache_stat_d(&fscache_n_cop_put_object);
-error:
- _leave(" = %d", ret);
- return ret;
+ if (aux_data || object_size)
+ __fscache_update_cookie(cookie, aux_data, object_size);
+ if (atomic_dec_and_test(&cookie->n_active))
+ wake_up_var(&cookie->n_active);
}
+EXPORT_SYMBOL(__fscache_unuse_cookie);
/*
- * attach a cache object to a cookie
+ * Attempt to attach the object to the list on the cookie or, if there's an
+ * object already attached, then that is used instead and a ref is taken on it
+ * for the caller. Returns a pointer to whichever object is selected.
*/
-static int fscache_attach_object(struct fscache_cookie *cookie,
- struct fscache_object *object)
+struct fscache_object *fscache_attach_object(struct fscache_cookie *cookie,
+ struct fscache_object *object)
{
- struct fscache_object *p;
+ struct fscache_object *parent, *p, *ret = NULL;
struct fscache_cache *cache = object->cache;
- int ret;
_enter("{%s},{OBJ%x}", cookie->type_name, object->debug_id);
@@ -575,38 +418,26 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
/* there may be multiple initial creations of this object, but we only
* want one */
- ret = -EEXIST;
hlist_for_each_entry(p, &cookie->backing_objects, cookie_link) {
- if (p->cache == object->cache) {
- if (fscache_object_is_dying(p))
- ret = -ENOBUFS;
- goto cant_attach_object;
- }
+ if (p->cache == object->cache)
+ goto exists;
}
/* pin the parent object */
spin_lock_nested(&cookie->parent->lock, 1);
- hlist_for_each_entry(p, &cookie->parent->backing_objects,
- cookie_link) {
- if (p->cache == object->cache) {
- if (fscache_object_is_dying(p)) {
- ret = -ENOBUFS;
- spin_unlock(&cookie->parent->lock);
- goto cant_attach_object;
- }
- object->parent = p;
- spin_lock(&p->lock);
- p->n_children++;
- spin_unlock(&p->lock);
- break;
- }
- }
+
+ parent = object->parent;
+
+ spin_lock(&parent->lock);
+ parent->n_children++;
+ spin_unlock(&parent->lock);
+
spin_unlock(&cookie->parent->lock);
/* attach to the cache's object list */
if (list_empty(&object->cache_link)) {
spin_lock(&cache->object_list_lock);
- list_add(&object->cache_link, &cache->object_list);
+ list_add_tail(&object->cache_link, &cache->object_list);
spin_unlock(&cache->object_list_lock);
}
@@ -614,12 +445,29 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
hlist_add_head(&object->cookie_link, &cookie->backing_objects);
fscache_objlist_add(object);
- ret = 0;
+ cookie->stage = FSCACHE_COOKIE_STAGE_LOOKING_UP;
+ wake_up_var(&cookie->stage);
-cant_attach_object:
+out_grab:
+ ret = cache->ops->grab_object(object, fscache_obj_get_attach);
spin_unlock(&cookie->lock);
- _leave(" = %d", ret);
+ _leave(" = %p", ret);
return ret;
+
+exists:
+ object = p;
+ goto out_grab;
+}
+
+void fscache_set_cookie_stage(struct fscache_cookie *cookie,
+ enum fscache_cookie_stage stage)
+{
+ if (cookie->stage != stage) {
+ spin_lock(&cookie->lock);
+ cookie->stage = stage;
+ spin_unlock(&cookie->lock);
+ wake_up_var(&cookie->stage);
+ }
}
/*
@@ -627,8 +475,6 @@ static int fscache_attach_object(struct fscache_cookie *cookie,
*/
void __fscache_invalidate(struct fscache_cookie *cookie)
{
- struct fscache_object *object;
-
_enter("{%s}", cookie->type_name);
fscache_stat(&fscache_n_invalidates);
@@ -640,26 +486,9 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
*/
ASSERTCMP(cookie->type, ==, FSCACHE_COOKIE_TYPE_DATAFILE);
- /* If there's an object, we tell the object state machine to handle the
- * invalidation on our behalf, otherwise there's nothing to do.
- */
- if (!hlist_empty(&cookie->backing_objects)) {
- spin_lock(&cookie->lock);
-
- if (fscache_cookie_enabled(cookie) &&
- !hlist_empty(&cookie->backing_objects) &&
- !test_and_set_bit(FSCACHE_COOKIE_INVALIDATING,
- &cookie->flags)) {
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object,
- cookie_link);
- /* TODO: Do invalidation */
- }
-
- spin_unlock(&cookie->lock);
- }
-
- _leave("");
+ if (!hlist_empty(&cookie->backing_objects) &&
+ test_and_set_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
+ fscache_dispatch(cookie, NULL, 0, fscache_invalidate_object);
}
EXPORT_SYMBOL(__fscache_invalidate);
@@ -678,9 +507,10 @@ void __fscache_wait_on_invalidate(struct fscache_cookie *cookie)
EXPORT_SYMBOL(__fscache_wait_on_invalidate);
/*
- * update the index entries backing a cookie
+ * Update the index entries backing a cookie. The writeback is done lazily.
*/
-void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data)
+void __fscache_update_cookie(struct fscache_cookie *cookie,
+ const void *aux_data, const loff_t *object_size)
{
struct fscache_object *object;
@@ -696,16 +526,9 @@ void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data
spin_lock(&cookie->lock);
- fscache_update_aux(cookie, aux_data);
-
- if (fscache_cookie_enabled(cookie)) {
- /* update the index entry on disk in each cache backing this
- * cookie.
- */
- hlist_for_each_entry(object,
- &cookie->backing_objects, cookie_link) {
- fscache_raise_event(object, FSCACHE_OBJECT_EV_UPDATE);
- }
+ fscache_update_aux(cookie, aux_data, object_size);
+ hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags);
}
spin_unlock(&cookie->lock);
@@ -713,90 +536,13 @@ void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data
}
EXPORT_SYMBOL(__fscache_update_cookie);
-/*
- * Disable a cookie to stop it from accepting new requests from the netfs.
- */
-void __fscache_disable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- bool invalidate)
-{
- struct fscache_object *object;
- bool awaken = false;
-
- _enter("%p,%u", cookie, invalidate);
-
- trace_fscache_disable(cookie);
-
- ASSERTCMP(atomic_read(&cookie->n_active), >, 0);
-
- if (atomic_read(&cookie->n_children) != 0) {
- pr_err("Cookie '%s' still has children\n",
- cookie->type_name);
- BUG();
- }
-
- wait_on_bit_lock(&cookie->flags, FSCACHE_COOKIE_ENABLEMENT_LOCK,
- TASK_UNINTERRUPTIBLE);
-
- fscache_update_aux(cookie, aux_data);
-
- if (!test_and_clear_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags))
- goto out_unlock_enable;
-
- /* If the cookie is being invalidated, wait for that to complete first
- * so that we can reuse the flag.
- */
- __fscache_wait_on_invalidate(cookie);
-
- /* Dispose of the backing objects */
- set_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags);
-
- spin_lock(&cookie->lock);
- if (!hlist_empty(&cookie->backing_objects)) {
- hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
- if (invalidate)
- set_bit(FSCACHE_OBJECT_RETIRED, &object->flags);
- fscache_raise_event(object, FSCACHE_OBJECT_EV_KILL);
- }
- } else {
- if (test_and_clear_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
- awaken = true;
- }
- spin_unlock(&cookie->lock);
- if (awaken)
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING);
-
- /* Wait for cessation of activity requiring access to the netfs (when
- * n_active reaches 0). This makes sure outstanding reads and writes
- * have completed.
- */
- if (!atomic_dec_and_test(&cookie->n_active)) {
- wait_var_event(&cookie->n_active,
- !atomic_read(&cookie->n_active));
- }
-
- /* Reset the cookie state if it wasn't relinquished */
- if (!test_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags)) {
- atomic_inc(&cookie->n_active);
- set_bit(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
- }
-
-out_unlock_enable:
- clear_bit_unlock(FSCACHE_COOKIE_ENABLEMENT_LOCK, &cookie->flags);
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_ENABLEMENT_LOCK);
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_disable_cookie);
-
/*
* release a cookie back to the cache
* - the object will be marked as recyclable on disk if retire is true
* - all dependents of this cookie must have already been unregistered
* (indices/files/pages)
*/
-void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- bool retire)
+void __fscache_relinquish_cookie(struct fscache_cookie *cookie, bool retire)
{
fscache_stat(&fscache_n_relinquishes);
if (retire)
@@ -818,7 +564,23 @@ void __fscache_relinquish_cookie(struct fscache_cookie *cookie,
if (test_and_set_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags))
BUG();
- __fscache_disable_cookie(cookie, aux_data, retire);
+ /* Mark the cookie out of service and wait for cessation of I/O. */
+ spin_lock(&cookie->lock);
+ cookie->stage = FSCACHE_COOKIE_STAGE_DEAD;
+ atomic_dec(&cookie->n_ops);
+ spin_unlock(&cookie->lock);
+
+ wait_var_event(&cookie->n_ops, atomic_read(&cookie->n_ops) == 0);
+
+ /* Dispose of the backing objects */
+ if (!hlist_empty(&cookie->backing_objects))
+ fscache_dispatch(cookie, NULL, retire, fscache_discard_objects);
+
+ if (atomic_read(&cookie->n_children) != 0) {
+ pr_err("Cookie '%s' still has children\n",
+ cookie->type_name);
+ BUG();
+ }
if (cookie->parent) {
ASSERTCMP(atomic_read(&cookie->parent->usage), >, 0);
@@ -879,6 +641,7 @@ void fscache_cookie_put(struct fscache_cookie *cookie,
_leave("");
}
+EXPORT_SYMBOL(fscache_cookie_put);
/*
* Generate a list of extant cookies in /proc/fs/fscache/cookies
@@ -892,8 +655,8 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
if (v == &fscache_cookies) {
seq_puts(m,
- "COOKIE PARENT USAGE CHILD ACT TY FL DEF \n"
- "======== ======== ===== ===== === == === ================\n"
+ "COOKIE PARENT USAGE CHILD ACT OPS TY S FL DEF \n"
+ "======== ======== ===== ===== === === == = === ================\n"
);
return 0;
}
@@ -915,13 +678,15 @@ static int fscache_cookies_seq_show(struct seq_file *m, void *v)
}
seq_printf(m,
- "%08x %08x %5u %5u %3u %s %03lx %-16s",
+ "%08x %08x %5d %5d %3d %3d %s %u %03lx %-16s",
cookie->debug_id,
cookie->parent ? cookie->parent->debug_id : 0,
atomic_read(&cookie->usage),
atomic_read(&cookie->n_children),
atomic_read(&cookie->n_active),
+ atomic_read(&cookie->n_ops) - 1,
type,
+ cookie->stage,
cookie->flags,
cookie->type_name);
diff --git a/fs/fscache/fsdef.c b/fs/fscache/fsdef.c
index b802cbddb578..377151a6a54a 100644
--- a/fs/fscache/fsdef.c
+++ b/fs/fscache/fsdef.c
@@ -40,7 +40,6 @@ struct fscache_cookie fscache_fsdef_index = {
.lock = __SPIN_LOCK_UNLOCKED(fscache_fsdef_index.lock),
.backing_objects = HLIST_HEAD_INIT,
.type_name = ".fscach",
- .flags = 1 << FSCACHE_COOKIE_ENABLED,
.type = FSCACHE_COOKIE_TYPE_INDEX,
};
EXPORT_SYMBOL(fscache_fsdef_index);
diff --git a/fs/fscache/histogram.c b/fs/fscache/histogram.c
index 4e5beeaaf454..4fd5dda1d0ad 100644
--- a/fs/fscache/histogram.c
+++ b/fs/fscache/histogram.c
@@ -5,7 +5,7 @@
* Written by David Howells (dhowells@redhat.com)
*/
-#define FSCACHE_DEBUG_LEVEL THREAD
+#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 2100e2222884..443671310e31 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -74,6 +74,10 @@ extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
extern struct fscache_cookie *fscache_hash_cookie(struct fscache_cookie *);
extern void fscache_cookie_put(struct fscache_cookie *,
enum fscache_cookie_trace);
+extern struct fscache_object *fscache_attach_object(struct fscache_cookie *,
+ struct fscache_object *);
+extern void fscache_set_cookie_stage(struct fscache_cookie *,
+ enum fscache_cookie_stage);
/*
* dispatcher.c
@@ -119,19 +123,15 @@ extern unsigned fscache_defer_lookup;
extern unsigned fscache_defer_create;
extern unsigned fscache_debug;
extern struct kobject *fscache_root;
-extern struct workqueue_struct *fscache_object_wq;
extern struct workqueue_struct *fscache_op_wq;
-DECLARE_PER_CPU(wait_queue_head_t, fscache_object_cong_wait);
-
-static inline bool fscache_object_congested(void)
-{
- return workqueue_congested(WORK_CPU_UNBOUND, fscache_object_wq);
-}
/*
- * object.c
+ * obj.c
*/
-extern void fscache_enqueue_object(struct fscache_object *);
+extern void fscache_lookup_object(struct fscache_cookie *, struct fscache_object *, int);
+extern void fscache_invalidate_object(struct fscache_cookie *, struct fscache_object *, int);
+extern void fscache_drop_object(struct fscache_cookie *, struct fscache_object *, bool);
+extern void fscache_discard_objects(struct fscache_cookie *, struct fscache_object *, int);
/*
* object-list.c
@@ -239,44 +239,29 @@ int fscache_stats_show(struct seq_file *m, void *v);
#define fscache_stat_d(stat) do {} while (0)
#endif
-/*
- * raise an event on an object
- * - if the event is not masked for that object, then the object is
- * queued for attention by the thread pool.
- */
-static inline void fscache_raise_event(struct fscache_object *object,
- unsigned event)
-{
- BUG_ON(event >= NR_FSCACHE_OBJECT_EVENTS);
-#if 0
- printk("*** fscache_raise_event(OBJ%d{%lx},%x)\n",
- object->debug_id, object->event_mask, (1 << event));
-#endif
- if (!test_and_set_bit(event, &object->events) &&
- test_bit(event, &object->event_mask))
- fscache_enqueue_object(object);
-}
-
-static inline void fscache_cookie_get(struct fscache_cookie *cookie,
- enum fscache_cookie_trace where)
+static inline
+struct fscache_cookie *fscache_cookie_get(struct fscache_cookie *cookie,
+ enum fscache_cookie_trace where)
{
int usage = atomic_inc_return(&cookie->usage);
trace_fscache_cookie(cookie, where, usage);
+ return cookie;
}
/*
* Update the auxiliary data on a cookie.
*/
static inline
-void fscache_update_aux(struct fscache_cookie *cookie, const void *aux_data)
+void fscache_update_aux(struct fscache_cookie *cookie,
+ const void *aux_data, const loff_t *object_size)
{
void *p = fscache_get_aux(cookie);
- if (p && memcmp(p, aux_data, cookie->aux_len) != 0) {
+ if (aux_data && p)
memcpy(p, aux_data, cookie->aux_len);
- set_bit(FSCACHE_COOKIE_AUX_UPDATED, &cookie->flags);
- }
+ if (object_size)
+ cookie->object_size = *object_size;
}
/*****************************************************************************/
@@ -337,7 +322,7 @@ do { \
#define FSCACHE_DEBUG_CACHE 0
#define FSCACHE_DEBUG_COOKIE 1
-#define FSCACHE_DEBUG_PAGE 2
+#define FSCACHE_DEBUG_OBJECT 2
#define FSCACHE_DEBUG_OPERATION 3
#define FSCACHE_POINT_ENTER 1
diff --git a/fs/fscache/main.c b/fs/fscache/main.c
index 6f225ae0fd99..e25a2c592026 100644
--- a/fs/fscache/main.c
+++ b/fs/fscache/main.c
@@ -38,11 +38,8 @@ MODULE_PARM_DESC(fscache_debug,
"FS-Cache debugging mask");
struct kobject *fscache_root;
-struct workqueue_struct *fscache_object_wq;
struct workqueue_struct *fscache_op_wq;
-DEFINE_PER_CPU(wait_queue_head_t, fscache_object_cong_wait);
-
/* these values serve as lower bounds, will be adjusted in fscache_init() */
static unsigned fscache_object_max_active = 4;
static unsigned fscache_op_max_active = 2;
@@ -65,14 +62,6 @@ static int fscache_max_active_sysctl(struct ctl_table *table, int write,
}
static struct ctl_table fscache_sysctls[] = {
- {
- .procname = "object_max_active",
- .data = &fscache_object_max_active,
- .maxlen = sizeof(unsigned),
- .mode = 0644,
- .proc_handler = fscache_max_active_sysctl,
- .extra1 = &fscache_object_wq,
- },
{
.procname = "operation_max_active",
.data = &fscache_op_max_active,
@@ -99,20 +88,8 @@ static struct ctl_table fscache_sysctls_root[] = {
*/
static int __init fscache_init(void)
{
- unsigned int nr_cpus = num_possible_cpus();
- unsigned int cpu;
int ret;
- fscache_object_max_active =
- clamp_val(nr_cpus,
- fscache_object_max_active, WQ_UNBOUND_MAX_ACTIVE);
-
- ret = -ENOMEM;
- fscache_object_wq = alloc_workqueue("fscache_object", WQ_UNBOUND,
- fscache_object_max_active);
- if (!fscache_object_wq)
- goto error_object_wq;
-
fscache_op_max_active =
clamp_val(fscache_object_max_active / 2,
fscache_op_max_active, WQ_UNBOUND_MAX_ACTIVE);
@@ -123,9 +100,6 @@ static int __init fscache_init(void)
if (!fscache_op_wq)
goto error_op_wq;
- for_each_possible_cpu(cpu)
- init_waitqueue_head(&per_cpu(fscache_object_cong_wait, cpu));
-
ret = fscache_init_dispatchers();
if (ret < 0)
goto error_dispatchers;
@@ -170,8 +144,6 @@ static int __init fscache_init(void)
error_proc:
destroy_workqueue(fscache_op_wq);
error_op_wq:
- destroy_workqueue(fscache_object_wq);
-error_object_wq:
return ret;
}
@@ -192,7 +164,6 @@ static void __exit fscache_exit(void)
fscache_proc_cleanup();
fscache_kill_dispatchers();
destroy_workqueue(fscache_op_wq);
- destroy_workqueue(fscache_object_wq);
pr_notice("Unloaded\n");
}
diff --git a/fs/fscache/netfs.c b/fs/fscache/netfs.c
index f8a816f844f6..7ebf733b068b 100644
--- a/fs/fscache/netfs.c
+++ b/fs/fscache/netfs.c
@@ -32,7 +32,7 @@ int __fscache_register_netfs(struct fscache_netfs *netfs)
return -ENOMEM;
}
- candidate->flags = 1 << FSCACHE_COOKIE_ENABLED;
+ trace_fscache_cookie(candidate, fscache_cookie_new_netfs, 1);
/* check the netfs type is not already present */
cookie = fscache_hash_cookie(candidate);
@@ -68,7 +68,7 @@ void __fscache_unregister_netfs(struct fscache_netfs *netfs)
{
_enter("{%s.%u}", netfs->name, netfs->version);
- fscache_relinquish_cookie(netfs->primary_index, NULL, false);
+ fscache_relinquish_cookie(netfs->primary_index, false);
pr_notice("Netfs '%s' unregistered from caching\n", netfs->name);
_leave("");
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
new file mode 100644
index 000000000000..1bf2bee00d18
--- /dev/null
+++ b/fs/fscache/obj.c
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Cache object management
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * See Documentation/filesystems/caching/netfs-api.txt for more information on
+ * the netfs API.
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/module.h>
+#include <linux/slab.h>
+#include "internal.h"
+
+static struct fscache_object *fscache_do_alloc_object(struct fscache_cookie *cookie,
+ struct fscache_cache *cache,
+ struct fscache_object *parent)
+{
+ struct fscache_object *object;
+
+ fscache_stat(&fscache_n_cop_alloc_object);
+ object = cache->ops->alloc_object(cookie, cache, parent);
+ fscache_stat_d(&fscache_n_cop_alloc_object);
+
+ if (!IS_ERR(object)) {
+ ASSERTCMP(object->cookie, ==, cookie);
+ _debug("ALLOC OBJ%x: %s", object->debug_id, cookie->type_name);
+ fscache_stat(&fscache_n_object_alloc);
+ } else {
+ fscache_stat(&fscache_n_object_no_alloc);
+ }
+
+ return object;
+}
+
+static int fscache_do_lookup_object(struct fscache_object *object, void *data)
+{
+ int ret;
+ fscache_stat(&fscache_n_object_lookups);
+ fscache_stat(&fscache_n_cop_lookup_object);
+ ret = object->cache->ops->lookup_object(object, data);
+ fscache_stat_d(&fscache_n_cop_lookup_object);
+ return ret;
+}
+
+static int fscache_do_create_object(struct fscache_object *object, void *data)
+{
+ int ret;
+ fscache_stat(&fscache_n_object_lookups);
+ fscache_stat(&fscache_n_cop_lookup_object);
+ ret = object->cache->ops->create_object(object, data);
+ fscache_stat_d(&fscache_n_cop_lookup_object);
+ return ret;
+}
+
+static void fscache_do_update_object(struct fscache_object *object)
+{
+ fscache_stat(&fscache_n_updates_run);
+ fscache_stat(&fscache_n_cop_update_object);
+ object->cache->ops->update_object(object);
+ fscache_stat_d(&fscache_n_cop_update_object);
+}
+
+static void fscache_do_drop_object(struct fscache_cache *cache,
+ struct fscache_object *object,
+ bool invalidate)
+{
+ fscache_stat(&fscache_n_cop_drop_object);
+ cache->ops->drop_object(object, invalidate);
+ fscache_stat_d(&fscache_n_cop_drop_object);
+}
+
+static void fscache_do_put_object(struct fscache_object *object,
+ enum fscache_obj_ref_trace why)
+{
+ fscache_stat(&fscache_n_cop_put_object);
+ object->cache->ops->put_object(object, why);
+ fscache_stat_d(&fscache_n_cop_put_object);
+}
+
+/*
+ * Do the actual on-disk wrangling involved in object lookup/creation.
+ */
+static bool fscache_wrangle_object(struct fscache_cookie *cookie,
+ struct fscache_cache *cache,
+ struct fscache_object *object)
+{
+
+ void *lookup_data;
+ bool ret = false;
+
+ lookup_data = cache->ops->prepare_lookup_data(object);
+ if (IS_ERR(lookup_data))
+ goto out;
+
+ if (!fscache_do_lookup_object(object, lookup_data))
+ goto out_free;
+
+ if (object->stage < FSCACHE_OBJECT_STAGE_LIVE_EMPTY &&
+ !fscache_do_create_object(object, lookup_data))
+ goto out_free;
+
+ fscache_set_cookie_stage(cookie,
+ (object->stage < FSCACHE_OBJECT_STAGE_LIVE ?
+ FSCACHE_COOKIE_STAGE_NO_DATA_YET :
+ FSCACHE_COOKIE_STAGE_ACTIVE));
+ ret = true;
+
+out_free:
+ cache->ops->free_lookup_data(object, lookup_data);
+out:
+ return ret;
+}
+
+/*
+ * Create an object chain, making sure that the index chain is fully created.
+ */
+static struct fscache_object *fscache_lookup_object_chain(struct fscache_cookie *cookie,
+ struct fscache_cache *cache)
+{
+ struct fscache_object *object = NULL, *parent, *xobject;
+
+ _enter("c=%08x", cookie->debug_id);
+
+ spin_lock(&cookie->lock);
+ hlist_for_each_entry(object, &cookie->backing_objects, cookie_link) {
+ if (object->cache == cache)
+ goto object_exists_grab;
+ }
+ spin_unlock(&cookie->lock);
+
+ /* Recurse to look up/create the parent index. */
+ parent = fscache_lookup_object_chain(cookie->parent, cache);
+ if (IS_ERR(parent))
+ goto error;
+
+ /* Ask the cache to allocate an object (we may end up with duplicate
+ * objects at this stage, but we sort that out later).
+ *
+ * The object may be created, say, with O_TMPFILE at this point if the
+ * parent index was unpopulated. Note that this may race on index
+ * creation with other callers.
+ */
+ object = fscache_do_alloc_object(cookie, cache, parent);
+ if (!object)
+ goto error;
+
+ xobject = fscache_attach_object(cookie, object);
+ if (xobject != object) {
+ fscache_do_put_object(object, fscache_obj_put_alloc_dup);
+ goto object_exists;
+ }
+
+ if (!fscache_wrangle_object(cookie, cache, object))
+ goto error_detach;
+
+ _leave(" = %u", object->debug_id);
+ return object;
+
+object_exists_grab:
+ object = cache->ops->grab_object(object, fscache_obj_get_exists);
+object_exists:
+ if (fscache_cache_is_broken(object)) {
+ spin_unlock(&cookie->lock);
+ goto error_put;
+ }
+
+ spin_unlock(&cookie->lock);
+
+ wait_var_event(&object->stage,
+ object->stage >= FSCACHE_OBJECT_STAGE_LIVE_EMPTY);
+
+ if (object->stage >= FSCACHE_OBJECT_STAGE_DESTROYING)
+ goto error_put;
+
+ _leave(" = %u [found]", object->debug_id);
+ return object;
+
+error_detach:
+ spin_lock(&cookie->lock);
+ spin_lock(&object->lock);
+ object->parent = NULL;
+ object->stage = FSCACHE_OBJECT_STAGE_DEAD;
+ hlist_del_init(&object->cookie_link);
+ spin_unlock(&object->lock);
+ spin_unlock(&cookie->lock);
+ fscache_drop_object(cookie, object, false);
+error_put:
+ fscache_do_put_object(object, fscache_obj_put_lookup_fail);
+error:
+ fscache_set_cookie_stage(cookie, FSCACHE_COOKIE_STAGE_QUIESCENT);
+ _leave(" = NULL");
+ return NULL;
+}
+
+/*
+ * Create an object in the cache.
+ * - this must make sure the index chain is instantiated and instantiate the
+ * object representation too
+ */
+static void fscache_lookup_object_locked(struct fscache_cookie *cookie)
+{
+ struct fscache_object *object;
+ struct fscache_cache *cache;
+
+ _enter("");
+
+ /* select a cache in which to store the object */
+ cache = fscache_select_cache_for_object(cookie);
+ if (!cache) {
+ fscache_stat(&fscache_n_acquires_no_cache);
+ fscache_set_cookie_stage(cookie, FSCACHE_COOKIE_STAGE_QUIESCENT);
+ _leave(" [no cache]");
+ return;
+ }
+
+ _debug("cache %s", cache->tag->name);
+
+ object = fscache_lookup_object_chain(cookie, cache);
+ if (!object) {
+ _leave(" [fail]");
+ return;
+ }
+
+ fscache_do_put_object(object, fscache_obj_put);
+ _leave(" [done]");
+}
+
+void fscache_lookup_object(struct fscache_cookie *cookie,
+ struct fscache_object *object, int param)
+{
+ down_read(&fscache_addremove_sem);
+ fscache_lookup_object_locked(cookie);
+ up_read(&fscache_addremove_sem);
+ __fscache_unuse_cookie(cookie, NULL, NULL);
+}
+
+/*
+ * Invalidate an object
+ */
+void fscache_invalidate_object(struct fscache_cookie *cookie,
+ struct fscache_object *unused, int param)
+{
+ struct fscache_object *object = NULL;
+
+ spin_lock(&cookie->lock);
+
+ if (!hlist_empty(&cookie->backing_objects)) {
+ object = hlist_entry(cookie->backing_objects.first,
+ struct fscache_object,
+ cookie_link);
+ object = object->cache->ops->grab_object(object,
+ fscache_obj_get_inval);
+ }
+
+ spin_unlock(&cookie->lock);
+
+ if (object) {
+ object->cache->ops->invalidate_object(object);
+ fscache_do_put_object(object, fscache_obj_put_inval);
+ }
+
+ clear_bit_unlock(FSCACHE_COOKIE_INVALIDATING, &cookie->flags);
+ wake_up_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING);
+}
+
+/*
+ * Drop an object's attachments
+ */
+void fscache_drop_object(struct fscache_cookie *cookie,
+ struct fscache_object *object,
+ bool invalidate)
+{
+ struct fscache_object *parent = object->parent;
+ struct fscache_cache *cache = object->cache;
+
+ ASSERTCMP(cookie->stage, ==, FSCACHE_COOKIE_STAGE_DEAD);
+
+ _enter("{OBJ%x,%d},%u",
+ object->debug_id, object->n_children, invalidate);
+
+ if (!invalidate &&
+ test_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags)) {
+ _debug("final update");
+ fscache_do_update_object(object);
+ }
+
+ spin_lock(&cache->object_list_lock);
+ list_del_init(&object->cache_link);
+ spin_unlock(&cache->object_list_lock);
+
+ fscache_do_drop_object(cache, object, invalidate);
+
+ /* The parent object wants to know when all it dependents have gone */
+ if (parent) {
+ _debug("release parent OBJ%x {%d}",
+ parent->debug_id, parent->n_children);
+
+ spin_lock(&parent->lock);
+ parent->n_children--;
+ if (parent->n_children == 0)
+ wake_up_var(&parent->n_children);
+ spin_unlock(&parent->lock);
+ object->parent = NULL;
+ fscache_do_put_object(parent, fscache_obj_put_drop_child);
+ }
+
+ fscache_do_put_object(object, fscache_obj_put_drop_obj);
+ fscache_stat(&fscache_n_object_dead);
+ _leave("");
+}
+
+/*
+ * Discard objects. param==1 to invalidate it at the same time.
+ */
+void fscache_discard_objects(struct fscache_cookie *cookie,
+ struct fscache_object *unused, int param)
+{
+ _enter("c=%08x", cookie->debug_id);
+
+ /* Take the cache lock to prevent a fight with cache withdrawal */
+ down_read(&fscache_addremove_sem);
+
+ for (;;) {
+ struct fscache_object *object = NULL;
+
+ trace_fscache_cookie(cookie, fscache_cookie_see_discard,
+ atomic_read(&cookie->usage));
+
+ spin_lock(&cookie->lock);
+ if (!hlist_empty(&cookie->backing_objects)) {
+ object = hlist_entry(cookie->backing_objects.first,
+ struct fscache_object,
+ cookie_link);
+ hlist_del_init(&object->cookie_link);
+ }
+ spin_unlock(&cookie->lock);
+
+ if (!object)
+ break;
+
+ _debug("DISCARD %x", object->debug_id);
+ fscache_drop_object(cookie, object, param);
+ }
+
+ up_read(&fscache_addremove_sem);
+ _leave("");
+}
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 147c556ee01b..5777f909d31a 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -23,10 +23,6 @@ struct fscache_objlist_data {
#define FSCACHE_OBJLIST_CONFIG_AUX 0x00000002 /* show object auxdata */
#define FSCACHE_OBJLIST_CONFIG_BUSY 0x00000010 /* show busy objects */
#define FSCACHE_OBJLIST_CONFIG_IDLE 0x00000020 /* show idle objects */
-#define FSCACHE_OBJLIST_CONFIG_EVENTS 0x00000400 /* show objects with events */
-#define FSCACHE_OBJLIST_CONFIG_NOEVENTS 0x00000800 /* show objects without no events */
-#define FSCACHE_OBJLIST_CONFIG_WORK 0x00001000 /* show objects with work */
-#define FSCACHE_OBJLIST_CONFIG_NOWORK 0x00002000 /* show objects without work */
};
/*
@@ -162,32 +158,15 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
u8 *p;
if ((unsigned long) v == 1) {
- seq_puts(m, "OBJECT PARENT STAT CHLDN OPS OOP"
- " EM EV FL S"
- " | COOKIE TYPE TY FL");
- if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
- FSCACHE_OBJLIST_CONFIG_AUX))
- seq_puts(m, " ");
- if (config & FSCACHE_OBJLIST_CONFIG_KEY)
- seq_puts(m, "OBJECT_KEY");
- if ((config & (FSCACHE_OBJLIST_CONFIG_KEY |
- FSCACHE_OBJLIST_CONFIG_AUX)) ==
- (FSCACHE_OBJLIST_CONFIG_KEY | FSCACHE_OBJLIST_CONFIG_AUX))
- seq_puts(m, ", ");
- if (config & FSCACHE_OBJLIST_CONFIG_AUX)
- seq_puts(m, "AUX_DATA");
+ seq_puts(m, "OBJECT PARENT USE CHLDN OPS FL S"
+ " | COOKIE TYPE TY S FLG CONTENT_MAP");
seq_puts(m, "\n");
return 0;
}
if ((unsigned long) v == 2) {
- seq_puts(m, "======== ======== ==== ===== === ==="
- " == == == ="
- " | ======== ======= == ===");
- if (config & (FSCACHE_OBJLIST_CONFIG_KEY |
- FSCACHE_OBJLIST_CONFIG_AUX))
- seq_puts(m, " ================");
- seq_puts(m, "\n");
+ seq_puts(m, "======== ======== === ===== === === ="
+ " | ======== ======= == = === ================\n");
return 0;
}
@@ -207,29 +186,19 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
cookie = obj->cookie;
if (~config) {
- FILTER(fscache_object_is_active(obj) ||
- obj->n_ops != 0 ||
- obj->n_obj_ops != 0 ||
- obj->flags ||
- !list_empty(&obj->dependents),
+ FILTER(atomic_read(&cookie->n_ops) > 0,
BUSY, IDLE);
- FILTER(obj->events & obj->event_mask,
- EVENTS, NOEVENTS);
- FILTER(work_busy(&obj->work), WORK, NOWORK);
}
seq_printf(m,
- "%08x %08x %s %5u %3u %3u %2lx %2lx %2lx %1x | ",
+ "%08x %08x %3u %5u %3u %3lx %u | ",
obj->debug_id,
obj->parent ? obj->parent->debug_id : UINT_MAX,
- obj->state->short_name,
+ obj->cache->ops->get_object_usage(obj),
obj->n_children,
- obj->n_ops,
- obj->n_obj_ops,
- obj->event_mask,
- obj->events,
+ atomic_read(&obj->cookie->n_ops),
obj->flags,
- work_busy(&obj->work));
+ obj->stage);
if (obj->cookie) {
uint16_t keylen = 0, auxlen = 0;
@@ -248,10 +217,11 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
break;
}
- seq_printf(m, "%08x %-7s %s %3lx",
+ seq_printf(m, "%08x %-7s %s %u %3lx",
cookie->debug_id,
cookie->type_name,
type,
+ cookie->stage,
cookie->flags);
if (config & FSCACHE_OBJLIST_CONFIG_KEY)
@@ -325,8 +295,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
case 'A': config |= FSCACHE_OBJLIST_CONFIG_AUX; break;
case 'B': config |= FSCACHE_OBJLIST_CONFIG_BUSY; break;
case 'b': config |= FSCACHE_OBJLIST_CONFIG_IDLE; break;
- case 'S': config |= FSCACHE_OBJLIST_CONFIG_WORK; break;
- case 's': config |= FSCACHE_OBJLIST_CONFIG_NOWORK; break;
}
}
@@ -335,10 +303,6 @@ static void fscache_objlist_config(struct fscache_objlist_data *data)
if (!(config & (FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE)))
config |= FSCACHE_OBJLIST_CONFIG_BUSY | FSCACHE_OBJLIST_CONFIG_IDLE;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS)))
- config |= FSCACHE_OBJLIST_CONFIG_EVENTS | FSCACHE_OBJLIST_CONFIG_NOEVENTS;
- if (!(config & (FSCACHE_OBJLIST_CONFIG_WORK | FSCACHE_OBJLIST_CONFIG_NOWORK)))
- config |= FSCACHE_OBJLIST_CONFIG_WORK | FSCACHE_OBJLIST_CONFIG_NOWORK;
data->config = config;
return;
diff --git a/fs/fscache/object.c b/fs/fscache/object.c
deleted file mode 100644
index 8cb21dd4b1cc..000000000000
--- a/fs/fscache/object.c
+++ /dev/null
@@ -1,982 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* FS-Cache object state machine handler
- *
- * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * See Documentation/filesystems/caching/object.txt for a description of the
- * object state machine and the in-kernel representations.
- */
-
-#define FSCACHE_DEBUG_LEVEL COOKIE
-#include <linux/module.h>
-#include <linux/slab.h>
-#include <linux/prefetch.h>
-#include "internal.h"
-
-static const struct fscache_state *fscache_abort_initialisation(struct fscache_object *, int);
-static const struct fscache_state *fscache_kill_dependents(struct fscache_object *, int);
-static const struct fscache_state *fscache_drop_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_initialise_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_invalidate_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_jumpstart_dependents(struct fscache_object *, int);
-static const struct fscache_state *fscache_kill_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_lookup_failure(struct fscache_object *, int);
-static const struct fscache_state *fscache_look_up_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_object_available(struct fscache_object *, int);
-static const struct fscache_state *fscache_parent_ready(struct fscache_object *, int);
-static const struct fscache_state *fscache_update_object(struct fscache_object *, int);
-static const struct fscache_state *fscache_object_dead(struct fscache_object *, int);
-
-#define __STATE_NAME(n) fscache_osm_##n
-#define STATE(n) (&__STATE_NAME(n))
-
-/*
- * Define a work state. Work states are execution states. No event processing
- * is performed by them. The function attached to a work state returns a
- * pointer indicating the next state to which the state machine should
- * transition. Returning NO_TRANSIT repeats the current state, but goes back
- * to the scheduler first.
- */
-#define WORK_STATE(n, sn, f) \
- const struct fscache_state __STATE_NAME(n) = { \
- .name = #n, \
- .short_name = sn, \
- .work = f \
- }
-
-/*
- * Returns from work states.
- */
-#define transit_to(state) ({ prefetch(&STATE(state)->work); STATE(state); })
-
-#define NO_TRANSIT ((struct fscache_state *)NULL)
-
-/*
- * Define a wait state. Wait states are event processing states. No execution
- * is performed by them. Wait states are just tables of "if event X occurs,
- * clear it and transition to state Y". The dispatcher returns to the
- * scheduler if none of the events in which the wait state has an interest are
- * currently pending.
- */
-#define WAIT_STATE(n, sn, ...) \
- const struct fscache_state __STATE_NAME(n) = { \
- .name = #n, \
- .short_name = sn, \
- .work = NULL, \
- .transitions = { __VA_ARGS__, { 0, NULL } } \
- }
-
-#define TRANSIT_TO(state, emask) \
- { .events = (emask), .transit_to = STATE(state) }
-
-/*
- * The object state machine.
- */
-static WORK_STATE(INIT_OBJECT, "INIT", fscache_initialise_object);
-static WORK_STATE(PARENT_READY, "PRDY", fscache_parent_ready);
-static WORK_STATE(ABORT_INIT, "ABRT", fscache_abort_initialisation);
-static WORK_STATE(LOOK_UP_OBJECT, "LOOK", fscache_look_up_object);
-static WORK_STATE(CREATE_OBJECT, "CRTO", fscache_look_up_object);
-static WORK_STATE(OBJECT_AVAILABLE, "AVBL", fscache_object_available);
-static WORK_STATE(JUMPSTART_DEPS, "JUMP", fscache_jumpstart_dependents);
-
-static WORK_STATE(INVALIDATE_OBJECT, "INVL", fscache_invalidate_object);
-static WORK_STATE(UPDATE_OBJECT, "UPDT", fscache_update_object);
-
-static WORK_STATE(LOOKUP_FAILURE, "LCFL", fscache_lookup_failure);
-static WORK_STATE(KILL_OBJECT, "KILL", fscache_kill_object);
-static WORK_STATE(KILL_DEPENDENTS, "KDEP", fscache_kill_dependents);
-static WORK_STATE(DROP_OBJECT, "DROP", fscache_drop_object);
-static WORK_STATE(OBJECT_DEAD, "DEAD", fscache_object_dead);
-
-static WAIT_STATE(WAIT_FOR_INIT, "?INI",
- TRANSIT_TO(INIT_OBJECT, 1 << FSCACHE_OBJECT_EV_NEW_CHILD));
-
-static WAIT_STATE(WAIT_FOR_PARENT, "?PRN",
- TRANSIT_TO(PARENT_READY, 1 << FSCACHE_OBJECT_EV_PARENT_READY));
-
-static WAIT_STATE(WAIT_FOR_CMD, "?CMD",
- TRANSIT_TO(INVALIDATE_OBJECT, 1 << FSCACHE_OBJECT_EV_INVALIDATE),
- TRANSIT_TO(UPDATE_OBJECT, 1 << FSCACHE_OBJECT_EV_UPDATE),
- TRANSIT_TO(JUMPSTART_DEPS, 1 << FSCACHE_OBJECT_EV_NEW_CHILD));
-
-static WAIT_STATE(WAIT_FOR_CLEARANCE, "?CLR",
- TRANSIT_TO(KILL_OBJECT, 1 << FSCACHE_OBJECT_EV_CLEARED));
-
-/*
- * Out-of-band event transition tables. These are for handling unexpected
- * events, such as an I/O error. If an OOB event occurs, the state machine
- * clears and disables the event and forces a transition to the nominated work
- * state (acurrently executing work states will complete first).
- *
- * In such a situation, object->state remembers the state the machine should
- * have been in/gone to and returning NO_TRANSIT returns to that.
- */
-static const struct fscache_transition fscache_osm_init_oob[] = {
- TRANSIT_TO(ABORT_INIT,
- (1 << FSCACHE_OBJECT_EV_ERROR) |
- (1 << FSCACHE_OBJECT_EV_KILL)),
- { 0, NULL }
-};
-
-static const struct fscache_transition fscache_osm_lookup_oob[] = {
- TRANSIT_TO(LOOKUP_FAILURE,
- (1 << FSCACHE_OBJECT_EV_ERROR) |
- (1 << FSCACHE_OBJECT_EV_KILL)),
- { 0, NULL }
-};
-
-static const struct fscache_transition fscache_osm_run_oob[] = {
- TRANSIT_TO(KILL_OBJECT,
- (1 << FSCACHE_OBJECT_EV_ERROR) |
- (1 << FSCACHE_OBJECT_EV_KILL)),
- { 0, NULL }
-};
-
-static int fscache_get_object(struct fscache_object *,
- enum fscache_obj_ref_trace);
-static void fscache_put_object(struct fscache_object *,
- enum fscache_obj_ref_trace);
-static bool fscache_enqueue_dependents(struct fscache_object *, int);
-static void fscache_dequeue_object(struct fscache_object *);
-static void fscache_update_aux_data(struct fscache_object *);
-
-/*
- * we need to notify the parent when an op completes that we had outstanding
- * upon it
- */
-static inline void fscache_done_parent_op(struct fscache_object *object)
-{
- struct fscache_object *parent = object->parent;
-
- _enter("OBJ%x {OBJ%x,%x}",
- object->debug_id, parent->debug_id, parent->n_ops);
-
- spin_lock_nested(&parent->lock, 1);
- parent->n_obj_ops--;
- parent->n_ops--;
- if (parent->n_ops == 0)
- fscache_raise_event(parent, FSCACHE_OBJECT_EV_CLEARED);
- spin_unlock(&parent->lock);
-}
-
-/*
- * Object state machine dispatcher.
- */
-static void fscache_object_sm_dispatcher(struct fscache_object *object)
-{
- const struct fscache_transition *t;
- const struct fscache_state *state, *new_state;
- unsigned long events, event_mask;
- bool oob;
- int event = -1;
-
- ASSERT(object != NULL);
-
- _enter("{OBJ%x,%s,%lx}",
- object->debug_id, object->state->name, object->events);
-
- event_mask = object->event_mask;
-restart:
- object->event_mask = 0; /* Mask normal event handling */
- state = object->state;
-restart_masked:
- events = object->events;
-
- /* Handle any out-of-band events (typically an error) */
- if (events & object->oob_event_mask) {
- _debug("{OBJ%x} oob %lx",
- object->debug_id, events & object->oob_event_mask);
- oob = true;
- for (t = object->oob_table; t->events; t++) {
- if (events & t->events) {
- state = t->transit_to;
- ASSERT(state->work != NULL);
- event = fls(events & t->events) - 1;
- __clear_bit(event, &object->oob_event_mask);
- clear_bit(event, &object->events);
- goto execute_work_state;
- }
- }
- }
- oob = false;
-
- /* Wait states are just transition tables */
- if (!state->work) {
- if (events & event_mask) {
- for (t = state->transitions; t->events; t++) {
- if (events & t->events) {
- new_state = t->transit_to;
- event = fls(events & t->events) - 1;
- trace_fscache_osm(object, state,
- true, false, event);
- clear_bit(event, &object->events);
- _debug("{OBJ%x} ev %d: %s -> %s",
- object->debug_id, event,
- state->name, new_state->name);
- object->state = state = new_state;
- goto execute_work_state;
- }
- }
-
- /* The event mask didn't include all the tabled bits */
- BUG();
- }
- /* Randomly woke up */
- goto unmask_events;
- }
-
-execute_work_state:
- _debug("{OBJ%x} exec %s", object->debug_id, state->name);
-
- trace_fscache_osm(object, state, false, oob, event);
- new_state = state->work(object, event);
- event = -1;
- if (new_state == NO_TRANSIT) {
- _debug("{OBJ%x} %s notrans", object->debug_id, state->name);
- if (unlikely(state == STATE(OBJECT_DEAD))) {
- _leave(" [dead]");
- return;
- }
- fscache_enqueue_object(object);
- event_mask = object->oob_event_mask;
- goto unmask_events;
- }
-
- _debug("{OBJ%x} %s -> %s",
- object->debug_id, state->name, new_state->name);
- object->state = state = new_state;
-
- if (state->work) {
- if (unlikely(state == STATE(OBJECT_DEAD))) {
- _leave(" [dead]");
- return;
- }
- goto restart_masked;
- }
-
- /* Transited to wait state */
- event_mask = object->oob_event_mask;
- for (t = state->transitions; t->events; t++)
- event_mask |= t->events;
-
-unmask_events:
- object->event_mask = event_mask;
- smp_mb();
- events = object->events;
- if (events & event_mask)
- goto restart;
- _leave(" [msk %lx]", event_mask);
-}
-
-/*
- * execute an object
- */
-static void fscache_object_work_func(struct work_struct *work)
-{
- struct fscache_object *object =
- container_of(work, struct fscache_object, work);
- unsigned long start;
-
- _enter("{OBJ%x}", object->debug_id);
-
- start = jiffies;
- fscache_object_sm_dispatcher(object);
- fscache_hist(fscache_objs_histogram, start);
- fscache_put_object(object, fscache_obj_put_work);
-}
-
-/**
- * fscache_object_init - Initialise a cache object description
- * @object: Object description
- * @cookie: Cookie object will be attached to
- * @cache: Cache in which backing object will be found
- *
- * Initialise a cache object description to its basic values.
- *
- * See Documentation/filesystems/caching/backend-api.txt for a complete
- * description.
- */
-void fscache_object_init(struct fscache_object *object,
- struct fscache_cookie *cookie,
- struct fscache_cache *cache)
-{
- const struct fscache_transition *t;
-
- atomic_inc(&cache->object_count);
-
- object->state = STATE(WAIT_FOR_INIT);
- object->oob_table = fscache_osm_init_oob;
- object->flags = 1 << FSCACHE_OBJECT_IS_LIVE;
- spin_lock_init(&object->lock);
- INIT_LIST_HEAD(&object->cache_link);
- INIT_HLIST_NODE(&object->cookie_link);
- INIT_WORK(&object->work, fscache_object_work_func);
- INIT_LIST_HEAD(&object->dependents);
- INIT_LIST_HEAD(&object->dep_link);
- object->n_children = 0;
- object->n_ops = 0;
- object->events = 0;
- object->cache = cache;
- object->cookie = cookie;
- fscache_cookie_get(cookie, fscache_cookie_get_attach_object);
- object->parent = NULL;
-#ifdef CONFIG_FSCACHE_OBJECT_LIST
- RB_CLEAR_NODE(&object->objlist_link);
-#endif
-
- object->oob_event_mask = 0;
- for (t = object->oob_table; t->events; t++)
- object->oob_event_mask |= t->events;
- object->event_mask = object->oob_event_mask;
- for (t = object->state->transitions; t->events; t++)
- object->event_mask |= t->events;
-}
-EXPORT_SYMBOL(fscache_object_init);
-
-/*
- * Mark the object as no longer being live, making sure that we synchronise
- * against op submission.
- */
-static inline void fscache_mark_object_dead(struct fscache_object *object)
-{
- spin_lock(&object->lock);
- clear_bit(FSCACHE_OBJECT_IS_LIVE, &object->flags);
- spin_unlock(&object->lock);
-}
-
-/*
- * Abort object initialisation before we start it.
- */
-static const struct fscache_state *fscache_abort_initialisation(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- object->oob_event_mask = 0;
- fscache_dequeue_object(object);
- return transit_to(KILL_OBJECT);
-}
-
-/*
- * initialise an object
- * - check the specified object's parent to see if we can make use of it
- * immediately to do a creation
- * - we may need to start the process of creating a parent and we need to wait
- * for the parent's lookup and creation to complete if it's not there yet
- */
-static const struct fscache_state *fscache_initialise_object(struct fscache_object *object,
- int event)
-{
- struct fscache_object *parent;
- bool success;
-
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- ASSERT(list_empty(&object->dep_link));
-
- parent = object->parent;
- if (!parent) {
- _leave(" [no parent]");
- return transit_to(DROP_OBJECT);
- }
-
- _debug("parent: %s of:%lx", parent->state->name, parent->flags);
-
- if (fscache_object_is_dying(parent)) {
- _leave(" [bad parent]");
- return transit_to(DROP_OBJECT);
- }
-
- if (fscache_object_is_available(parent)) {
- _leave(" [ready]");
- return transit_to(PARENT_READY);
- }
-
- _debug("wait");
-
- spin_lock(&parent->lock);
- fscache_stat(&fscache_n_cop_grab_object);
- success = false;
- if (fscache_object_is_live(parent) &&
- object->cache->ops->grab_object(object, fscache_obj_get_add_to_deps)) {
- list_add(&object->dep_link, &parent->dependents);
- success = true;
- }
- fscache_stat_d(&fscache_n_cop_grab_object);
- spin_unlock(&parent->lock);
- if (!success) {
- _leave(" [grab failed]");
- return transit_to(DROP_OBJECT);
- }
-
- /* fscache_acquire_non_index_cookie() uses this
- * to wake the chain up */
- fscache_raise_event(parent, FSCACHE_OBJECT_EV_NEW_CHILD);
- _leave(" [wait]");
- return transit_to(WAIT_FOR_PARENT);
-}
-
-/*
- * Once the parent object is ready, we should kick off our lookup op.
- */
-static const struct fscache_state *fscache_parent_ready(struct fscache_object *object,
- int event)
-{
- struct fscache_object *parent = object->parent;
-
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- ASSERT(parent != NULL);
-
- spin_lock(&parent->lock);
- parent->n_ops++;
- parent->n_obj_ops++;
- object->lookup_jif = jiffies;
- spin_unlock(&parent->lock);
-
- _leave("");
- return transit_to(LOOK_UP_OBJECT);
-}
-
-/*
- * look an object up in the cache from which it was allocated
- * - we hold an "access lock" on the parent object, so the parent object cannot
- * be withdrawn by either party till we've finished
- */
-static const struct fscache_state *fscache_look_up_object(struct fscache_object *object,
- int event)
-{
- struct fscache_cookie *cookie = object->cookie;
- struct fscache_object *parent = object->parent;
- int ret;
-
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- object->oob_table = fscache_osm_lookup_oob;
-
- ASSERT(parent != NULL);
- ASSERTCMP(parent->n_ops, >, 0);
- ASSERTCMP(parent->n_obj_ops, >, 0);
-
- /* make sure the parent is still available */
- ASSERT(fscache_object_is_available(parent));
-
- if (fscache_object_is_dying(parent) ||
- test_bit(FSCACHE_IOERROR, &object->cache->flags) ||
- !fscache_use_cookie(object)) {
- _leave(" [unavailable]");
- return transit_to(LOOKUP_FAILURE);
- }
-
- _debug("LOOKUP \"%s\" in \"%s\"",
- cookie->type_name, object->cache->tag->name);
-
- fscache_stat(&fscache_n_object_lookups);
- fscache_stat(&fscache_n_cop_lookup_object);
- ret = object->cache->ops->lookup_object(object);
- fscache_stat_d(&fscache_n_cop_lookup_object);
-
- fscache_unuse_cookie(object);
-
- if (ret == -ETIMEDOUT) {
- /* probably stuck behind another object, so move this one to
- * the back of the queue */
- fscache_stat(&fscache_n_object_lookups_timed_out);
- _leave(" [timeout]");
- return NO_TRANSIT;
- }
-
- if (ret < 0) {
- _leave(" [error]");
- return transit_to(LOOKUP_FAILURE);
- }
-
- _leave(" [ok]");
- return transit_to(OBJECT_AVAILABLE);
-}
-
-/**
- * fscache_object_lookup_negative - Note negative cookie lookup
- * @object: Object pointing to cookie to mark
- *
- * Note negative lookup, permitting those waiting to read data from an already
- * existing backing object to continue as there's no data for them to read.
- */
-void fscache_object_lookup_negative(struct fscache_object *object)
-{
- struct fscache_cookie *cookie = object->cookie;
-
- _enter("{OBJ%x,%s}", object->debug_id, object->state->name);
-
- if (!test_and_set_bit(FSCACHE_OBJECT_IS_LOOKED_UP, &object->flags)) {
- fscache_stat(&fscache_n_object_lookups_negative);
-
- /* Allow write requests to begin stacking up and read requests to begin
- * returning ENODATA.
- */
- set_bit(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
- clear_bit(FSCACHE_COOKIE_UNAVAILABLE, &cookie->flags);
-
- _debug("wake up lookup %p", &cookie->flags);
- clear_bit_unlock(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags);
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP);
- }
- _leave("");
-}
-EXPORT_SYMBOL(fscache_object_lookup_negative);
-
-/**
- * fscache_obtained_object - Note successful object lookup or creation
- * @object: Object pointing to cookie to mark
- *
- * Note successful lookup and/or creation, permitting those waiting to write
- * data to a backing object to continue.
- *
- * Note that after calling this, an object's cookie may be relinquished by the
- * netfs, and so must be accessed with object lock held.
- */
-void fscache_obtained_object(struct fscache_object *object)
-{
- struct fscache_cookie *cookie = object->cookie;
-
- _enter("{OBJ%x,%s}", object->debug_id, object->state->name);
-
- /* if we were still looking up, then we must have a positive lookup
- * result, in which case there may be data available */
- if (!test_and_set_bit(FSCACHE_OBJECT_IS_LOOKED_UP, &object->flags)) {
- fscache_stat(&fscache_n_object_lookups_positive);
-
- /* We do (presumably) have data */
- clear_bit_unlock(FSCACHE_COOKIE_NO_DATA_YET, &cookie->flags);
- clear_bit(FSCACHE_COOKIE_UNAVAILABLE, &cookie->flags);
-
- /* Allow write requests to begin stacking up and read requests
- * to begin shovelling data.
- */
- clear_bit_unlock(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags);
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP);
- } else {
- fscache_stat(&fscache_n_object_created);
- }
-
- set_bit(FSCACHE_OBJECT_IS_AVAILABLE, &object->flags);
- _leave("");
-}
-EXPORT_SYMBOL(fscache_obtained_object);
-
-/*
- * handle an object that has just become available
- */
-static const struct fscache_state *fscache_object_available(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- object->oob_table = fscache_osm_run_oob;
-
- spin_lock(&object->lock);
-
- fscache_done_parent_op(object);
- spin_unlock(&object->lock);
-
- fscache_stat(&fscache_n_cop_lookup_complete);
- object->cache->ops->lookup_complete(object);
- fscache_stat_d(&fscache_n_cop_lookup_complete);
-
- fscache_hist(fscache_obj_instantiate_histogram, object->lookup_jif);
- fscache_stat(&fscache_n_object_avail);
-
- _leave("");
- return transit_to(JUMPSTART_DEPS);
-}
-
-/*
- * Wake up this object's dependent objects now that we've become available.
- */
-static const struct fscache_state *fscache_jumpstart_dependents(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- if (!fscache_enqueue_dependents(object, FSCACHE_OBJECT_EV_PARENT_READY))
- return NO_TRANSIT; /* Not finished; requeue */
- return transit_to(WAIT_FOR_CMD);
-}
-
-/*
- * Handle lookup or creation failute.
- */
-static const struct fscache_state *fscache_lookup_failure(struct fscache_object *object,
- int event)
-{
- struct fscache_cookie *cookie;
-
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- object->oob_event_mask = 0;
-
- fscache_stat(&fscache_n_cop_lookup_complete);
- object->cache->ops->lookup_complete(object);
- fscache_stat_d(&fscache_n_cop_lookup_complete);
-
- set_bit(FSCACHE_OBJECT_KILLED_BY_CACHE, &object->flags);
-
- cookie = object->cookie;
- set_bit(FSCACHE_COOKIE_UNAVAILABLE, &cookie->flags);
- if (test_and_clear_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags))
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP);
-
- fscache_done_parent_op(object);
- return transit_to(KILL_OBJECT);
-}
-
-/*
- * Wait for completion of all active operations on this object and the death of
- * all child objects of this object.
- */
-static const struct fscache_state *fscache_kill_object(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x,%d,%d},%d",
- object->debug_id, object->n_ops, object->n_children, event);
-
- fscache_mark_object_dead(object);
- object->oob_event_mask = 0;
-
- if (list_empty(&object->dependents) &&
- object->n_ops == 0 &&
- object->n_children == 0)
- return transit_to(DROP_OBJECT);
-
- if (!list_empty(&object->dependents))
- return transit_to(KILL_DEPENDENTS);
-
- return transit_to(WAIT_FOR_CLEARANCE);
-}
-
-/*
- * Kill dependent objects.
- */
-static const struct fscache_state *fscache_kill_dependents(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- if (!fscache_enqueue_dependents(object, FSCACHE_OBJECT_EV_KILL))
- return NO_TRANSIT; /* Not finished */
- return transit_to(WAIT_FOR_CLEARANCE);
-}
-
-/*
- * Drop an object's attachments
- */
-static const struct fscache_state *fscache_drop_object(struct fscache_object *object,
- int event)
-{
- struct fscache_object *parent = object->parent;
- struct fscache_cookie *cookie = object->cookie;
- struct fscache_cache *cache = object->cache;
- bool awaken = false;
-
- _enter("{OBJ%x,%d},%d", object->debug_id, object->n_children, event);
-
- ASSERT(cookie != NULL);
- ASSERT(!hlist_unhashed(&object->cookie_link));
-
- if (test_bit(FSCACHE_COOKIE_AUX_UPDATED, &cookie->flags)) {
- _debug("final update");
- fscache_update_aux_data(object);
- }
-
- /* Make sure the cookie no longer points here and that the netfs isn't
- * waiting for us.
- */
- spin_lock(&cookie->lock);
- hlist_del_init(&object->cookie_link);
- if (hlist_empty(&cookie->backing_objects) &&
- test_and_clear_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
- awaken = true;
- spin_unlock(&cookie->lock);
-
- if (awaken)
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING);
- if (test_and_clear_bit(FSCACHE_COOKIE_LOOKING_UP, &cookie->flags))
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_LOOKING_UP);
-
-
- /* Prevent a race with our last child, which has to signal EV_CLEARED
- * before dropping our spinlock.
- */
- spin_lock(&object->lock);
- spin_unlock(&object->lock);
-
- /* Discard from the cache's collection of objects */
- spin_lock(&cache->object_list_lock);
- list_del_init(&object->cache_link);
- spin_unlock(&cache->object_list_lock);
-
- fscache_stat(&fscache_n_cop_drop_object);
- cache->ops->drop_object(object);
- fscache_stat_d(&fscache_n_cop_drop_object);
-
- /* The parent object wants to know when all it dependents have gone */
- if (parent) {
- _debug("release parent OBJ%x {%d}",
- parent->debug_id, parent->n_children);
-
- spin_lock(&parent->lock);
- parent->n_children--;
- if (parent->n_children == 0)
- fscache_raise_event(parent, FSCACHE_OBJECT_EV_CLEARED);
- spin_unlock(&parent->lock);
- object->parent = NULL;
- }
-
- /* this just shifts the object release to the work processor */
- fscache_put_object(object, fscache_obj_put_drop_obj);
- fscache_stat(&fscache_n_object_dead);
-
- _leave("");
- return transit_to(OBJECT_DEAD);
-}
-
-/*
- * get a ref on an object
- */
-static int fscache_get_object(struct fscache_object *object,
- enum fscache_obj_ref_trace why)
-{
- int ret;
-
- fscache_stat(&fscache_n_cop_grab_object);
- ret = object->cache->ops->grab_object(object, why) ? 0 : -EAGAIN;
- fscache_stat_d(&fscache_n_cop_grab_object);
- return ret;
-}
-
-/*
- * Discard a ref on an object
- */
-static void fscache_put_object(struct fscache_object *object,
- enum fscache_obj_ref_trace why)
-{
- fscache_stat(&fscache_n_cop_put_object);
- object->cache->ops->put_object(object, why);
- fscache_stat_d(&fscache_n_cop_put_object);
-}
-
-/**
- * fscache_object_destroy - Note that a cache object is about to be destroyed
- * @object: The object to be destroyed
- *
- * Note the imminent destruction and deallocation of a cache object record.
- */
-void fscache_object_destroy(struct fscache_object *object)
-{
- fscache_objlist_remove(object);
-
- /* We can get rid of the cookie now */
- fscache_cookie_put(object->cookie, fscache_cookie_put_object);
- object->cookie = NULL;
-}
-EXPORT_SYMBOL(fscache_object_destroy);
-
-/*
- * enqueue an object for metadata-type processing
- */
-void fscache_enqueue_object(struct fscache_object *object)
-{
- _enter("{OBJ%x}", object->debug_id);
-
- if (fscache_get_object(object, fscache_obj_get_queue) >= 0) {
- wait_queue_head_t *cong_wq =
- &get_cpu_var(fscache_object_cong_wait);
-
- if (queue_work(fscache_object_wq, &object->work)) {
- if (fscache_object_congested())
- wake_up(cong_wq);
- } else
- fscache_put_object(object, fscache_obj_put_queue);
-
- put_cpu_var(fscache_object_cong_wait);
- }
-}
-
-/**
- * fscache_object_sleep_till_congested - Sleep until object wq is congested
- * @timeoutp: Scheduler sleep timeout
- *
- * Allow an object handler to sleep until the object workqueue is congested.
- *
- * The caller must set up a wake up event before calling this and must have set
- * the appropriate sleep mode (such as TASK_UNINTERRUPTIBLE) and tested its own
- * condition before calling this function as no test is made here.
- *
- * %true is returned if the object wq is congested, %false otherwise.
- */
-bool fscache_object_sleep_till_congested(signed long *timeoutp)
-{
- wait_queue_head_t *cong_wq = this_cpu_ptr(&fscache_object_cong_wait);
- DEFINE_WAIT(wait);
-
- if (fscache_object_congested())
- return true;
-
- add_wait_queue_exclusive(cong_wq, &wait);
- if (!fscache_object_congested())
- *timeoutp = schedule_timeout(*timeoutp);
- finish_wait(cong_wq, &wait);
-
- return fscache_object_congested();
-}
-EXPORT_SYMBOL_GPL(fscache_object_sleep_till_congested);
-
-/*
- * Enqueue the dependents of an object for metadata-type processing.
- *
- * If we don't manage to finish the list before the scheduler wants to run
- * again then return false immediately. We return true if the list was
- * cleared.
- */
-static bool fscache_enqueue_dependents(struct fscache_object *object, int event)
-{
- struct fscache_object *dep;
- bool ret = true;
-
- _enter("{OBJ%x}", object->debug_id);
-
- if (list_empty(&object->dependents))
- return true;
-
- spin_lock(&object->lock);
-
- while (!list_empty(&object->dependents)) {
- dep = list_entry(object->dependents.next,
- struct fscache_object, dep_link);
- list_del_init(&dep->dep_link);
-
- fscache_raise_event(dep, event);
- fscache_put_object(dep, fscache_obj_put_enq_dep);
-
- if (!list_empty(&object->dependents) && need_resched()) {
- ret = false;
- break;
- }
- }
-
- spin_unlock(&object->lock);
- return ret;
-}
-
-/*
- * remove an object from whatever queue it's waiting on
- */
-static void fscache_dequeue_object(struct fscache_object *object)
-{
- _enter("{OBJ%x}", object->debug_id);
-
- if (!list_empty(&object->dep_link)) {
- spin_lock(&object->parent->lock);
- list_del_init(&object->dep_link);
- spin_unlock(&object->parent->lock);
- }
-
- _leave("");
-}
-
-static const struct fscache_state *fscache_invalidate_object(struct fscache_object *object,
- int event)
-{
- return transit_to(UPDATE_OBJECT);
-}
-
-/*
- * Update auxiliary data.
- */
-static void fscache_update_aux_data(struct fscache_object *object)
-{
- fscache_stat(&fscache_n_updates_run);
- fscache_stat(&fscache_n_cop_update_object);
- object->cache->ops->update_object(object);
- fscache_stat_d(&fscache_n_cop_update_object);
-}
-
-/*
- * Asynchronously update an object.
- */
-static const struct fscache_state *fscache_update_object(struct fscache_object *object,
- int event)
-{
- _enter("{OBJ%x},%d", object->debug_id, event);
-
- fscache_update_aux_data(object);
-
- _leave("");
- return transit_to(WAIT_FOR_CMD);
-}
-
-/**
- * fscache_object_retrying_stale - Note retrying stale object
- * @object: The object that will be retried
- *
- * Note that an object lookup found an on-disk object that was adjudged to be
- * stale and has been deleted. The lookup will be retried.
- */
-void fscache_object_retrying_stale(struct fscache_object *object)
-{
- fscache_stat(&fscache_n_cache_no_space_reject);
-}
-EXPORT_SYMBOL(fscache_object_retrying_stale);
-
-/**
- * fscache_object_mark_killed - Note that an object was killed
- * @object: The object that was culled
- * @why: The reason the object was killed.
- *
- * Note that an object was killed. Returns true if the object was
- * already marked killed, false if it wasn't.
- */
-void fscache_object_mark_killed(struct fscache_object *object,
- enum fscache_why_object_killed why)
-{
- if (test_and_set_bit(FSCACHE_OBJECT_KILLED_BY_CACHE, &object->flags)) {
- pr_err("Error: Object already killed by cache [%s]\n",
- object->cache->identifier);
- return;
- }
-
- switch (why) {
- case FSCACHE_OBJECT_NO_SPACE:
- fscache_stat(&fscache_n_cache_no_space_reject);
- break;
- case FSCACHE_OBJECT_IS_STALE:
- fscache_stat(&fscache_n_cache_stale_objects);
- break;
- case FSCACHE_OBJECT_WAS_RETIRED:
- fscache_stat(&fscache_n_cache_retired_objects);
- break;
- case FSCACHE_OBJECT_WAS_CULLED:
- fscache_stat(&fscache_n_cache_culled_objects);
- break;
- }
-}
-EXPORT_SYMBOL(fscache_object_mark_killed);
-
-/*
- * The object is dead. We can get here if an object gets queued by an event
- * that would lead to its death (such as EV_KILL) when the dispatcher is
- * already running (and so can be requeued) but hasn't yet cleared the event
- * mask.
- */
-static const struct fscache_state *fscache_object_dead(struct fscache_object *object,
- int event)
-{
- if (!test_and_set_bit(FSCACHE_OBJECT_RUN_AFTER_DEAD,
- &object->flags))
- return NO_TRANSIT;
-
- WARN(true, "FS-Cache object redispatched after death");
- return NO_TRANSIT;
-}
diff --git a/fs/fscache/object_bits.c b/fs/fscache/object_bits.c
new file mode 100644
index 000000000000..5316d7d33ce0
--- /dev/null
+++ b/fs/fscache/object_bits.c
@@ -0,0 +1,120 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Miscellaneous object routines.
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * See Documentation/filesystems/caching/netfs-api.txt for more information on
+ * the netfs API.
+ */
+
+#define FSCACHE_DEBUG_LEVEL OBJECT
+#include <linux/module.h>
+#include <linux/slab.h>
+#include "internal.h"
+
+static atomic_t fscache_object_debug_id;
+
+/**
+ * fscache_object_init - Initialise a cache object description
+ * @object: Object description
+ * @cookie: Cookie object will be attached to
+ * @cache: Cache in which backing object will be found
+ *
+ * Initialise a cache object description to its basic values.
+ *
+ * See Documentation/filesystems/caching/backend-api.txt for a complete
+ * description.
+ */
+void fscache_object_init(struct fscache_object *object,
+ struct fscache_cookie *cookie,
+ struct fscache_cache *cache)
+{
+ atomic_inc(&cache->object_count);
+
+ spin_lock_init(&object->lock);
+ INIT_LIST_HEAD(&object->cache_link);
+ INIT_HLIST_NODE(&object->cookie_link);
+ object->n_children = 0;
+ object->cache = cache;
+ object->cookie = fscache_cookie_get(cookie, fscache_cookie_get_attach_object);
+ object->parent = NULL;
+#ifdef CONFIG_FSCACHE_OBJECT_LIST
+ RB_CLEAR_NODE(&object->objlist_link);
+#endif
+ object->debug_id = atomic_inc_return(&fscache_object_debug_id);
+}
+EXPORT_SYMBOL(fscache_object_init);
+
+/**
+ * fscache_object_destroy - Note that a cache object is about to be destroyed
+ * @object: The object to be destroyed
+ *
+ * Note the imminent destruction and deallocation of a cache object record.
+ */
+void fscache_object_destroy(struct fscache_object *object)
+{
+ _enter("%u", atomic_read(&object->cache->object_count));
+
+ fscache_objlist_remove(object);
+
+ /* We can get rid of the cookie now */
+ fscache_cookie_put(object->cookie, fscache_cookie_put_object);
+ object->cookie = NULL;
+}
+EXPORT_SYMBOL(fscache_object_destroy);
+
+/**
+ * fscache_object_destroyed - Note destruction of an object in a cache
+ * @cache: The cache from which the object came
+ *
+ * Note the destruction and deallocation of an object record in a cache.
+ */
+void fscache_object_destroyed(struct fscache_cache *cache)
+{
+ _enter("%d", atomic_read(&cache->object_count));
+ if (atomic_dec_and_test(&cache->object_count))
+ wake_up_all(&fscache_cache_cleared_wq);
+}
+EXPORT_SYMBOL(fscache_object_destroyed);
+
+/**
+ * fscache_object_mark_killed - Note that an object was killed
+ * @object: The object that was culled
+ * @why: The reason the object was killed.
+ *
+ * Note that an object was killed. Returns true if the object was
+ * already marked killed, false if it wasn't.
+ */
+void fscache_object_mark_killed(struct fscache_object *object,
+ enum fscache_why_object_killed why)
+{
+ switch (why) {
+ case FSCACHE_OBJECT_NO_SPACE:
+ fscache_stat(&fscache_n_cache_no_space_reject);
+ break;
+ case FSCACHE_OBJECT_IS_STALE:
+ fscache_stat(&fscache_n_cache_stale_objects);
+ break;
+ case FSCACHE_OBJECT_WAS_RETIRED:
+ fscache_stat(&fscache_n_cache_retired_objects);
+ break;
+ case FSCACHE_OBJECT_WAS_CULLED:
+ fscache_stat(&fscache_n_cache_culled_objects);
+ break;
+ }
+}
+EXPORT_SYMBOL(fscache_object_mark_killed);
+
+/**
+ * fscache_object_retrying_stale - Note retrying stale object
+ * @object: The object that will be retried
+ *
+ * Note that an object lookup found an on-disk object that was adjudged to be
+ * stale and has been deleted. The lookup will be retried.
+ */
+void fscache_object_retrying_stale(struct fscache_object *object)
+{
+ fscache_stat(&fscache_n_cache_stale_objects);
+}
+EXPORT_SYMBOL(fscache_object_retrying_stale);
diff --git a/fs/fscache/proc.c b/fs/fscache/proc.c
index da51fdfc8641..729d083f1e91 100644
--- a/fs/fscache/proc.c
+++ b/fs/fscache/proc.c
@@ -5,7 +5,7 @@
* Written by David Howells (dhowells@redhat.com)
*/
-#define FSCACHE_DEBUG_LEVEL OPERATION
+#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
@@ -16,8 +16,6 @@
*/
int __init fscache_proc_init(void)
{
- _enter("");
-
if (!proc_mkdir("fs/fscache", NULL))
goto error_dir;
@@ -43,7 +41,6 @@ int __init fscache_proc_init(void)
goto error_objects;
#endif
- _leave(" = 0");
return 0;
#ifdef CONFIG_FSCACHE_OBJECT_LIST
@@ -61,7 +58,6 @@ int __init fscache_proc_init(void)
error_cookies:
remove_proc_entry("fs/fscache", NULL);
error_dir:
- _leave(" = -ENOMEM");
return -ENOMEM;
}
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 5b1cec456199..583817f4f113 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -5,7 +5,7 @@
* Written by David Howells (dhowells@redhat.com)
*/
-#define FSCACHE_DEBUG_LEVEL THREAD
+#define FSCACHE_DEBUG_LEVEL CACHE
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 5e63d4a7feb0..fbf1b60149aa 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -16,7 +16,6 @@
#include <linux/fscache.h>
#include <linux/sched.h>
-#include <linux/workqueue.h>
#define NR_MAXCACHES BITS_PER_LONG
@@ -25,14 +24,21 @@ struct fscache_cache_ops;
struct fscache_object;
enum fscache_obj_ref_trace {
- fscache_obj_get_add_to_deps,
- fscache_obj_get_queue,
+ fscache_obj_get_attach,
+ fscache_obj_get_exists,
+ fscache_obj_get_inval,
+ fscache_obj_get_wait,
+ fscache_obj_get_withdraw,
+ fscache_obj_new,
+ fscache_obj_put,
+ fscache_obj_put_alloc_dup,
fscache_obj_put_alloc_fail,
fscache_obj_put_attach_fail,
+ fscache_obj_put_drop_child,
fscache_obj_put_drop_obj,
- fscache_obj_put_enq_dep,
- fscache_obj_put_queue,
- fscache_obj_put_work,
+ fscache_obj_put_inval,
+ fscache_obj_put_lookup_fail,
+ fscache_obj_put_withdraw,
fscache_obj_ref__nr_traces
};
@@ -80,16 +86,21 @@ struct fscache_cache_ops {
const char *name;
/* allocate an object record for a cookie */
- struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
- struct fscache_cookie *cookie);
+ struct fscache_object *(*alloc_object)(struct fscache_cookie *cookie,
+ struct fscache_cache *cache,
+ struct fscache_object *parent);
- /* look up the object for a cookie
- * - return -ETIMEDOUT to be requeued
- */
- int (*lookup_object)(struct fscache_object *object);
+ /* Prepare data used in lookup */
+ void *(*prepare_lookup_data)(struct fscache_object *object);
- /* finished looking up */
- void (*lookup_complete)(struct fscache_object *object);
+ /* Look up the object for a cookie */
+ bool (*lookup_object)(struct fscache_object *object, void *lookup_data);
+
+ /* Create the object for a cookie */
+ bool (*create_object)(struct fscache_object *object, void *lookup_data);
+
+ /* Clean up lookup data */
+ void (*free_lookup_data)(struct fscache_object *object, void *lookup_data);
/* increment the usage count on this object (may fail if unmounting) */
struct fscache_object *(*grab_object)(struct fscache_object *object,
@@ -109,12 +120,15 @@ struct fscache_cache_ops {
/* discard the resources pinned by an object and effect retirement if
* necessary */
- void (*drop_object)(struct fscache_object *object);
+ void (*drop_object)(struct fscache_object *object, bool invalidate);
/* dispose of a reference to an object */
void (*put_object)(struct fscache_object *object,
enum fscache_obj_ref_trace why);
+ /* Get object usage count */
+ unsigned int (*get_object_usage)(const struct fscache_object *object);
+
/* sync a cache */
void (*sync_cache)(struct fscache_cache *cache);
@@ -124,74 +138,34 @@ struct fscache_cache_ops {
extern struct fscache_cookie fscache_fsdef_index;
-/*
- * Event list for fscache_object::{event_mask,events}
- */
-enum {
- FSCACHE_OBJECT_EV_NEW_CHILD, /* T if object has a new child */
- FSCACHE_OBJECT_EV_PARENT_READY, /* T if object's parent is ready */
- FSCACHE_OBJECT_EV_UPDATE, /* T if object should be updated */
- FSCACHE_OBJECT_EV_INVALIDATE, /* T if cache requested object invalidation */
- FSCACHE_OBJECT_EV_CLEARED, /* T if accessors all gone */
- FSCACHE_OBJECT_EV_ERROR, /* T if fatal error occurred during processing */
- FSCACHE_OBJECT_EV_KILL, /* T if netfs relinquished or cache withdrew object */
- NR_FSCACHE_OBJECT_EVENTS
-};
-
-#define FSCACHE_OBJECT_EVENTS_MASK ((1UL << NR_FSCACHE_OBJECT_EVENTS) - 1)
-
-/*
- * States for object state machine.
- */
-struct fscache_transition {
- unsigned long events;
- const struct fscache_state *transit_to;
-};
-
-struct fscache_state {
- char name[24];
- char short_name[8];
- const struct fscache_state *(*work)(struct fscache_object *object,
- int event);
- const struct fscache_transition transitions[];
+enum fscache_object_stage {
+ FSCACHE_OBJECT_STAGE_INITIAL,
+ FSCACHE_OBJECT_STAGE_LOOKING_UP,
+ FSCACHE_OBJECT_STAGE_UNCREATED, /* Needs creation */
+ FSCACHE_OBJECT_STAGE_LIVE_TEMP, /* Temporary object created, can be no hits */
+ FSCACHE_OBJECT_STAGE_LIVE_EMPTY, /* Object was freshly created, can be no hits */
+ FSCACHE_OBJECT_STAGE_LIVE, /* Object is populated */
+ FSCACHE_OBJECT_STAGE_DESTROYING,
+ FSCACHE_OBJECT_STAGE_DEAD,
};
/*
* on-disk cache file or index handle
*/
struct fscache_object {
- const struct fscache_state *state; /* Object state machine state */
- const struct fscache_transition *oob_table; /* OOB state transition table */
int debug_id; /* debugging ID */
int n_children; /* number of child objects */
- int n_ops; /* number of extant ops on object */
- int n_obj_ops; /* number of object ops outstanding on object */
+ enum fscache_object_stage stage; /* Stage of object's lifecycle */
spinlock_t lock; /* state and operations lock */
- unsigned long lookup_jif; /* time at which lookup started */
- unsigned long oob_event_mask; /* OOB events this object is interested in */
- unsigned long event_mask; /* events this object is interested in */
- unsigned long events; /* events to be processed by this object
- * (order is important - using fls) */
-
unsigned long flags;
-#define FSCACHE_OBJECT_LOCK 0 /* T if object is busy being processed */
-#define FSCACHE_OBJECT_WAITING 2 /* T if object is waiting on its parent */
-#define FSCACHE_OBJECT_IS_LIVE 3 /* T if object is not withdrawn or relinquished */
-#define FSCACHE_OBJECT_IS_LOOKED_UP 4 /* T if object has been looked up */
-#define FSCACHE_OBJECT_IS_AVAILABLE 5 /* T if object has become active */
-#define FSCACHE_OBJECT_RETIRED 6 /* T if object was retired on relinquishment */
-#define FSCACHE_OBJECT_KILLED_BY_CACHE 7 /* T if object was killed by the cache */
-#define FSCACHE_OBJECT_RUN_AFTER_DEAD 8 /* T if object has been dispatched after death */
+#define FSCACHE_OBJECT_NEEDS_UPDATE 9 /* T if object attrs need writing to disk */
struct list_head cache_link; /* link in cache->object_list */
struct hlist_node cookie_link; /* link in cookie->backing_objects */
struct fscache_cache *cache; /* cache that supplied this object */
struct fscache_cookie *cookie; /* netfs's file/index object */
struct fscache_object *parent; /* parent object */
- struct work_struct work; /* attention scheduling record */
- struct list_head dependents; /* FIFO of dependent objects */
- struct list_head dep_link; /* link in parent's dependents list */
#ifdef CONFIG_FSCACHE_OBJECT_LIST
struct rb_node objlist_link; /* link in global object list */
#endif
@@ -201,101 +175,12 @@ extern void fscache_object_init(struct fscache_object *, struct fscache_cookie *
struct fscache_cache *);
extern void fscache_object_destroy(struct fscache_object *);
-extern void fscache_object_lookup_negative(struct fscache_object *object);
-extern void fscache_obtained_object(struct fscache_object *object);
-
-static inline bool fscache_object_is_live(struct fscache_object *object)
-{
- return test_bit(FSCACHE_OBJECT_IS_LIVE, &object->flags);
-}
-
-static inline bool fscache_object_is_dying(struct fscache_object *object)
-{
- return !fscache_object_is_live(object);
-}
-
-static inline bool fscache_object_is_available(struct fscache_object *object)
-{
- return test_bit(FSCACHE_OBJECT_IS_AVAILABLE, &object->flags);
-}
-
static inline bool fscache_cache_is_broken(struct fscache_object *object)
{
return test_bit(FSCACHE_IOERROR, &object->cache->flags);
}
-static inline bool fscache_object_is_active(struct fscache_object *object)
-{
- return fscache_object_is_available(object) &&
- fscache_object_is_live(object) &&
- !fscache_cache_is_broken(object);
-}
-
-/**
- * fscache_object_destroyed - Note destruction of an object in a cache
- * @cache: The cache from which the object came
- *
- * Note the destruction and deallocation of an object record in a cache.
- */
-static inline void fscache_object_destroyed(struct fscache_cache *cache)
-{
- if (atomic_dec_and_test(&cache->object_count))
- wake_up_all(&fscache_cache_cleared_wq);
-}
-
-/**
- * fscache_object_lookup_error - Note an object encountered an error
- * @object: The object on which the error was encountered
- *
- * Note that an object encountered a fatal error (usually an I/O error) and
- * that it should be withdrawn as soon as possible.
- */
-static inline void fscache_object_lookup_error(struct fscache_object *object)
-{
- set_bit(FSCACHE_OBJECT_EV_ERROR, &object->events);
-}
-
-static inline void __fscache_use_cookie(struct fscache_cookie *cookie)
-{
- atomic_inc(&cookie->n_active);
-}
-
-/**
- * fscache_use_cookie - Request usage of cookie attached to an object
- * @object: Object description
- *
- * Request usage of the cookie attached to an object. NULL is returned if the
- * relinquishment had reduced the cookie usage count to 0.
- */
-static inline bool fscache_use_cookie(struct fscache_object *object)
-{
- struct fscache_cookie *cookie = object->cookie;
- return atomic_inc_not_zero(&cookie->n_active) != 0;
-}
-
-static inline bool __fscache_unuse_cookie(struct fscache_cookie *cookie)
-{
- return atomic_dec_and_test(&cookie->n_active);
-}
-
-static inline void __fscache_wake_unused_cookie(struct fscache_cookie *cookie)
-{
- wake_up_var(&cookie->n_active);
-}
-
-/**
- * fscache_unuse_cookie - Cease usage of cookie attached to an object
- * @object: Object description
- *
- * Cease usage of the cookie attached to an object. When the users count
- * reaches zero then the cookie relinquishment will be permitted to proceed.
- */
-static inline void fscache_unuse_cookie(struct fscache_object *object)
-{
- struct fscache_cookie *cookie = object->cookie;
- if (__fscache_unuse_cookie(cookie))
- __fscache_wake_unused_cookie(cookie);
-}
+extern void fscache_object_destroyed(struct fscache_cache *cache);
/*
* out-of-line cache backend functions
@@ -312,8 +197,6 @@ extern void fscache_withdraw_cache(struct fscache_cache *cache);
extern void fscache_io_error(struct fscache_cache *cache);
-extern bool fscache_object_sleep_till_congested(signed long *timeoutp);
-
extern void fscache_object_retrying_stale(struct fscache_object *object);
enum fscache_why_object_killed {
@@ -347,4 +230,13 @@ static inline void *fscache_get_aux(struct fscache_cookie *cookie)
return cookie->aux;
}
+/*
+ * Complete an I/O operation
+ */
+static inline void fscache_end_io_operation(struct fscache_cookie *cookie)
+{
+ if (atomic_dec_and_test(&cookie->n_ops))
+ wake_up_var(&cookie->n_ops);
+}
+
#endif /* _LINUX_FSCACHE_CACHE_H */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 5eb2ac5f2bf1..e4fe28cfdf5c 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -31,7 +31,7 @@
/*
* overload PG_private_2 to give us PG_fscache - this is used to indicate that
- * a page is currently backed by a local disk cache
+ * a page is currently being written to the cache, possibly by direct I/O.
*/
#define PageFsCache(page) PagePrivate2((page))
#define SetPageFsCache(page) SetPagePrivate2((page))
@@ -52,6 +52,8 @@ enum fscache_cookie_type {
};
#define FSCACHE_ADV_SINGLE_CHUNK 0x01 /* The object is a single chunk of data */
+#define FSCACHE_ADV_WRITE_CACHE 0x00 /* Do cache if written to locally */
+#define FSCACHE_ADV_WRITE_NOCACHE 0x02 /* Don't cache if written to locally */
/*
* fscache cached network filesystem type
@@ -64,6 +66,19 @@ struct fscache_netfs {
struct fscache_cookie *primary_index;
};
+/*
+ * Data object state.
+ */
+enum fscache_cookie_stage {
+ FSCACHE_COOKIE_STAGE_QUIESCENT, /* The cookie is uncached */
+ FSCACHE_COOKIE_STAGE_INITIALISING, /* The in-memory structs are being inited */
+ FSCACHE_COOKIE_STAGE_LOOKING_UP, /* The cache object is being looked up */
+ FSCACHE_COOKIE_STAGE_NO_DATA_YET, /* The cache has no data, read to network */
+ FSCACHE_COOKIE_STAGE_ACTIVE, /* The cache is active, readable and writable */
+ FSCACHE_COOKIE_STAGE_INVALIDATING, /* The cache is being invalidated */
+ FSCACHE_COOKIE_STAGE_DEAD, /* The cache object is dead */
+} __attribute__((mode(byte)));
+
/*
* data file or index object cookie
* - a file will only appear in one cache
@@ -74,7 +89,8 @@ struct fscache_netfs {
struct fscache_cookie {
atomic_t usage; /* number of users of this cookie */
atomic_t n_children; /* number of children of this cookie */
- atomic_t n_active; /* number of active users of netfs ptrs */
+ atomic_t n_active; /* number of active users of cookie */
+ atomic_t n_ops; /* Number of active ops on this cookie */
unsigned int debug_id;
spinlock_t lock;
struct hlist_head backing_objects; /* object(s) backing this file/index */
@@ -86,19 +102,13 @@ struct fscache_cookie {
loff_t object_size; /* Size of the netfs object */
unsigned long flags;
-#define FSCACHE_COOKIE_LOOKING_UP 0 /* T if non-index cookie being looked up still */
-#define FSCACHE_COOKIE_NO_DATA_YET 1 /* T if new object with no cached data yet */
-#define FSCACHE_COOKIE_UNAVAILABLE 2 /* T if cookie is unavailable (error, etc) */
-#define FSCACHE_COOKIE_INVALIDATING 3 /* T if cookie is being invalidated */
-#define FSCACHE_COOKIE_RELINQUISHED 4 /* T if cookie has been relinquished */
-#define FSCACHE_COOKIE_ENABLED 5 /* T if cookie is enabled */
-#define FSCACHE_COOKIE_ENABLEMENT_LOCK 6 /* T if cookie is being en/disabled */
-#define FSCACHE_COOKIE_AUX_UPDATED 8 /* T if the auxiliary data was updated */
-#define FSCACHE_COOKIE_ACQUIRED 9 /* T if cookie is in use */
-#define FSCACHE_COOKIE_RELINQUISHING 10 /* T if cookie is being relinquished */
+#define FSCACHE_COOKIE_INVALIDATING 4 /* T if cookie is being invalidated */
+#define FSCACHE_COOKIE_ACQUIRED 5 /* T if cookie is in use */
+#define FSCACHE_COOKIE_RELINQUISHED 6 /* T if cookie has been relinquished */
+ enum fscache_cookie_stage stage;
enum fscache_cookie_type type:8;
- u8 advice; /* FSCACHE_COOKIE_ADV_* */
+ u8 advice; /* FSCACHE_ADV_* */
u8 key_len; /* Length of index key */
u8 aux_len; /* Length of auxiliary data */
u32 key_hash; /* Hash of parent, type, key, len */
@@ -112,11 +122,6 @@ struct fscache_cookie {
};
};
-static inline bool fscache_cookie_enabled(struct fscache_cookie *cookie)
-{
- return test_bit(FSCACHE_COOKIE_ENABLED, &cookie->flags);
-}
-
/*
* slow-path functions for when there is actually caching available, and the
* netfs does actually have a valid token
@@ -137,14 +142,13 @@ extern struct fscache_cookie *__fscache_acquire_cookie(
struct fscache_cache_tag *,
const void *, size_t,
const void *, size_t,
- loff_t, bool);
-extern void __fscache_relinquish_cookie(struct fscache_cookie *, const void *, bool);
-extern void __fscache_update_cookie(struct fscache_cookie *, const void *);
+ loff_t);
+extern void __fscache_use_cookie(struct fscache_cookie *, bool);
+extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *);
+extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
+extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_invalidate(struct fscache_cookie *);
extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
-extern void __fscache_disable_cookie(struct fscache_cookie *, const void *, bool);
-extern void __fscache_enable_cookie(struct fscache_cookie *, const void *, loff_t,
- bool (*)(void *), void *);
/**
* fscache_register_netfs - Register a filesystem as desiring caching services
@@ -231,7 +235,6 @@ void fscache_release_cache_tag(struct fscache_cache_tag *tag)
* @netfs_data: An arbitrary piece of data to be kept in the cookie to
* represent the cache object to the netfs
* @object_size: The initial size of object
- * @enable: Whether or not to enable a data cookie immediately
*
* This function is used to inform FS-Cache about part of an index hierarchy
* that can be used to locate files. This is done by requesting a cookie for
@@ -251,60 +254,92 @@ struct fscache_cookie *fscache_acquire_cookie(
size_t index_key_len,
const void *aux_data,
size_t aux_data_len,
- loff_t object_size,
- bool enable)
+ loff_t object_size)
{
- if (fscache_cookie_valid(parent) && fscache_cookie_enabled(parent))
+ if (fscache_cookie_valid(parent))
return __fscache_acquire_cookie(parent, type, type_name, advice,
preferred_cache,
index_key, index_key_len,
aux_data, aux_data_len,
- object_size, enable);
+ object_size);
else
return NULL;
}
+/**
+ * fscache_use_cookie - Request usage of cookie attached to an object
+ * @object: Object description
+ * @will_modify: If cache is expected to be modified locally
+ *
+ * Request usage of the cookie attached to an object. The caller should tell
+ * the cache if the object's contents are about to be modified locally and then
+ * the cache can apply the policy that has been set to handle this case.
+ */
+static inline void fscache_use_cookie(struct fscache_cookie *cookie,
+ bool will_modify)
+{
+ if (fscache_cookie_valid(cookie) &&
+ cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
+ __fscache_use_cookie(cookie, will_modify);
+}
+
+/**
+ * fscache_unuse_cookie - Cease usage of cookie attached to an object
+ * @object: Object description
+ * @aux_data: Updated auxiliary data (or NULL)
+ * @object_size: Revised size of the object (or NULL)
+ *
+ * Cease usage of the cookie attached to an object. When the users count
+ * reaches zero then the cookie relinquishment will be permitted to proceed.
+ */
+static inline void fscache_unuse_cookie(struct fscache_cookie *cookie,
+ const void *aux_data,
+ const loff_t *object_size)
+{
+ if (fscache_cookie_valid(cookie) &&
+ cookie->type != FSCACHE_COOKIE_TYPE_INDEX)
+ __fscache_unuse_cookie(cookie, aux_data, object_size);
+}
+
/**
* fscache_relinquish_cookie - Return the cookie to the cache, maybe discarding
* it
* @cookie: The cookie being returned
- * @aux_data: The updated auxiliary data for the cookie (may be NULL)
* @retire: True if the cache object the cookie represents is to be discarded
*
* This function returns a cookie to the cache, forcibly discarding the
- * associated cache object if retire is set to true. The opportunity is
- * provided to update the auxiliary data in the cache before the object is
- * disconnected.
+ * associated cache object if retire is set to true.
*
* See Documentation/filesystems/caching/netfs-api.txt for a complete
* description.
*/
static inline
-void fscache_relinquish_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- bool retire)
+void fscache_relinquish_cookie(struct fscache_cookie *cookie, bool retire)
{
if (fscache_cookie_valid(cookie))
- __fscache_relinquish_cookie(cookie, aux_data, retire);
+ __fscache_relinquish_cookie(cookie, retire);
}
/**
* fscache_update_cookie - Request that a cache object be updated
* @cookie: The cookie representing the cache object
* @aux_data: The updated auxiliary data for the cookie (may be NULL)
+ * @object_size: The current size of the object (may be NULL)
*
* Request an update of the index data for the cache object associated with the
* cookie. The auxiliary data on the cookie will be updated first if @aux_data
- * is set.
+ * is set and the object size will be updated and the object possibly trimmed
+ * if @object_size is set.
*
* See Documentation/filesystems/caching/netfs-api.txt for a complete
* description.
*/
static inline
-void fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data)
+void fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data,
+ const loff_t *object_size)
{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- __fscache_update_cookie(cookie, aux_data);
+ if (fscache_cookie_valid(cookie))
+ __fscache_update_cookie(cookie, aux_data, object_size);
}
/**
@@ -352,7 +387,7 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
static inline
void fscache_invalidate(struct fscache_cookie *cookie)
{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
+ if (fscache_cookie_valid(cookie))
__fscache_invalidate(cookie);
}
@@ -372,58 +407,4 @@ void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
__fscache_wait_on_invalidate(cookie);
}
-/**
- * fscache_disable_cookie - Disable a cookie
- * @cookie: The cookie representing the cache object
- * @aux_data: The updated auxiliary data for the cookie (may be NULL)
- * @invalidate: Invalidate the backing object
- *
- * Disable a cookie from accepting further alloc, read, write, invalidate,
- * update or acquire operations. Outstanding operations can still be waited
- * upon and pages can still be uncached and the cookie relinquished.
- *
- * This will not return until all outstanding operations have completed.
- *
- * If @invalidate is set, then the backing object will be invalidated and
- * detached, otherwise it will just be detached.
- *
- * If @aux_data is set, then auxiliary data will be updated from that.
- */
-static inline
-void fscache_disable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- bool invalidate)
-{
- if (fscache_cookie_valid(cookie) && fscache_cookie_enabled(cookie))
- __fscache_disable_cookie(cookie, aux_data, invalidate);
-}
-
-/**
- * fscache_enable_cookie - Reenable a cookie
- * @cookie: The cookie representing the cache object
- * @aux_data: The updated auxiliary data for the cookie (may be NULL)
- * @object_size: Current size of object
- * @can_enable: A function to permit enablement once lock is held
- * @data: Data for can_enable()
- *
- * Reenable a previously disabled cookie, allowing it to accept further alloc,
- * read, write, invalidate, update or acquire operations. An attempt will be
- * made to immediately reattach the cookie to a backing object. If @aux_data
- * is set, the auxiliary data attached to the cookie will be updated.
- *
- * The can_enable() function is called (if not NULL) once the enablement lock
- * is held to rule on whether enablement is still permitted to go ahead.
- */
-static inline
-void fscache_enable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- loff_t object_size,
- bool (*can_enable)(void *data),
- void *data)
-{
- if (fscache_cookie_valid(cookie) && !fscache_cookie_enabled(cookie))
- __fscache_enable_cookie(cookie, aux_data, object_size,
- can_enable, data);
-}
-
#endif /* _LINUX_FSCACHE_H */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index c877035c2946..4fedc2e9c428 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -36,14 +36,21 @@ enum cachefiles_obj_ref_trace {
E_(FSCACHE_OBJECT_WAS_CULLED, "was_culled")
#define cachefiles_obj_ref_traces \
- EM(fscache_obj_get_add_to_deps, "GET add_to_deps") \
- EM(fscache_obj_get_queue, "GET queue") \
+ EM(fscache_obj_get_attach, "GET attach") \
+ EM(fscache_obj_get_exists, "GET exists") \
+ EM(fscache_obj_get_inval, "GET inval") \
+ EM(fscache_obj_get_wait, "GET wait") \
+ EM(fscache_obj_get_withdraw, "GET withdraw") \
+ EM(fscache_obj_new, "NEW obj") \
+ EM(fscache_obj_put, "PUT general") \
+ EM(fscache_obj_put_alloc_dup, "PUT alloc_dup") \
EM(fscache_obj_put_alloc_fail, "PUT alloc_fail") \
EM(fscache_obj_put_attach_fail, "PUT attach_fail") \
+ EM(fscache_obj_put_drop_child, "PUT drop_child") \
EM(fscache_obj_put_drop_obj, "PUT drop_obj") \
- EM(fscache_obj_put_enq_dep, "PUT enq_dep") \
- EM(fscache_obj_put_queue, "PUT queue") \
- EM(fscache_obj_put_work, "PUT work") \
+ EM(fscache_obj_put_inval, "PUT inval") \
+ EM(fscache_obj_put_withdraw, "PUT withdraw") \
+ EM(fscache_obj_put_lookup_fail, "PUT lookup_fail") \
EM(cachefiles_obj_put_wait_retry, "PUT wait_retry") \
E_(cachefiles_obj_put_wait_timeo, "PUT wait_timeo")
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index fb3fdf2921ee..4ce81f3f4c5b 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -24,15 +24,19 @@ enum fscache_cookie_trace {
fscache_cookie_discard,
fscache_cookie_get_acquire_parent,
fscache_cookie_get_attach_object,
+ fscache_cookie_get_ioreq,
fscache_cookie_get_reacquire,
fscache_cookie_get_register_netfs,
fscache_cookie_get_work,
- fscache_cookie_put_acquire_nobufs,
+ fscache_cookie_new_acquire,
+ fscache_cookie_new_netfs,
fscache_cookie_put_dup_netfs,
- fscache_cookie_put_relinquish,
+ fscache_cookie_put_ioreq,
fscache_cookie_put_object,
fscache_cookie_put_parent,
+ fscache_cookie_put_relinquish,
fscache_cookie_put_work,
+ fscache_cookie_see_discard,
};
#endif
@@ -41,19 +45,23 @@ enum fscache_cookie_trace {
* Declare tracing information enums and their string mappings for display.
*/
#define fscache_cookie_traces \
- EM(fscache_cookie_collision, "*COLLISION*") \
- EM(fscache_cookie_discard, "DISCARD") \
- EM(fscache_cookie_get_acquire_parent, "GET prn") \
- EM(fscache_cookie_get_attach_object, "GET obj") \
- EM(fscache_cookie_get_reacquire, "GET raq") \
- EM(fscache_cookie_get_register_netfs, "GET net") \
- EM(fscache_cookie_get_work, "GET wrk") \
- EM(fscache_cookie_put_acquire_nobufs, "PUT nbf") \
- EM(fscache_cookie_put_dup_netfs, "PUT dnt") \
- EM(fscache_cookie_put_relinquish, "PUT rlq") \
- EM(fscache_cookie_put_object, "PUT obj") \
- EM(fscache_cookie_put_parent, "PUT prn") \
- E_(fscache_cookie_put_work, "PUT wrk")
+ EM(fscache_cookie_collision, "*COLLIDE*") \
+ EM(fscache_cookie_discard, "DISCARD ") \
+ EM(fscache_cookie_get_acquire_parent, "GET paren") \
+ EM(fscache_cookie_get_attach_object, "GET obj ") \
+ EM(fscache_cookie_get_ioreq, "GET ioreq") \
+ EM(fscache_cookie_get_reacquire, "GET reacq") \
+ EM(fscache_cookie_get_register_netfs, "GET rgstr") \
+ EM(fscache_cookie_get_work, "GET work ") \
+ EM(fscache_cookie_new_acquire, "NEW acq ") \
+ EM(fscache_cookie_new_netfs, "NEW netfs") \
+ EM(fscache_cookie_put_dup_netfs, "PUT dupnf") \
+ EM(fscache_cookie_put_ioreq, "GET ioreq") \
+ EM(fscache_cookie_put_object, "PUT obj ") \
+ EM(fscache_cookie_put_parent, "PUT paren") \
+ EM(fscache_cookie_put_relinquish, "PUT relnq") \
+ EM(fscache_cookie_put_work, "PUT work ") \
+ E_(fscache_cookie_see_discard, "SEE discd")
/*
* Export enum symbols via userspace.
@@ -102,9 +110,10 @@ TRACE_EVENT(fscache_cookie,
__entry->flags = cookie->flags;
),
- TP_printk("%s c=%08x u=%d p=%08x Nc=%d Na=%d f=%02x",
+ TP_printk("c=%08x %s u=%d p=%08x Nc=%d Na=%d f=%02x",
+ __entry->cookie,
__print_symbolic(__entry->where, fscache_cookie_traces),
- __entry->cookie, __entry->usage,
+ __entry->usage,
__entry->parent, __entry->n_children, __entry->n_active,
__entry->flags)
);
@@ -189,96 +198,6 @@ TRACE_EVENT(fscache_relinquish,
__entry->flags, __entry->retire)
);
-TRACE_EVENT(fscache_enable,
- TP_PROTO(struct fscache_cookie *cookie),
-
- TP_ARGS(cookie),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(int, usage )
- __field(int, n_children )
- __field(int, n_active )
- __field(u8, flags )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->usage = atomic_read(&cookie->usage);
- __entry->n_children = atomic_read(&cookie->n_children);
- __entry->n_active = atomic_read(&cookie->n_active);
- __entry->flags = cookie->flags;
- ),
-
- TP_printk("c=%08x u=%d Nc=%d Na=%d f=%02x",
- __entry->cookie, __entry->usage,
- __entry->n_children, __entry->n_active, __entry->flags)
- );
-
-TRACE_EVENT(fscache_disable,
- TP_PROTO(struct fscache_cookie *cookie),
-
- TP_ARGS(cookie),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(int, usage )
- __field(int, n_children )
- __field(int, n_active )
- __field(u8, flags )
- ),
-
- TP_fast_assign(
- __entry->cookie = cookie->debug_id;
- __entry->usage = atomic_read(&cookie->usage);
- __entry->n_children = atomic_read(&cookie->n_children);
- __entry->n_active = atomic_read(&cookie->n_active);
- __entry->flags = cookie->flags;
- ),
-
- TP_printk("c=%08x u=%d Nc=%d Na=%d f=%02x",
- __entry->cookie, __entry->usage,
- __entry->n_children, __entry->n_active, __entry->flags)
- );
-
-TRACE_EVENT(fscache_osm,
- TP_PROTO(struct fscache_object *object,
- const struct fscache_state *state,
- bool wait, bool oob, s8 event_num),
-
- TP_ARGS(object, state, wait, oob, event_num),
-
- TP_STRUCT__entry(
- __field(unsigned int, cookie )
- __field(unsigned int, object )
- __array(char, state, 8 )
- __field(bool, wait )
- __field(bool, oob )
- __field(s8, event_num )
- ),
-
- TP_fast_assign(
- __entry->cookie = object->cookie->debug_id;
- __entry->object = object->debug_id;
- __entry->wait = wait;
- __entry->oob = oob;
- __entry->event_num = event_num;
- memcpy(__entry->state, state->short_name, 8);
- ),
-
- TP_printk("c=%08x o=%08d %s %s%sev=%d",
- __entry->cookie,
- __entry->object,
- __entry->state,
- __print_symbolic(__entry->wait,
- { true, "WAIT" },
- { false, "WORK" }),
- __print_symbolic(__entry->oob,
- { true, " OOB " },
- { false, " " }),
- __entry->event_num)
- );
-
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 23/61] fscache: Rewrite the I/O API based on iov_iter
2020-05-04 17:07 ` David Howells
` (22 preceding siblings ...)
(?)
@ 2020-05-04 17:10 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:10 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Rewrite the fscache I/O API by introducing a number of new routines based
on a number of principles:
(1) The cache provides *only* write-to-cache and read-from-cache calls for
transferring data to/from the cache.
(2) The bufferage for I/O to/from the cache is supplied with an iov_iter.
There is no requirement that the iov_iters involved have anything to
do with an inode's pagecache, though if it does, an ITER_MAPPING
iterator is available.
(3) I/O to/from any particular cache object is done in one of a number of
modes, set for the cache object at cookie acquisition time:
(A) Single blob. The blob must be written in its entirety in one
go.
(B) Granular. Writes to the cache should be done in granule
sized-blocks, where, for the moment, a granule will be 256KiB,
but could be variable. This allows the metadata indicating
which granules are present to be smaller at the cost of using
more disk space.
In both cases, reads from the cache may be done in smaller chunks and
small update writes may be done inside a block that exists.
(4) I/O to/from the cache must be aligned to the DIO block size of the
backing filesystem. The cache tells the caller what it should
consider the DIO block size to be. This will never be larger than
page size.
(5) Completion of the I/O results in a callback - after which the cache
no longer knows about it.
(6) The cache doesn't retain any pointers back into the netfs, either the
code, its state or its pagecache.
To do granular I/O, the netfs has to take the read or write request it got
from the VFS/VM and 'shape' it to fit the caching parameters. It does this
by filling in a form to indicate the extent of the operation it might like
to make:
struct fscache_extent {
pgoff_t start;
pgoff_t block_end;
pgoff_t limit;
unsigned int dio_block_size;
};
and then it calls:
unsigned int
fscache_shape_extent(struct fscache_cookie *cookie,
struct fscache_extent *extent,
loff_t i_size, bool for_write);
to shape it.
The netfs should set 'start' to be the first page to read, 'block_end' to
be last page in the proposed read + 1 and 'limit' to be or ULONG_MAX or the
point beyond which the read cannot be extended. 'dio_block_size' should be
set to 0.
The cache will then shape the proposed read to fit a blocking factor
appropriate for the cache object and region of the file. It may extend
start forward and may shrink or extend block_end to fit the granularity of
the cache. This will be trimmed to the end of file as specified by the
'i_size' parameter.
Upon return, 'start' and 'limit' in the fscache_extent struct will be
updated to indicate the cache's idea of the desired size and position of
the operation. 'block_end' will hold the minimum size that will satisfy
the cache. Note that the shaped extent will always include the start page.
'dio_block_size' will be set to whatever I/O size the cache must
communicate with its storage in. This is necessary to set up the iov_iter
to be passed to the cache for reading and writing so that it can do direct
I/O.
The return value of fscache_shape_extent() is a bitmask, with
FSCACHE_READ_FROM_CACHE indicating that the cache shaped extent should be
read from the cache and FSCACHE_WRITE_TO_CACHE indicating that the data
should be written to the cache after it has been fetched.
Once the netfs has set up its iterator, if FSCACHE_READ_FROM_CACHE was
returned, it should then call:
void fscache_read(struct fscache_io_request *req)
to read data from the cache. To do this, it needs to fill out a request
descriptor:
struct fscache_io_request {
const struct fscache_io_request_ops *ops;
struct fscache_cookie *cookie;
loff_t pos;
loff_t len;
int error;
bool (*is_still_valid)(struct fscache_io_request *);
void (*done)(struct fscache_io_request *);
...
};
The ops pointer, cookie, position and length should be set to describe the
I/O operation to be performed. An 'is_still_valid' function may be
provided to check whether the operation should still go ahead after a wait
in case it got invalidated by the server.
A 'done' function may be provided that will be called to finalise the
operation. If provided, the 'done' function will be always be called, even
when the operation doesn't take place because there's no cache. If no done
function is called, the operation will be synchronous.
Note that the pages must be pinned - typically by locking them.
If FSCACHE_WRITE_TO_CACHE was set, then once the data is read from the
server, the netfs should write it to the cache by calling:
void fscache_write(struct fscache_io_request *req)
The request descriptor is set as for fscache_read(). Note that the pages
must be pinned. In this case, PG_fscache can be set on the page and the
pages can be unlocked; the bit can then be cleared by the done handler.
The releasepage, invalidatepage, launderpage and page_mkwrite functions
should be used to suspend progress until the bit is cleared. The following
functions are made available in an earlier patch for this:
void unlock_page_fscache(struct page *page);
void wait_on_page_fscache(struct page *page)
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/Makefile | 1
fs/fscache/io.c | 173 ++++++++++++++++++++++++++++++++++++
include/linux/fscache-cache.h | 29 ++++++
include/linux/fscache.h | 179 +++++++++++++++++++++++++++++++++++++
include/trace/events/cachefiles.h | 2
5 files changed, 384 insertions(+)
create mode 100644 fs/fscache/io.c
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 396e1b5fdc28..3caf66810e7b 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -8,6 +8,7 @@ fscache-y := \
cookie.o \
dispatcher.o \
fsdef.o \
+ io.o \
main.o \
netfs.o \
obj.o \
diff --git a/fs/fscache/io.c b/fs/fscache/io.c
new file mode 100644
index 000000000000..0cea98bbb8ad
--- /dev/null
+++ b/fs/fscache/io.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Data I/O routines
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/module.h>
+#include <linux/fscache-cache.h>
+#include <linux/slab.h>
+#include "internal.h"
+
+/*
+ * Initialise an I/O request
+ */
+void __fscache_init_io_request(struct fscache_io_request *req,
+ struct fscache_cookie *cookie)
+{
+ req->cookie = fscache_cookie_get(cookie, fscache_cookie_get_ioreq);
+}
+EXPORT_SYMBOL(__fscache_init_io_request);
+
+/*
+ * Clean up an I/O request
+ */
+void __fscache_free_io_request(struct fscache_io_request *req)
+{
+ if (req->object)
+ req->object->cache->ops->put_object(req->object,
+ fscache_obj_put_ioreq);
+ fscache_cookie_put(req->cookie, fscache_cookie_put_ioreq);
+}
+EXPORT_SYMBOL(__fscache_free_io_request);
+
+enum fscache_want_stage {
+ FSCACHE_WANT_PARAMS,
+ FSCACHE_WANT_WRITE,
+ FSCACHE_WANT_READ,
+};
+
+/*
+ * Begin an I/O operation on the cache, waiting till we reach the right state.
+ *
+ * Returns a pointer to the object to use or an error. If an object is
+ * returned, it will have an extra ref on it.
+ */
+static struct fscache_object *fscache_begin_io_operation(
+ struct fscache_cookie *cookie,
+ enum fscache_want_stage want,
+ struct fscache_io_request *req)
+{
+ struct fscache_object *object;
+ enum fscache_cookie_stage stage;
+
+again:
+ spin_lock(&cookie->lock);
+
+ stage = cookie->stage;
+ _enter("c=%08x{%u},%x", cookie->debug_id, stage, want);
+
+ switch (stage) {
+ case FSCACHE_COOKIE_STAGE_QUIESCENT:
+ case FSCACHE_COOKIE_STAGE_DEAD:
+ goto not_live;
+ case FSCACHE_COOKIE_STAGE_INITIALISING:
+ case FSCACHE_COOKIE_STAGE_LOOKING_UP:
+ case FSCACHE_COOKIE_STAGE_INVALIDATING:
+ goto wait_and_validate;
+
+ case FSCACHE_COOKIE_STAGE_NO_DATA_YET:
+ if (want == FSCACHE_WANT_READ)
+ goto no_data_yet;
+ /* Fall through */
+ case FSCACHE_COOKIE_STAGE_ACTIVE:
+ goto ready;
+ }
+
+ready:
+ object = hlist_entry(cookie->backing_objects.first,
+ struct fscache_object, cookie_link);
+
+ if (fscache_cache_is_broken(object))
+ goto not_live;
+
+ object->cache->ops->grab_object(object, fscache_obj_get_ioreq);
+
+ atomic_inc(&cookie->n_ops);
+ spin_unlock(&cookie->lock);
+ return object;
+
+wait_and_validate:
+ spin_unlock(&cookie->lock);
+ wait_var_event(&cookie->stage, cookie->stage != stage);
+ if (req &&
+ req->ops->is_still_valid &&
+ !req->ops->is_still_valid(req)) {
+ _leave(" = -ESTALE");
+ return ERR_PTR(-ESTALE);
+ }
+ goto again;
+
+no_data_yet:
+ spin_unlock(&cookie->lock);
+ _leave(" = -ENODATA");
+ return ERR_PTR(-ENODATA);
+
+not_live:
+ spin_unlock(&cookie->lock);
+ _leave(" = -ENOBUFS");
+ return ERR_PTR(-ENOBUFS);
+}
+
+/*
+ * Determine the size of an allocation granule or a region of data in the
+ * cache.
+ */
+unsigned int __fscache_shape_extent(struct fscache_cookie *cookie,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write)
+{
+ struct fscache_object *object =
+ fscache_begin_io_operation(cookie, FSCACHE_WANT_PARAMS, NULL);
+ unsigned int ret = 0;
+
+ if (!IS_ERR(object)) {
+ ret = object->cache->ops->shape_extent(object, extent, i_size, for_write);
+ object->cache->ops->put_object(object, fscache_obj_put_ioreq);
+ fscache_end_io_operation(cookie);
+ }
+ return ret;
+}
+EXPORT_SYMBOL(__fscache_shape_extent);
+
+/*
+ * Read data from the cache.
+ */
+int __fscache_read(struct fscache_io_request *req, struct iov_iter *iter)
+{
+ struct fscache_object *object =
+ fscache_begin_io_operation(req->cookie, FSCACHE_WANT_READ, req);
+
+ if (!IS_ERR(object)) {
+ req->object = object;
+ return object->cache->ops->read(object, req, iter);
+ } else {
+ req->error = PTR_ERR(object);
+ if (req->io_done)
+ req->io_done(req);
+ return req->error;
+ }
+}
+EXPORT_SYMBOL(__fscache_read);
+
+/*
+ * Write data to the cache.
+ */
+int __fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
+{
+ struct fscache_object *object =
+ fscache_begin_io_operation(req->cookie, FSCACHE_WANT_WRITE, req);
+
+ if (!IS_ERR(object)) {
+ req->object = object;
+ return object->cache->ops->write(object, req, iter);
+ } else {
+ req->error = PTR_ERR(object);
+ if (req->io_done)
+ req->io_done(req);
+ return req->error;
+ }
+}
+EXPORT_SYMBOL(__fscache_write);
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index fbf1b60149aa..1d235072239d 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -22,11 +22,13 @@
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
+struct fscache_io_operations;
enum fscache_obj_ref_trace {
fscache_obj_get_attach,
fscache_obj_get_exists,
fscache_obj_get_inval,
+ fscache_obj_get_ioreq,
fscache_obj_get_wait,
fscache_obj_get_withdraw,
fscache_obj_new,
@@ -37,6 +39,7 @@ enum fscache_obj_ref_trace {
fscache_obj_put_drop_child,
fscache_obj_put_drop_obj,
fscache_obj_put_inval,
+ fscache_obj_put_ioreq,
fscache_obj_put_lookup_fail,
fscache_obj_put_withdraw,
fscache_obj_ref__nr_traces
@@ -134,6 +137,21 @@ struct fscache_cache_ops {
/* reserve space for an object's data and associated metadata */
int (*reserve_space)(struct fscache_object *object, loff_t i_size);
+
+ /* Shape the extent of a read or write */
+ unsigned int (*shape_extent)(struct fscache_object *object,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write);
+
+ /* Read data from the cache */
+ int (*read)(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
+
+ /* Write data to the cache */
+ int (*write)(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
};
extern struct fscache_cookie fscache_fsdef_index;
@@ -239,4 +257,15 @@ static inline void fscache_end_io_operation(struct fscache_cookie *cookie)
wake_up_var(&cookie->n_ops);
}
+static inline void fscache_get_io_request(struct fscache_io_request *req)
+{
+ req->ops->get(req);
+}
+
+static inline void fscache_put_io_request(struct fscache_io_request *req)
+{
+ if (req)
+ req->ops->put(req);
+}
+
#endif /* _LINUX_FSCACHE_CACHE_H */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index e4fe28cfdf5c..98a6bd668f48 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -42,9 +42,11 @@
/* pattern used to fill dead space in an index entry */
#define FSCACHE_INDEX_DEADFILL_PATTERN 0x79
+struct iov_iter;
struct fscache_cache_tag;
struct fscache_cookie;
struct fscache_netfs;
+struct fscache_io_request_ops;
enum fscache_cookie_type {
FSCACHE_COOKIE_TYPE_INDEX,
@@ -122,6 +124,44 @@ struct fscache_cookie {
};
};
+/*
+ * The extent of the allocation granule in the cache, modulated for the
+ * available data on doing a read, the page size and non-contiguities.
+ *
+ * This also includes the block size to which I/O requests must be aligned.
+ */
+struct fscache_extent {
+ pgoff_t start; /* First page in the extent */
+ pgoff_t block_end; /* End of first block */
+ pgoff_t limit; /* Limit of extent (or ULONG_MAX) */
+ unsigned int dio_block_size; /* Block size required for direct I/O */
+};
+
+/*
+ * Descriptor for an fscache I/O request.
+ */
+struct fscache_io_request {
+ const struct fscache_io_request_ops *ops;
+ struct fscache_cookie *cookie;
+ struct fscache_object *object;
+ loff_t pos; /* Where to start the I/O */
+ loff_t len; /* Size of the I/O */
+ loff_t transferred; /* Amount of data transferred */
+ short error; /* 0 or error that occurred */
+ unsigned long flags;
+#define FSCACHE_IO_DATA_FROM_SERVER 0 /* Set if data was read from server */
+#define FSCACHE_IO_DATA_FROM_CACHE 1 /* Set if data was read from the cache */
+ void (*io_done)(struct fscache_io_request *);
+};
+
+struct fscache_io_request_ops {
+ bool (*is_still_valid)(struct fscache_io_request *);
+ void (*issue_op)(struct fscache_io_request *);
+ void (*done)(struct fscache_io_request *);
+ void (*get)(struct fscache_io_request *);
+ void (*put)(struct fscache_io_request *);
+};
+
/*
* slow-path functions for when there is actually caching available, and the
* netfs does actually have a valid token
@@ -149,6 +189,14 @@ extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_invalidate(struct fscache_cookie *);
extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
+extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
+ struct fscache_extent *,
+ loff_t, bool);
+extern void __fscache_init_io_request(struct fscache_io_request *,
+ struct fscache_cookie *);
+extern void __fscache_free_io_request(struct fscache_io_request *);
+extern int __fscache_read(struct fscache_io_request *, struct iov_iter *);
+extern int __fscache_write(struct fscache_io_request *, struct iov_iter *);
/**
* fscache_register_netfs - Register a filesystem as desiring caching services
@@ -407,4 +455,135 @@ void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
__fscache_wait_on_invalidate(cookie);
}
+/**
+ * fscache_init_io_request - Initialise an I/O request
+ * @req: The I/O request to initialise
+ * @cookie: The I/O cookie to access
+ * @ops: The operations table to set
+ */
+static inline void fscache_init_io_request(struct fscache_io_request *req,
+ struct fscache_cookie *cookie,
+ const struct fscache_io_request_ops *ops)
+{
+ req->ops = ops;
+ if (fscache_cookie_valid(cookie))
+ __fscache_init_io_request(req, cookie);
+}
+
+/**
+ * fscache_free_io_request - Clean up an I/O request
+ * @req: The I/O request to clean
+ */
+static inline
+void fscache_free_io_request(struct fscache_io_request *req)
+{
+ if (req->cookie)
+ __fscache_free_io_request(req);
+}
+
+#define FSCACHE_READ_FROM_CACHE 0x01
+#define FSCACHE_WRITE_TO_CACHE 0x02
+#define FSCACHE_FILL_WITH_ZERO 0x04
+
+/**
+ * fscache_shape_extent - Shape an extent to fit cache granulation
+ * @cookie: The cache cookie to access
+ * @extent: The extent proposed by the VM/filesystem and the reply.
+ * @i_size: The size to consider the file to be.
+ * @for_write: If the determination is for a write.
+ *
+ * Determine the size and position of the extent that will cover the first page
+ * in the cache such that either that extent will entirely be read from the
+ * server or entirely read from the cache. The provided extent may be
+ * adjusted, by a combination of extending the front of the extent forward
+ * and/or extending or shrinking the end of the extent. In any case, the
+ * starting page of the proposed extent will be contained in the revised
+ * extent.
+ *
+ * The function returns FSCACHE_READ_FROM_CACHE to indicate that the data is
+ * resident in the cache and can be read from there, FSCACHE_WRITE_TO_CACHE to
+ * indicate that the data isn't present, but the netfs should write it,
+ * FSCACHE_FILL_WITH_ZERO to indicate that the data should be all zeros on the
+ * server and can just be fabricated locally in or 0 to indicate that there's
+ * no cache or an error occurred and the netfs should just read from the
+ * server.
+ */
+static inline
+unsigned int fscache_shape_extent(struct fscache_cookie *cookie,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write)
+{
+ if (fscache_cookie_valid(cookie))
+ return __fscache_shape_extent(cookie, extent, i_size,
+ for_write);
+ return 0;
+}
+
+/**
+ * fscache_read - Read data from the cache.
+ * @req: The I/O request descriptor
+ * @iter: The buffer to read into
+ *
+ * The cache will attempt to read from the object referred to by the cookie,
+ * using the size and position described in the request. The data will be
+ * transferred to the buffer described by the iterator specified in the request.
+ *
+ * If this fails or can't be done, an error will be set in the request
+ * descriptor and the netfs must reissue the read to the server.
+ *
+ * Note that the length and position of the request should be aligned to the DIO
+ * block size returned by fscache_shape_extent().
+ *
+ * If req->done is set, the request will be submitted as asynchronous I/O and
+ * -EIOCBQUEUED may be returned to indicate that the operation is in progress.
+ * The done function will be called when the operation is concluded either way.
+ *
+ * If req->done is not set, the request will be submitted as synchronous I/O and
+ * will be completed before the function returns.
+ */
+static inline
+int fscache_read(struct fscache_io_request *req, struct iov_iter *iter)
+{
+ if (fscache_cookie_valid(req->cookie))
+ return __fscache_read(req, iter);
+ req->error = -ENODATA;
+ if (req->io_done)
+ req->io_done(req);
+ return -ENODATA;
+}
+
+
+/**
+ * fscache_write - Write data to the cache.
+ * @req: The I/O request description
+ * @iter: The data to write
+ *
+ * The cache will attempt to write to the object referred to by the cookie,
+ * using the size and position described in the request. The data will be
+ * transferred from the iterator specified in the request.
+ *
+ * If this fails or can't be done, an error will be set in the request
+ * descriptor.
+ *
+ * Note that the length and position of the request should be aligned to the DIO
+ * block size returned by fscache_shape_extent().
+ *
+ * If req->io_done is set, the request will be submitted as asynchronous I/O and
+ * -EIOCBQUEUED may be returned to indicate that the operation is in progress.
+ * The done function will be called when the operation is concluded either way.
+ *
+ * If req->io_done is not set, the request will be submitted as synchronous I/O and
+ * will be completed before the function returns.
+ */
+static inline
+int fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
+{
+ if (fscache_cookie_valid(req->cookie))
+ return __fscache_write(req, iter);
+ req->error = -ENOBUFS;
+ if (req->io_done)
+ req->io_done(req);
+ return -ENOBUFS;
+}
+
#endif /* _LINUX_FSCACHE_H */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 4fedc2e9c428..0aa3f3126f6e 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -39,6 +39,7 @@ enum cachefiles_obj_ref_trace {
EM(fscache_obj_get_attach, "GET attach") \
EM(fscache_obj_get_exists, "GET exists") \
EM(fscache_obj_get_inval, "GET inval") \
+ EM(fscache_obj_get_ioreq, "GET ioreq") \
EM(fscache_obj_get_wait, "GET wait") \
EM(fscache_obj_get_withdraw, "GET withdraw") \
EM(fscache_obj_new, "NEW obj") \
@@ -49,6 +50,7 @@ enum cachefiles_obj_ref_trace {
EM(fscache_obj_put_drop_child, "PUT drop_child") \
EM(fscache_obj_put_drop_obj, "PUT drop_obj") \
EM(fscache_obj_put_inval, "PUT inval") \
+ EM(fscache_obj_put_ioreq, "PUT ioreq") \
EM(fscache_obj_put_withdraw, "PUT withdraw") \
EM(fscache_obj_put_lookup_fail, "PUT lookup_fail") \
EM(cachefiles_obj_put_wait_retry, "PUT wait_retry") \
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 24/61] fscache: Remove fscache_wait_on_invalidate()
@ 2020-05-04 17:11 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove fscache_wait_on_invalidate() as the invalidation wait is now built into
the I/O path.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/cookie.c | 14 --------------
include/linux/fscache.h | 17 -----------------
2 files changed, 31 deletions(-)
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 3a1b71e09c6b..38b8fcf33114 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -492,20 +492,6 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
}
EXPORT_SYMBOL(__fscache_invalidate);
-/*
- * Wait for object invalidation to complete.
- */
-void __fscache_wait_on_invalidate(struct fscache_cookie *cookie)
-{
- _enter("%p", cookie);
-
- wait_on_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING,
- TASK_UNINTERRUPTIBLE);
-
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_wait_on_invalidate);
-
/*
* Update the index entries backing a cookie. The writeback is done lazily.
*/
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 98a6bd668f48..607249ff1bc5 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -188,7 +188,6 @@ extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_invalidate(struct fscache_cookie *);
-extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
loff_t, bool);
@@ -439,22 +438,6 @@ void fscache_invalidate(struct fscache_cookie *cookie)
__fscache_invalidate(cookie);
}
-/**
- * fscache_wait_on_invalidate - Wait for invalidation to complete
- * @cookie: The cookie representing the cache object
- *
- * Wait for the invalidation of an object to complete.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_wait_on_invalidate(cookie);
-}
-
/**
* fscache_init_io_request - Initialise an I/O request
* @req: The I/O request to initialise
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 24/61] fscache: Remove fscache_wait_on_invalidate()
@ 2020-05-04 17:11 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove fscache_wait_on_invalidate() as the invalidation wait is now built into
the I/O path.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/fscache/cookie.c | 14 --------------
include/linux/fscache.h | 17 -----------------
2 files changed, 31 deletions(-)
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 3a1b71e09c6b..38b8fcf33114 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -492,20 +492,6 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
}
EXPORT_SYMBOL(__fscache_invalidate);
-/*
- * Wait for object invalidation to complete.
- */
-void __fscache_wait_on_invalidate(struct fscache_cookie *cookie)
-{
- _enter("%p", cookie);
-
- wait_on_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING,
- TASK_UNINTERRUPTIBLE);
-
- _leave("");
-}
-EXPORT_SYMBOL(__fscache_wait_on_invalidate);
-
/*
* Update the index entries backing a cookie. The writeback is done lazily.
*/
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 98a6bd668f48..607249ff1bc5 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -188,7 +188,6 @@ extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_invalidate(struct fscache_cookie *);
-extern void __fscache_wait_on_invalidate(struct fscache_cookie *);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
loff_t, bool);
@@ -439,22 +438,6 @@ void fscache_invalidate(struct fscache_cookie *cookie)
__fscache_invalidate(cookie);
}
-/**
- * fscache_wait_on_invalidate - Wait for invalidation to complete
- * @cookie: The cookie representing the cache object
- *
- * Wait for the invalidation of an object to complete.
- *
- * See Documentation/filesystems/caching/netfs-api.txt for a complete
- * description.
- */
-static inline
-void fscache_wait_on_invalidate(struct fscache_cookie *cookie)
-{
- if (fscache_cookie_valid(cookie))
- __fscache_wait_on_invalidate(cookie);
-}
-
/**
* fscache_init_io_request - Initialise an I/O request
* @req: The I/O request to initialise
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 25/61] fscache: Keep track of size of a file last set independently on the server
2020-05-04 17:07 ` David Howells
` (24 preceding siblings ...)
(?)
@ 2020-05-04 17:11 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Keep track of the size of a file that we're caching as last set
independently on the server by another client. As long as this does not
change, we can make the assumption that anything over that boundary, if not
represented in the local cache, will not be represented on the server
either and can be just cleared rather than being read, thereby saving a
trip to the server.
This only works if we make space in the cache by zapping whole files and
not just punching bits out of them as if we write to the server but don't
keep a copy in the cache, the assumption mentioned above no longer holds
true.
We also need to update this size when invalidation occurs.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/inode.c | 2 +-
fs/fscache/cookie.c | 8 +++++++-
include/linux/fscache.h | 8 +++++---
3 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 9485a5afa13d..1f248181d47d 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -577,7 +577,7 @@ static void afs_zap_data(struct afs_vnode *vnode)
_enter("{%llx:%llu}", vnode->fid.vid, vnode->fid.vnode);
#ifdef CONFIG_AFS_FSCACHE
- fscache_invalidate(vnode->cache);
+ fscache_invalidate(vnode->cache, i_size_read(&vnode->vfs_inode));
#endif
/* nuke all the non-dirty pages that aren't locked, mapped or being
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 38b8fcf33114..284017fe4a5d 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -159,6 +159,7 @@ struct fscache_cookie *fscache_alloc_cookie(
cookie->key_len = index_key_len;
cookie->aux_len = aux_data_len;
cookie->object_size = object_size;
+ cookie->zero_point = object_size;
strlcpy(cookie->type_name, type_name, sizeof(cookie->type_name));
if (fscache_set_key(cookie, index_key, index_key_len) < 0)
@@ -473,7 +474,7 @@ void fscache_set_cookie_stage(struct fscache_cookie *cookie,
/*
* Invalidate an object. Callable with spinlocks held.
*/
-void __fscache_invalidate(struct fscache_cookie *cookie)
+void __fscache_invalidate(struct fscache_cookie *cookie, loff_t new_size)
{
_enter("{%s}", cookie->type_name);
@@ -486,6 +487,11 @@ void __fscache_invalidate(struct fscache_cookie *cookie)
*/
ASSERTCMP(cookie->type, ==, FSCACHE_COOKIE_TYPE_DATAFILE);
+ spin_lock(&cookie->lock);
+ cookie->object_size = new_size;
+ cookie->zero_point = new_size;
+ spin_unlock(&cookie->lock);
+
if (!hlist_empty(&cookie->backing_objects) &&
test_and_set_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
fscache_dispatch(cookie, NULL, 0, fscache_invalidate_object);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 607249ff1bc5..b46df865911c 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -102,6 +102,7 @@ struct fscache_cookie {
struct list_head proc_link; /* Link in proc list */
char type_name[8]; /* Cookie type name */
loff_t object_size; /* Size of the netfs object */
+ loff_t zero_point; /* Size after which no data on server */
unsigned long flags;
#define FSCACHE_COOKIE_INVALIDATING 4 /* T if cookie is being invalidated */
@@ -187,7 +188,7 @@ extern void __fscache_use_cookie(struct fscache_cookie *, bool);
extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
-extern void __fscache_invalidate(struct fscache_cookie *);
+extern void __fscache_invalidate(struct fscache_cookie *, loff_t);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
loff_t, bool);
@@ -421,6 +422,7 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
/**
* fscache_invalidate - Notify cache that an object needs invalidation
* @cookie: The cookie representing the cache object
+ * @size: The revised size of the object.
*
* Notify the cache that an object is needs to be invalidated and that it
* should abort any retrievals or stores it is doing on the cache. The object
@@ -432,10 +434,10 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
* description.
*/
static inline
-void fscache_invalidate(struct fscache_cookie *cookie)
+void fscache_invalidate(struct fscache_cookie *cookie, loff_t size)
{
if (fscache_cookie_valid(cookie))
- __fscache_invalidate(cookie);
+ __fscache_invalidate(cookie, size);
}
/**
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 26/61] fscache, cachefiles: Fix disabled histogram warnings
2020-05-04 17:07 ` David Howells
` (25 preceding siblings ...)
(?)
@ 2020-05-04 17:11 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Fix variable unused warnings due to disabled histogram stuff.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/internal.h | 7 +++++--
fs/fscache/internal.h | 6 ++++--
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index b89f76a03546..16d15291a629 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -143,11 +143,11 @@ extern int cachefiles_check_in_use(struct cachefiles_cache *cache,
/*
* proc.c
*/
-#ifdef CONFIG_CACHEFILES_HISTOGRAM
extern atomic_t cachefiles_lookup_histogram[HZ];
extern atomic_t cachefiles_mkdir_histogram[HZ];
extern atomic_t cachefiles_create_histogram[HZ];
+#ifdef CONFIG_CACHEFILES_HISTOGRAM
extern int __init cachefiles_proc_init(void);
extern void cachefiles_proc_cleanup(void);
static inline
@@ -162,7 +162,10 @@ void cachefiles_hist(atomic_t histogram[], unsigned long start_jif)
#else
#define cachefiles_proc_init() (0)
#define cachefiles_proc_cleanup() do {} while (0)
-#define cachefiles_hist(hist, start_jif) do {} while (0)
+static inline
+void cachefiles_hist(atomic_t histogram[], unsigned long start_jif)
+{
+}
#endif
/*
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 443671310e31..a70c1a612309 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -95,13 +95,13 @@ extern struct fscache_cookie fscache_fsdef_index;
/*
* histogram.c
*/
-#ifdef CONFIG_FSCACHE_HISTOGRAM
extern atomic_t fscache_obj_instantiate_histogram[HZ];
extern atomic_t fscache_objs_histogram[HZ];
extern atomic_t fscache_ops_histogram[HZ];
extern atomic_t fscache_retrieval_delay_histogram[HZ];
extern atomic_t fscache_retrieval_histogram[HZ];
+#ifdef CONFIG_FSCACHE_HISTOGRAM
static inline void fscache_hist(atomic_t histogram[], unsigned long start_jif)
{
unsigned long jif = jiffies - start_jif;
@@ -113,7 +113,9 @@ static inline void fscache_hist(atomic_t histogram[], unsigned long start_jif)
extern const struct seq_operations fscache_histogram_ops;
#else
-#define fscache_hist(hist, start_jif) do {} while (0)
+static inline void fscache_hist(atomic_t histogram[], unsigned long start_jif)
+{
+}
#endif
/*
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 27/61] fscache: Recast assertion in terms of cookie not being an index
2020-05-04 17:07 ` David Howells
` (26 preceding siblings ...)
(?)
@ 2020-05-04 17:11 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Recast assertion in terms of cookie not being an index rather than being a
datafile.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/cookie.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index 284017fe4a5d..f02ed34beb0c 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -485,7 +485,7 @@ void __fscache_invalidate(struct fscache_cookie *cookie, loff_t new_size)
* there, and if it's doing that, it may as well just retire the
* cookie.
*/
- ASSERTCMP(cookie->type, ==, FSCACHE_COOKIE_TYPE_DATAFILE);
+ ASSERTCMP(cookie->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
spin_lock(&cookie->lock);
cookie->object_size = new_size;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 28/61] cachefiles: Remove some redundant checks on unsigned values
2020-05-04 17:07 ` David Howells
` (27 preceding siblings ...)
(?)
@ 2020-05-04 17:11 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove some redundant checks for unsigned values being >= 0.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/bind.c | 6 ++----
fs/cachefiles/daemon.c | 6 +++---
2 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/fs/cachefiles/bind.c b/fs/cachefiles/bind.c
index 4c59e1ef4500..84fe89d5999e 100644
--- a/fs/cachefiles/bind.c
+++ b/fs/cachefiles/bind.c
@@ -36,13 +36,11 @@ int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
args);
/* start by checking things over */
- ASSERT(cache->fstop_percent >= 0 &&
- cache->fstop_percent < cache->fcull_percent &&
+ ASSERT(cache->fstop_percent < cache->fcull_percent &&
cache->fcull_percent < cache->frun_percent &&
cache->frun_percent < 100);
- ASSERT(cache->bstop_percent >= 0 &&
- cache->bstop_percent < cache->bcull_percent &&
+ ASSERT(cache->bstop_percent < cache->bcull_percent &&
cache->bcull_percent < cache->brun_percent &&
cache->brun_percent < 100);
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 8a937d6d5e22..e8ab3ab57147 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -221,7 +221,7 @@ static ssize_t cachefiles_daemon_write(struct file *file,
if (test_bit(CACHEFILES_DEAD, &cache->flags))
return -EIO;
- if (datalen < 0 || datalen > PAGE_SIZE - 1)
+ if (datalen > PAGE_SIZE - 1)
return -EOPNOTSUPP;
/* drag the command string into the kernel so we can parse it */
@@ -378,7 +378,7 @@ static int cachefiles_daemon_fstop(struct cachefiles_cache *cache, char *args)
if (args[0] != '%' || args[1] != '\0')
return -EINVAL;
- if (fstop < 0 || fstop >= cache->fcull_percent)
+ if (fstop >= cache->fcull_percent)
return cachefiles_daemon_range_error(cache, args);
cache->fstop_percent = fstop;
@@ -450,7 +450,7 @@ static int cachefiles_daemon_bstop(struct cachefiles_cache *cache, char *args)
if (args[0] != '%' || args[1] != '\0')
return -EINVAL;
- if (bstop < 0 || bstop >= cache->bcull_percent)
+ if (bstop >= cache->bcull_percent)
return cachefiles_daemon_range_error(cache, args);
cache->bstop_percent = bstop;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 29/61] cachefiles: trace: Log coherency checks
2020-05-04 17:07 ` David Howells
` (28 preceding siblings ...)
(?)
@ 2020-05-04 17:11 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:11 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Add a cachefiles tracepoint that logs the result of coherency management
when the coherency data on a file in the cache is checked or committed.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/xattr.c | 45 ++++++++++++++++++++++--------
include/trace/events/cachefiles.h | 56 +++++++++++++++++++++++++++++++++++++
2 files changed, 89 insertions(+), 12 deletions(-)
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 5b2f6da91cc8..17c16c2bd07e 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -125,12 +125,21 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
buf, sizeof(struct cachefiles_xattr) + len,
xattr_flags);
- kfree(buf);
- if (ret < 0 && ret != -ENOMEM)
- cachefiles_io_error_obj(
- object,
- "Failed to set xattr with error %d", ret);
+ if (ret < 0) {
+ trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
+ 0,
+ cachefiles_coherency_set_fail);
+ if (ret != -ENOMEM)
+ cachefiles_io_error_obj(
+ object,
+ "Failed to set xattr with error %d", ret);
+ } else {
+ trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
+ 0,
+ cachefiles_coherency_set_ok);
+ }
+ kfree(buf);
_leave(" = %d", ret);
return ret;
}
@@ -144,7 +153,9 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
struct dentry *dentry = object->dentry;
unsigned int len = object->fscache.cookie->aux_len, tlen;
const void *p = fscache_get_aux(object->fscache.cookie);
- ssize_t ret;
+ enum cachefiles_coherency_trace why;
+ ssize_t xlen;
+ int ret = -ESTALE;
ASSERT(dentry);
ASSERT(d_backing_inode(dentry));
@@ -154,14 +165,24 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
if (!buf)
return -ENOMEM;
- ret = vfs_getxattr(dentry, cachefiles_xattr_cache, buf, tlen);
- if (ret == tlen &&
- buf->type == object->fscache.cookie->type &&
- memcmp(buf->data, p, len) == 0)
+ xlen = vfs_getxattr(dentry, cachefiles_xattr_cache, buf, tlen);
+ if (xlen != tlen) {
+ if (xlen == -EIO)
+ cachefiles_io_error_obj(
+ object,
+ "Failed to read aux with error %zd", xlen);
+ why = cachefiles_coherency_check_xattr;
+ } else if (buf->type != object->fscache.cookie->type) {
+ why = cachefiles_coherency_check_type;
+ } else if (memcmp(buf->data, p, len) != 0) {
+ why = cachefiles_coherency_check_aux;
+ } else {
+ why = cachefiles_coherency_check_ok;
ret = 0;
- else
- ret = -ESTALE;
+ }
+ trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
+ 0, why);
kfree(buf);
return ret;
}
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 0aa3f3126f6e..bf588c3f4a07 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -24,6 +24,19 @@ enum cachefiles_obj_ref_trace {
cachefiles_obj_ref__nr_traces
};
+enum cachefiles_coherency_trace {
+ cachefiles_coherency_check_aux,
+ cachefiles_coherency_check_content,
+ cachefiles_coherency_check_dirty,
+ cachefiles_coherency_check_len,
+ cachefiles_coherency_check_objsize,
+ cachefiles_coherency_check_ok,
+ cachefiles_coherency_check_type,
+ cachefiles_coherency_check_xattr,
+ cachefiles_coherency_set_fail,
+ cachefiles_coherency_set_ok,
+};
+
#endif
/*
@@ -56,6 +69,18 @@ enum cachefiles_obj_ref_trace {
EM(cachefiles_obj_put_wait_retry, "PUT wait_retry") \
E_(cachefiles_obj_put_wait_timeo, "PUT wait_timeo")
+#define cachefiles_coherency_traces \
+ EM(cachefiles_coherency_check_aux, "BAD aux ") \
+ EM(cachefiles_coherency_check_content, "BAD cont") \
+ EM(cachefiles_coherency_check_dirty, "BAD dirt") \
+ EM(cachefiles_coherency_check_len, "BAD len ") \
+ EM(cachefiles_coherency_check_objsize, "BAD osiz") \
+ EM(cachefiles_coherency_check_ok, "OK ") \
+ EM(cachefiles_coherency_check_type, "BAD type") \
+ EM(cachefiles_coherency_check_xattr, "BAD xatt") \
+ EM(cachefiles_coherency_set_fail, "SET fail") \
+ E_(cachefiles_coherency_set_ok, "SET ok ")
+
/*
* Export enum symbols via userspace.
*/
@@ -66,6 +91,7 @@ enum cachefiles_obj_ref_trace {
cachefiles_obj_kill_traces;
cachefiles_obj_ref_traces;
+cachefiles_coherency_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -295,6 +321,36 @@ TRACE_EVENT(cachefiles_mark_buried,
__print_symbolic(__entry->why, cachefiles_obj_kill_traces))
);
+TRACE_EVENT(cachefiles_coherency,
+ TP_PROTO(struct cachefiles_object *obj,
+ ino_t ino,
+ int content,
+ enum cachefiles_coherency_trace why),
+
+ TP_ARGS(obj, ino, content, why),
+
+ /* Note that obj may be NULL */
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(enum cachefiles_coherency_trace, why )
+ __field(int, content )
+ __field(u64, ino )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->why = why;
+ __entry->content = content;
+ __entry->ino = ino;
+ ),
+
+ TP_printk("o=%08x %s i=%llx c=%u",
+ __entry->obj,
+ __print_symbolic(__entry->why, cachefiles_coherency_traces),
+ __entry->ino,
+ __entry->content)
+ );
+
#endif /* _TRACE_CACHEFILES_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 30/61] cachefiles: Split cachefiles_drop_object() up a bit
@ 2020-05-04 17:12 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Split cachefiles_drop_object() up a bit to make it easier to modify later.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 58 ++++++++++++++++++++++++++++++---------------
1 file changed, 39 insertions(+), 19 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index daa4f316d104..47596b58c2da 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -192,6 +192,42 @@ static void cachefiles_update_object(struct fscache_object *_object)
_leave("");
}
+/*
+ * Commit changes to the object as we drop it.
+ */
+static void cachefiles_commit_object(struct cachefiles_object *object,
+ struct cachefiles_cache *cache)
+{
+}
+
+/*
+ * Finalise and object and close the VFS structs that we have.
+ */
+static void cachefiles_clean_up_object(struct cachefiles_object *object,
+ struct cachefiles_cache *cache,
+ bool invalidate)
+{
+ if (invalidate && &object->fscache != cache->cache.fsdef) {
+ _debug("- inval object OBJ%x", object->fscache.debug_id);
+ cachefiles_delete_object(cache, object);
+ } else {
+ cachefiles_commit_object(object, cache);
+ }
+
+ /* close the filesystem stuff attached to the object */
+ if (object->backing_file)
+ fput(object->backing_file);
+ object->backing_file = NULL;
+
+ if (object->backer != object->dentry)
+ dput(object->backer);
+ object->backer = NULL;
+
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ dput(object->dentry);
+ object->dentry = NULL;
+}
+
/*
* discard the resources pinned by an object and effect retirement if
* requested
@@ -223,25 +259,9 @@ static void cachefiles_drop_object(struct fscache_object *_object,
* before we set it up.
*/
if (object->dentry) {
- if (invalidate && _object != cache->cache.fsdef) {
- _debug("- inval object OBJ%x", object->fscache.debug_id);
- cachefiles_begin_secure(cache, &saved_cred);
- cachefiles_delete_object(cache, object);
- cachefiles_end_secure(cache, saved_cred);
- }
-
- /* close the filesystem stuff attached to the object */
- if (object->backing_file)
- fput(object->backing_file);
- object->backing_file = NULL;
-
- if (object->backer != object->dentry)
- dput(object->backer);
- object->backer = NULL;
-
- cachefiles_unmark_inode_in_use(object, object->dentry);
- dput(object->dentry);
- object->dentry = NULL;
+ cachefiles_begin_secure(cache, &saved_cred);
+ cachefiles_clean_up_object(object, cache, invalidate);
+ cachefiles_end_secure(cache, saved_cred);
}
_leave("");
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 30/61] cachefiles: Split cachefiles_drop_object() up a bit
@ 2020-05-04 17:12 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Split cachefiles_drop_object() up a bit to make it easier to modify later.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/interface.c | 58 ++++++++++++++++++++++++++++++---------------
1 file changed, 39 insertions(+), 19 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index daa4f316d104..47596b58c2da 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -192,6 +192,42 @@ static void cachefiles_update_object(struct fscache_object *_object)
_leave("");
}
+/*
+ * Commit changes to the object as we drop it.
+ */
+static void cachefiles_commit_object(struct cachefiles_object *object,
+ struct cachefiles_cache *cache)
+{
+}
+
+/*
+ * Finalise and object and close the VFS structs that we have.
+ */
+static void cachefiles_clean_up_object(struct cachefiles_object *object,
+ struct cachefiles_cache *cache,
+ bool invalidate)
+{
+ if (invalidate && &object->fscache != cache->cache.fsdef) {
+ _debug("- inval object OBJ%x", object->fscache.debug_id);
+ cachefiles_delete_object(cache, object);
+ } else {
+ cachefiles_commit_object(object, cache);
+ }
+
+ /* close the filesystem stuff attached to the object */
+ if (object->backing_file)
+ fput(object->backing_file);
+ object->backing_file = NULL;
+
+ if (object->backer != object->dentry)
+ dput(object->backer);
+ object->backer = NULL;
+
+ cachefiles_unmark_inode_in_use(object, object->dentry);
+ dput(object->dentry);
+ object->dentry = NULL;
+}
+
/*
* discard the resources pinned by an object and effect retirement if
* requested
@@ -223,25 +259,9 @@ static void cachefiles_drop_object(struct fscache_object *_object,
* before we set it up.
*/
if (object->dentry) {
- if (invalidate && _object != cache->cache.fsdef) {
- _debug("- inval object OBJ%x", object->fscache.debug_id);
- cachefiles_begin_secure(cache, &saved_cred);
- cachefiles_delete_object(cache, object);
- cachefiles_end_secure(cache, saved_cred);
- }
-
- /* close the filesystem stuff attached to the object */
- if (object->backing_file)
- fput(object->backing_file);
- object->backing_file = NULL;
-
- if (object->backer != object->dentry)
- dput(object->backer);
- object->backer = NULL;
-
- cachefiles_unmark_inode_in_use(object, object->dentry);
- dput(object->dentry);
- object->dentry = NULL;
+ cachefiles_begin_secure(cache, &saved_cred);
+ cachefiles_clean_up_object(object, cache, invalidate);
+ cachefiles_end_secure(cache, saved_cred);
}
_leave("");
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 31/61] cachefiles: Implement new fscache I/O backend API
2020-05-04 17:07 ` David Howells
` (30 preceding siblings ...)
(?)
@ 2020-05-04 17:12 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Implement the new fscache I/O backend API in cachefiles. The
cachefiles_object struct carries a non-accounted file to the cachefiles
object (so that it doesn't cause ENFILE).
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/Makefile | 1 +
fs/cachefiles/interface.c | 3 ++
fs/cachefiles/internal.h | 14 +++++++
fs/cachefiles/io.c | 88 +++++++++++++++++++++++++++++++++++++++++++++
fs/cachefiles/namei.c | 3 ++
5 files changed, 109 insertions(+)
create mode 100644 fs/cachefiles/io.c
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index 3455d3646547..d894d317d6e7 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -7,6 +7,7 @@ cachefiles-y := \
bind.o \
daemon.o \
interface.o \
+ io.o \
key.o \
main.o \
namei.o \
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 47596b58c2da..747211363ae0 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -469,4 +469,7 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.put_object = cachefiles_put_object,
.get_object_usage = cachefiles_get_object_usage,
.sync_cache = cachefiles_sync_cache,
+ .shape_extent = cachefiles_shape_extent,
+ .read = cachefiles_read,
+ .write = cachefiles_write,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 16d15291a629..9f9e33893b4f 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -115,6 +115,20 @@ extern const struct fscache_cache_ops cachefiles_cache_ops;
extern struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
enum fscache_obj_ref_trace why);
+/*
+ * io.c
+ */
+extern unsigned int cachefiles_shape_extent(struct fscache_object *object,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write);
+extern int cachefiles_read(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
+extern int cachefiles_write(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
+extern bool cachefiles_open_object(struct cachefiles_object *obj);
+
/*
* key.c
*/
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
new file mode 100644
index 000000000000..d472e9d9173c
--- /dev/null
+++ b/fs/cachefiles/io.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Data I/O routines
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/mount.h>
+#include <linux/slab.h>
+#include <linux/file.h>
+#include <linux/uio.h>
+#include <linux/xattr.h>
+#include "internal.h"
+
+/*
+ * Determine the size of a data extent in a cache object. This must be written
+ * as a whole unit, but can be read piecemeal.
+ */
+unsigned int cachefiles_shape_extent(struct fscache_object *object,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write)
+{
+ return 0;
+}
+
+/*
+ * Initiate a read from the cache.
+ */
+int cachefiles_read(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter)
+{
+ req->error = -ENODATA;
+ if (req->io_done)
+ req->io_done(req);
+ return -ENODATA;
+}
+
+/*
+ * Initiate a write to the cache.
+ */
+int cachefiles_write(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter)
+{
+ req->error = -ENOBUFS;
+ if (req->io_done)
+ req->io_done(req);
+ return -ENOBUFS;
+}
+
+/*
+ * Open a cache object.
+ */
+bool cachefiles_open_object(struct cachefiles_object *object)
+{
+ struct cachefiles_cache *cache =
+ container_of(object->fscache.cache, struct cachefiles_cache, cache);
+ struct file *file;
+ struct path path;
+
+ path.mnt = cache->mnt;
+ path.dentry = object->backer;
+
+ file = open_with_fake_path(&path,
+ O_RDWR | O_LARGEFILE | O_DIRECT,
+ d_backing_inode(object->backer),
+ cache->cache_cred);
+ if (IS_ERR(file))
+ goto error;
+
+ if (!S_ISREG(file_inode(file)->i_mode))
+ goto error_file;
+
+ if (unlikely(!file->f_op->read_iter) ||
+ unlikely(!file->f_op->write_iter)) {
+ pr_notice("Cache does not support read_iter and write_iter\n");
+ goto error_file;
+ }
+
+ object->backing_file = file;
+ return true;
+
+error_file:
+ fput(file);
+error:
+ return false;
+}
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index ad7edd41ce27..c33b5ee65ac1 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -492,6 +492,9 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
} else {
BUG(); // TODO: open file in data-class subdir
}
+
+ if (!cachefiles_open_object(object))
+ goto check_error;
}
if (object->new)
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 32/61] cachefiles: Merge object->backer into object->dentry
@ 2020-05-04 17:12 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Merge the object->backer pointer into the object->dentry pointer and assume
that data objects are always going to be just regular files.
object->dentry can then more easily be overridden later by invalidation
without having two different things to update the xattrs on.
object->old maintains a pointer to the old file so that we can unlink the
it later.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 35 +++++++++++++++++------------------
fs/cachefiles/internal.h | 2 +-
fs/cachefiles/io.c | 4 ++--
fs/cachefiles/namei.c | 4 +++-
4 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 747211363ae0..bb68318fcc1c 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -171,16 +171,16 @@ static void cachefiles_update_object(struct fscache_object *_object)
cachefiles_begin_secure(cache, &saved_cred);
object_size = object->fscache.cookie->object_size;
- if (i_size_read(d_inode(object->backer)) > object_size) {
+ if (i_size_read(d_inode(object->dentry)) > object_size) {
struct path path = {
.mnt = cache->mnt,
- .dentry = object->backer
+ .dentry = object->dentry
};
- _debug("trunc %llx -> %llx", i_size_read(d_inode(object->backer)), object_size);
+ _debug("trunc %llx -> %llx", i_size_read(d_inode(object->dentry)), object_size);
ret = vfs_truncate(&path, object_size);
if (ret < 0) {
cachefiles_io_error_obj(object, "Trunc-to-size failed");
- cachefiles_remove_object_xattr(cache, object->backer);
+ cachefiles_remove_object_xattr(cache, object->dentry);
goto out;
}
}
@@ -219,9 +219,8 @@ static void cachefiles_clean_up_object(struct cachefiles_object *object,
fput(object->backing_file);
object->backing_file = NULL;
- if (object->backer != object->dentry)
- dput(object->backer);
- object->backer = NULL;
+ dput(object->old);
+ object->old = NULL;
cachefiles_unmark_inode_in_use(object, object->dentry);
dput(object->dentry);
@@ -299,7 +298,7 @@ static void cachefiles_put_object(struct fscache_object *_object,
_debug("- kill object OBJ%x", object->fscache.debug_id);
ASSERTCMP(object->fscache.parent, ==, NULL);
- ASSERTCMP(object->backer, ==, NULL);
+ ASSERTCMP(object->old, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
ASSERTCMP(object->fscache.n_children, ==, 0);
@@ -364,17 +363,17 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
if (ni_size == object->i_size)
return 0;
- if (!object->backer)
+ if (!object->dentry)
return -ENOBUFS;
- ASSERT(d_is_reg(object->backer));
+ ASSERT(d_is_reg(object->dentry));
- oi_size = i_size_read(d_backing_inode(object->backer));
+ oi_size = i_size_read(d_backing_inode(object->dentry));
if (oi_size == ni_size)
return 0;
cachefiles_begin_secure(cache, &saved_cred);
- inode_lock(d_inode(object->backer));
+ inode_lock(d_inode(object->dentry));
/* if there's an extension to a partial page at the end of the backing
* file, we need to discard the partial page so that we pick up new
@@ -383,17 +382,17 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
_debug("discard tail %llx", oi_size);
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = oi_size & PAGE_MASK;
- ret = notify_change(object->backer, &newattrs, NULL);
+ ret = notify_change(object->dentry, &newattrs, NULL);
if (ret < 0)
goto truncate_failed;
}
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = ni_size;
- ret = notify_change(object->backer, &newattrs, NULL);
+ ret = notify_change(object->dentry, &newattrs, NULL);
truncate_failed:
- inode_unlock(d_inode(object->backer));
+ inode_unlock(d_inode(object->dentry));
cachefiles_end_secure(cache, saved_cred);
if (ret == -EIO) {
@@ -426,10 +425,10 @@ static void cachefiles_invalidate_object(struct fscache_object *_object)
_enter("{OBJ%x},[%llu]",
object->fscache.debug_id, (unsigned long long)ni_size);
- if (object->backer) {
- ASSERT(d_is_reg(object->backer));
+ if (object->dentry) {
+ ASSERT(d_is_reg(object->dentry));
- path.dentry = object->backer;
+ path.dentry = object->dentry;
path.mnt = cache->mnt;
cachefiles_begin_secure(cache, &saved_cred);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 9f9e33893b4f..e605cffc284d 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -35,7 +35,7 @@ extern unsigned cachefiles_debug;
struct cachefiles_object {
struct fscache_object fscache; /* fscache handle */
struct dentry *dentry; /* the file/dir representing this object */
- struct dentry *backer; /* backing file */
+ struct dentry *old; /* backing file */
struct file *backing_file; /* File open on backing storage */
loff_t i_size; /* object size */
atomic_t usage; /* object usage count */
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d472e9d9173c..2384067e716d 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -60,11 +60,11 @@ bool cachefiles_open_object(struct cachefiles_object *object)
struct path path;
path.mnt = cache->mnt;
- path.dentry = object->backer;
+ path.dentry = object->dentry;
file = open_with_fake_path(&path,
O_RDWR | O_LARGEFILE | O_DIRECT,
- d_backing_inode(object->backer),
+ d_backing_inode(object->dentry),
cache->cache_cred);
if (IS_ERR(file))
goto error;
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index c33b5ee65ac1..d697a74436c0 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -488,7 +488,7 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
goto check_error;
}
- object->backer = object->dentry;
+ object->old = dget(object->dentry);
} else {
BUG(); // TODO: open file in data-class subdir
}
@@ -523,7 +523,9 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_unmark_inode_in_use(object, object->dentry);
cachefiles_mark_object_inactive(cache, object);
dput(object->dentry);
+ dput(object->old);
object->dentry = NULL;
+ object->old = NULL;
goto error_out;
lookup_error:
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 32/61] cachefiles: Merge object->backer into object->dentry
@ 2020-05-04 17:12 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Merge the object->backer pointer into the object->dentry pointer and assume
that data objects are always going to be just regular files.
object->dentry can then more easily be overridden later by invalidation
without having two different things to update the xattrs on.
object->old maintains a pointer to the old file so that we can unlink the
it later.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/interface.c | 35 +++++++++++++++++------------------
fs/cachefiles/internal.h | 2 +-
fs/cachefiles/io.c | 4 ++--
fs/cachefiles/namei.c | 4 +++-
4 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 747211363ae0..bb68318fcc1c 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -171,16 +171,16 @@ static void cachefiles_update_object(struct fscache_object *_object)
cachefiles_begin_secure(cache, &saved_cred);
object_size = object->fscache.cookie->object_size;
- if (i_size_read(d_inode(object->backer)) > object_size) {
+ if (i_size_read(d_inode(object->dentry)) > object_size) {
struct path path = {
.mnt = cache->mnt,
- .dentry = object->backer
+ .dentry = object->dentry
};
- _debug("trunc %llx -> %llx", i_size_read(d_inode(object->backer)), object_size);
+ _debug("trunc %llx -> %llx", i_size_read(d_inode(object->dentry)), object_size);
ret = vfs_truncate(&path, object_size);
if (ret < 0) {
cachefiles_io_error_obj(object, "Trunc-to-size failed");
- cachefiles_remove_object_xattr(cache, object->backer);
+ cachefiles_remove_object_xattr(cache, object->dentry);
goto out;
}
}
@@ -219,9 +219,8 @@ static void cachefiles_clean_up_object(struct cachefiles_object *object,
fput(object->backing_file);
object->backing_file = NULL;
- if (object->backer != object->dentry)
- dput(object->backer);
- object->backer = NULL;
+ dput(object->old);
+ object->old = NULL;
cachefiles_unmark_inode_in_use(object, object->dentry);
dput(object->dentry);
@@ -299,7 +298,7 @@ static void cachefiles_put_object(struct fscache_object *_object,
_debug("- kill object OBJ%x", object->fscache.debug_id);
ASSERTCMP(object->fscache.parent, ==, NULL);
- ASSERTCMP(object->backer, ==, NULL);
+ ASSERTCMP(object->old, ==, NULL);
ASSERTCMP(object->dentry, ==, NULL);
ASSERTCMP(object->fscache.n_children, ==, 0);
@@ -364,17 +363,17 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
if (ni_size == object->i_size)
return 0;
- if (!object->backer)
+ if (!object->dentry)
return -ENOBUFS;
- ASSERT(d_is_reg(object->backer));
+ ASSERT(d_is_reg(object->dentry));
- oi_size = i_size_read(d_backing_inode(object->backer));
+ oi_size = i_size_read(d_backing_inode(object->dentry));
if (oi_size == ni_size)
return 0;
cachefiles_begin_secure(cache, &saved_cred);
- inode_lock(d_inode(object->backer));
+ inode_lock(d_inode(object->dentry));
/* if there's an extension to a partial page at the end of the backing
* file, we need to discard the partial page so that we pick up new
@@ -383,17 +382,17 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
_debug("discard tail %llx", oi_size);
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = oi_size & PAGE_MASK;
- ret = notify_change(object->backer, &newattrs, NULL);
+ ret = notify_change(object->dentry, &newattrs, NULL);
if (ret < 0)
goto truncate_failed;
}
newattrs.ia_valid = ATTR_SIZE;
newattrs.ia_size = ni_size;
- ret = notify_change(object->backer, &newattrs, NULL);
+ ret = notify_change(object->dentry, &newattrs, NULL);
truncate_failed:
- inode_unlock(d_inode(object->backer));
+ inode_unlock(d_inode(object->dentry));
cachefiles_end_secure(cache, saved_cred);
if (ret == -EIO) {
@@ -426,10 +425,10 @@ static void cachefiles_invalidate_object(struct fscache_object *_object)
_enter("{OBJ%x},[%llu]",
object->fscache.debug_id, (unsigned long long)ni_size);
- if (object->backer) {
- ASSERT(d_is_reg(object->backer));
+ if (object->dentry) {
+ ASSERT(d_is_reg(object->dentry));
- path.dentry = object->backer;
+ path.dentry = object->dentry;
path.mnt = cache->mnt;
cachefiles_begin_secure(cache, &saved_cred);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 9f9e33893b4f..e605cffc284d 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -35,7 +35,7 @@ extern unsigned cachefiles_debug;
struct cachefiles_object {
struct fscache_object fscache; /* fscache handle */
struct dentry *dentry; /* the file/dir representing this object */
- struct dentry *backer; /* backing file */
+ struct dentry *old; /* backing file */
struct file *backing_file; /* File open on backing storage */
loff_t i_size; /* object size */
atomic_t usage; /* object usage count */
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d472e9d9173c..2384067e716d 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -60,11 +60,11 @@ bool cachefiles_open_object(struct cachefiles_object *object)
struct path path;
path.mnt = cache->mnt;
- path.dentry = object->backer;
+ path.dentry = object->dentry;
file = open_with_fake_path(&path,
O_RDWR | O_LARGEFILE | O_DIRECT,
- d_backing_inode(object->backer),
+ d_backing_inode(object->dentry),
cache->cache_cred);
if (IS_ERR(file))
goto error;
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index c33b5ee65ac1..d697a74436c0 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -488,7 +488,7 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
goto check_error;
}
- object->backer = object->dentry;
+ object->old = dget(object->dentry);
} else {
BUG(); // TODO: open file in data-class subdir
}
@@ -523,7 +523,9 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_unmark_inode_in_use(object, object->dentry);
cachefiles_mark_object_inactive(cache, object);
dput(object->dentry);
+ dput(object->old);
object->dentry = NULL;
+ object->old = NULL;
goto error_out;
lookup_error:
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 33/61] cachefiles: Implement a content-present indicator and bitmap
2020-05-04 17:07 ` David Howells
` (32 preceding siblings ...)
(?)
@ 2020-05-04 17:12 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Implement a content indicator that indicates the presence or absence of
content and a bitmap that indicates which blocks of granular content are
present in a granular file. This is added to the xattr that stores the
netfs coherency data, along with the file size and the file zero point (the
point after which it can be assumed that the server doesn't have any data).
In the content bitmap, if present, each bit indicates which 256KiB granules
of a cache file are present. This is stored in a separate xattr, which is
loaded when the first I/O handle is created on that cache object and saved
when the object is discarded from memory.
Non-index objects in the cache can be monolithic or granular. The content
map isn't used for monolithic objects (FSCACHE_COOKIE_ADV_SINGLE_CHUNK) as
they are expected to be all-or-nothing, so the content indicator alone
suffices. Examples of this would be AFS directory or symlink content.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/Makefile | 1
fs/cachefiles/bind.c | 1
fs/cachefiles/content-map.c | 251 +++++++++++++++++++++++++++++++++++++
fs/cachefiles/interface.c | 5 +
fs/cachefiles/internal.h | 31 +++++
fs/cachefiles/io.c | 4 +
fs/cachefiles/xattr.c | 24 +++-
include/trace/events/cachefiles.h | 4 -
8 files changed, 313 insertions(+), 8 deletions(-)
create mode 100644 fs/cachefiles/content-map.c
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index d894d317d6e7..84615aca866a 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -5,6 +5,7 @@
cachefiles-y := \
bind.o \
+ content-map.o \
daemon.o \
interface.o \
io.o \
diff --git a/fs/cachefiles/bind.c b/fs/cachefiles/bind.c
index 84fe89d5999e..40377633e3d9 100644
--- a/fs/cachefiles/bind.c
+++ b/fs/cachefiles/bind.c
@@ -102,6 +102,7 @@ static int cachefiles_daemon_add_cache(struct cachefiles_cache *cache)
goto error_root_object;
atomic_set(&fsdef->usage, 1);
+ rwlock_init(&fsdef->content_map_lock);
fsdef->type = FSCACHE_COOKIE_TYPE_INDEX;
_debug("- fsdef %p", fsdef);
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
new file mode 100644
index 000000000000..594624cb1cb9
--- /dev/null
+++ b/fs/cachefiles/content-map.c
@@ -0,0 +1,251 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Datafile content management
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/mount.h>
+#include <linux/slab.h>
+#include <linux/file.h>
+#include <linux/swap.h>
+#include <linux/xattr.h>
+#include "internal.h"
+
+static const char cachefiles_xattr_content_map[] =
+ XATTR_USER_PREFIX "CacheFiles.content";
+
+static bool cachefiles_granule_is_present(struct cachefiles_object *object,
+ size_t granule)
+{
+ bool res;
+
+ if (granule / 8 >= object->content_map_size)
+ return false;
+ read_lock_bh(&object->content_map_lock);
+ res = test_bit_le(granule, object->content_map);
+ read_unlock_bh(&object->content_map_lock);
+ return res;
+}
+
+/*
+ * Mark the content map to indicate stored granule.
+ */
+void cachefiles_mark_content_map(struct fscache_io_request *req)
+{
+ struct cachefiles_object *object =
+ container_of(req->object, struct cachefiles_object, fscache);
+ loff_t pos = req->pos;
+
+ _enter("%llx", pos);
+
+ read_lock_bh(&object->content_map_lock);
+
+ if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK) {
+ if (pos == 0) {
+ object->content_info = CACHEFILES_CONTENT_SINGLE;
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->fscache.flags);
+ }
+ } else {
+ pgoff_t granule;
+ loff_t end = pos + req->len;
+
+ pos = round_down(pos, CACHEFILES_GRAN_SIZE);
+ do {
+ granule = pos / CACHEFILES_GRAN_SIZE;
+ if (granule / 8 >= object->content_map_size)
+ break;
+
+ set_bit_le(granule, object->content_map);
+ object->content_map_changed = true;
+ pos += CACHEFILES_GRAN_SIZE;
+
+ } while (pos < end);
+
+ if (object->content_info != CACHEFILES_CONTENT_MAP) {
+ object->content_info = CACHEFILES_CONTENT_MAP;
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->fscache.flags);
+ }
+ }
+
+ read_unlock_bh(&object->content_map_lock);
+}
+
+/*
+ * Expand the content map to a larger file size.
+ */
+void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t size)
+{
+ u8 *map, *zap;
+
+ /* Determine the size. There's one bit per granule. We size it in
+ * terms of 8-byte chunks, where a 64-bit span * 256KiB bytes granules
+ * covers 16MiB of file space. At that, 512B will cover 1GiB.
+ */
+ if (size > 0) {
+ size += CACHEFILES_GRAN_SIZE - 1;
+ size /= CACHEFILES_GRAN_SIZE;
+ size += 8 - 1;
+ size /= 8;
+ size = roundup_pow_of_two(size);
+ } else {
+ size = 8;
+ }
+
+ if (size <= object->content_map_size)
+ return;
+
+ map = kzalloc(size, GFP_KERNEL);
+ if (!map)
+ return;
+
+ write_lock_bh(&object->content_map_lock);
+ if (size > object->content_map_size) {
+ zap = object->content_map;
+ memcpy(map, zap, object->content_map_size);
+ object->content_map = map;
+ object->content_map_size = size;
+ } else {
+ zap = map;
+ }
+ write_unlock_bh(&object->content_map_lock);
+
+ kfree(zap);
+}
+
+/*
+ * Adjust the content map when we shorten a backing object.
+ *
+ * We need to unmark any granules that are going to be discarded.
+ */
+void cachefiles_shorten_content_map(struct cachefiles_object *object,
+ loff_t new_size)
+{
+ struct fscache_cookie *cookie = object->fscache.cookie;
+ loff_t granule, o_granule;
+
+ if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK)
+ return;
+
+ write_lock_bh(&object->content_map_lock);
+
+ if (object->content_info == CACHEFILES_CONTENT_MAP) {
+ if (cookie->zero_point > new_size)
+ cookie->zero_point = new_size;
+
+ granule = new_size;
+ granule += CACHEFILES_GRAN_SIZE - 1;
+ granule /= CACHEFILES_GRAN_SIZE;
+
+ o_granule = cookie->object_size;
+ o_granule += CACHEFILES_GRAN_SIZE - 1;
+ o_granule /= CACHEFILES_GRAN_SIZE;
+
+ for (; o_granule > granule; o_granule--)
+ clear_bit_le(o_granule, object->content_map);
+ }
+
+ write_unlock_bh(&object->content_map_lock);
+}
+
+/*
+ * Load the content map.
+ */
+bool cachefiles_load_content_map(struct cachefiles_object *object)
+{
+ struct cachefiles_cache *cache = container_of(object->fscache.cache,
+ struct cachefiles_cache, cache);
+ const struct cred *saved_cred;
+ ssize_t got;
+ loff_t size;
+ u8 *map = NULL;
+
+ _enter("c=%08x,%llx",
+ object->fscache.cookie->debug_id,
+ object->fscache.cookie->object_size);
+
+ object->content_info = CACHEFILES_CONTENT_NO_DATA;
+ if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK) {
+ /* Single-chunk object. The presence or absence of the content
+ * map xattr is sufficient indication.
+ */
+ size = 0;
+ } else {
+ /* Granulated object. There's one bit per granule. We size it
+ * in terms of 8-byte chunks, where a 64-bit span * 256KiB
+ * bytes granules covers 16MiB of file space. At that, 512B
+ * will cover 1GiB.
+ */
+ size = object->fscache.cookie->object_size;
+ if (size > 0) {
+ size += CACHEFILES_GRAN_SIZE - 1;
+ size /= CACHEFILES_GRAN_SIZE;
+ size += 8 - 1;
+ size /= 8;
+ if (size < 8)
+ size = 8;
+ size = roundup_pow_of_two(size);
+ } else {
+ size = 8;
+ }
+
+ map = kzalloc(size, GFP_KERNEL);
+ if (!map)
+ return false;
+ }
+
+ cachefiles_begin_secure(cache, &saved_cred);
+ got = vfs_getxattr(object->dentry, cachefiles_xattr_content_map,
+ map, size);
+ cachefiles_end_secure(cache, saved_cred);
+ if (got < 0 && got != -ENODATA) {
+ kfree(map);
+ _leave(" = f [%zd]", got);
+ return false;
+ }
+
+ if (size == 0) {
+ if (got != -ENODATA)
+ object->content_info = CACHEFILES_CONTENT_SINGLE;
+ _leave(" = t [%zd]", got);
+ } else {
+ object->content_map = map;
+ object->content_map_size = size;
+ object->content_info = CACHEFILES_CONTENT_MAP;
+ _leave(" = t [%zd/%llu %*phN]", got, size, (int)size, map);
+ }
+
+ return true;
+}
+
+/*
+ * Save the content map.
+ */
+void cachefiles_save_content_map(struct cachefiles_object *object)
+{
+ ssize_t ret;
+ size_t size;
+ u8 *map;
+
+ _enter("c=%08x", object->fscache.cookie->debug_id);
+
+ if (object->content_info != CACHEFILES_CONTENT_MAP)
+ return;
+
+ size = object->content_map_size;
+ map = object->content_map;
+
+ /* Don't save trailing zeros, but do save at least one byte */
+ for (; size > 0; size--)
+ if (map[size - 1])
+ break;
+
+ ret = vfs_setxattr(object->dentry, cachefiles_xattr_content_map,
+ map, size, 0);
+ if (ret < 0) {
+ cachefiles_io_error_obj(object, "Unable to set xattr");
+ return;
+ }
+
+ _leave(" = %zd", ret);
+}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index bb68318fcc1c..d820051a9960 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -37,6 +37,7 @@ struct fscache_object *cachefiles_alloc_object(struct fscache_cookie *cookie,
return NULL;
}
+ rwlock_init(&object->content_map_lock);
fscache_object_init(&object->fscache, cookie, &cache->cache);
object->fscache.parent = parent;
object->fscache.stage = FSCACHE_OBJECT_STAGE_LOOKING_UP;
@@ -198,6 +199,8 @@ static void cachefiles_update_object(struct fscache_object *_object)
static void cachefiles_commit_object(struct cachefiles_object *object,
struct cachefiles_cache *cache)
{
+ if (object->content_map_changed)
+ cachefiles_save_content_map(object);
}
/*
@@ -302,6 +305,8 @@ static void cachefiles_put_object(struct fscache_object *_object,
ASSERTCMP(object->dentry, ==, NULL);
ASSERTCMP(object->fscache.n_children, ==, 0);
+ kfree(object->content_map);
+
cache = object->fscache.cache;
fscache_object_destroy(&object->fscache);
kmem_cache_free(cachefiles_object_jar, object);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index e605cffc284d..c7a2a3442061 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -19,6 +19,11 @@
#include <linux/workqueue.h>
#include <linux/security.h>
+/* Cachefile granularity */
+#define CACHEFILES_GRAN_SIZE (256 * 1024)
+#define CACHEFILES_GRAN_PAGES (CACHEFILES_GRAN_SIZE / PAGE_SIZE)
+#define CACHEFILES_DIO_BLOCK_SIZE 4096
+
struct cachefiles_cache;
struct cachefiles_object;
@@ -29,6 +34,16 @@ extern unsigned cachefiles_debug;
#define cachefiles_gfp (__GFP_RECLAIM | __GFP_NORETRY | __GFP_NOMEMALLOC)
+enum cachefiles_content {
+ /* These values are saved on disk */
+ CACHEFILES_CONTENT_NO_DATA = 0, /* No content stored */
+ CACHEFILES_CONTENT_SINGLE = 1, /* Content is monolithic, all is present */
+ CACHEFILES_CONTENT_ALL = 2, /* Content is all present, no map */
+ CACHEFILES_CONTENT_MAP = 3, /* Content is piecemeal, map in use */
+ CACHEFILES_CONTENT_DIRTY = 4, /* Content is dirty (only seen on disk) */
+ nr__cachefiles_content
+};
+
/*
* node records
*/
@@ -41,6 +56,13 @@ struct cachefiles_object {
atomic_t usage; /* object usage count */
uint8_t type; /* object type */
bool new; /* T if object new */
+
+ /* Map of the content blocks in the object */
+ enum cachefiles_content content_info:8; /* Info about content presence */
+ bool content_map_changed;
+ u8 *content_map; /* Content present bitmap */
+ unsigned int content_map_size; /* Size of buffer */
+ rwlock_t content_map_lock;
};
extern struct kmem_cache *cachefiles_object_jar;
@@ -100,6 +122,15 @@ static inline void cachefiles_state_changed(struct cachefiles_cache *cache)
extern int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args);
extern void cachefiles_daemon_unbind(struct cachefiles_cache *cache);
+/*
+ * content-map.c
+ */
+extern void cachefiles_mark_content_map(struct fscache_io_request *req);
+extern void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t size);
+extern void cachefiles_shorten_content_map(struct cachefiles_object *object, loff_t new_size);
+extern bool cachefiles_load_content_map(struct cachefiles_object *object);
+extern void cachefiles_save_content_map(struct cachefiles_object *object);
+
/*
* daemon.c
*/
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 2384067e716d..642c3fd34809 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -62,6 +62,10 @@ bool cachefiles_open_object(struct cachefiles_object *object)
path.mnt = cache->mnt;
path.dentry = object->dentry;
+ if (object->content_info == CACHEFILES_CONTENT_MAP &&
+ !cachefiles_load_content_map(object))
+ goto error;
+
file = open_with_fake_path(&path,
O_RDWR | O_LARGEFILE | O_DIRECT,
d_backing_inode(object->dentry),
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 17c16c2bd07e..a1d4a3d1db69 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -16,8 +16,11 @@
#include "internal.h"
struct cachefiles_xattr {
- uint8_t type;
- uint8_t data[];
+ __be64 object_size; /* Actual size of the object */
+ __be64 zero_point; /* Size after which server has no data not written by us */
+ __u8 type; /* Type of object */
+ __u8 content; /* Content presence (enum cachefiles_content) */
+ __u8 data[]; /* netfs coherency data */
} __packed;
static const char cachefiles_xattr_cache[] =
@@ -118,7 +121,10 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
if (!buf)
return -ENOMEM;
- buf->type = object->fscache.cookie->type;
+ buf->object_size = cpu_to_be64(object->fscache.cookie->object_size);
+ buf->zero_point = cpu_to_be64(object->fscache.cookie->zero_point);
+ buf->type = object->fscache.cookie->type;
+ buf->content = object->content_info;
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
@@ -127,7 +133,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
xattr_flags);
if (ret < 0) {
trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
- 0,
+ buf->content,
cachefiles_coherency_set_fail);
if (ret != -ENOMEM)
cachefiles_io_error_obj(
@@ -135,7 +141,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
"Failed to set xattr with error %d", ret);
} else {
trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
- 0,
+ buf->content,
cachefiles_coherency_set_ok);
}
@@ -174,15 +180,21 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
why = cachefiles_coherency_check_xattr;
} else if (buf->type != object->fscache.cookie->type) {
why = cachefiles_coherency_check_type;
+ } else if (buf->content >= nr__cachefiles_content) {
+ why = cachefiles_coherency_check_content;
} else if (memcmp(buf->data, p, len) != 0) {
why = cachefiles_coherency_check_aux;
+ } else if (be64_to_cpu(buf->object_size) != object->fscache.cookie->object_size) {
+ why = cachefiles_coherency_check_objsize;
} else {
+ object->fscache.cookie->zero_point = be64_to_cpu(buf->zero_point);
+ object->content_info = buf->content;
why = cachefiles_coherency_check_ok;
ret = 0;
}
trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
- 0, why);
+ buf->content, why);
kfree(buf);
return ret;
}
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index bf588c3f4a07..e7af1d683009 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -324,7 +324,7 @@ TRACE_EVENT(cachefiles_mark_buried,
TRACE_EVENT(cachefiles_coherency,
TP_PROTO(struct cachefiles_object *obj,
ino_t ino,
- int content,
+ enum cachefiles_content content,
enum cachefiles_coherency_trace why),
TP_ARGS(obj, ino, content, why),
@@ -333,7 +333,7 @@ TRACE_EVENT(cachefiles_coherency,
TP_STRUCT__entry(
__field(unsigned int, obj )
__field(enum cachefiles_coherency_trace, why )
- __field(int, content )
+ __field(enum cachefiles_content, content )
__field(u64, ino )
),
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 34/61] cachefiles: Implement extent shaper
2020-05-04 17:07 ` David Howells
` (33 preceding siblings ...)
(?)
@ 2020-05-04 17:12 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Implement the function that shapes extents to map onto the granules in a
cache file.
When setting to fetch data from the server to be cached, the extent will be
expanded to align with granule size and cut down so that it doesn't cross
the boundary between a non-present extent and a present extent.
When setting to read data from the cache, the extent will be cut down so
that it doesn't cross the boundary between a present extent and a
non-present extent.
If no caching is taking place, whatever was requested goes.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/content-map.c | 229 ++++++++++++++++++++++++++++++++++++-------
fs/cachefiles/internal.h | 6 +
fs/cachefiles/io.c | 11 --
3 files changed, 197 insertions(+), 49 deletions(-)
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
index 594624cb1cb9..dea28948f006 100644
--- a/fs/cachefiles/content-map.c
+++ b/fs/cachefiles/content-map.c
@@ -15,6 +15,31 @@
static const char cachefiles_xattr_content_map[] =
XATTR_USER_PREFIX "CacheFiles.content";
+/*
+ * Determine the map size for a granulated object.
+ *
+ * There's one bit per granule. We size it in terms of 8-byte chunks, where a
+ * 64-bit span * 256KiB bytes granules covers 16MiB of file space. At that,
+ * 512B will cover 1GiB.
+ */
+static size_t cachefiles_map_size(loff_t i_size)
+{
+ loff_t size;
+ size_t granules, bits, bytes, map_size;
+
+ if (i_size <= CACHEFILES_GRAN_SIZE * 64)
+ return 8;
+
+ size = i_size + CACHEFILES_GRAN_SIZE - 1;
+ granules = size / CACHEFILES_GRAN_SIZE;
+ bits = granules + (64 - 1);
+ bits &= ~(64 - 1);
+ bytes = bits / 8;
+ map_size = roundup_pow_of_two(bytes);
+ _leave(" = %zx [i=%llx g=%zu b=%zu]", map_size, i_size, granules, bits);
+ return map_size;
+}
+
static bool cachefiles_granule_is_present(struct cachefiles_object *object,
size_t granule)
{
@@ -28,6 +53,157 @@ static bool cachefiles_granule_is_present(struct cachefiles_object *object,
return res;
}
+/*
+ * Shape the extent of a single-chunk data object.
+ */
+static unsigned int cachefiles_shape_single(struct fscache_object *obj,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write)
+{
+ struct cachefiles_object *object =
+ container_of(obj, struct cachefiles_object, fscache);
+ unsigned int ret;
+ pgoff_t eof;
+
+ _enter("{%lx,%lx,%lx},%llx,%d",
+ extent->start, extent->block_end, extent->limit,
+ i_size, for_write);
+
+ extent->dio_block_size = CACHEFILES_DIO_BLOCK_SIZE;
+
+ if (object->content_info == CACHEFILES_CONTENT_SINGLE) {
+ ret = FSCACHE_READ_FROM_CACHE;
+ } else {
+ eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ extent->start = 0;
+ extent->block_end = eof;
+ extent->limit = eof;
+ ret = FSCACHE_WRITE_TO_CACHE;
+ }
+
+ _leave(" = %u", ret);
+ return ret;
+}
+
+/*
+ * Determine the size of a data extent in a cache object.
+ *
+ * In cachefiles, a data cache object is divided into granules of 256KiB, each
+ * of which must be written as a whole unit when the cache is being loaded.
+ * Data may be read out piecemeal.
+ *
+ * The extent is resized, but the result will always contain the starting page
+ * from the extent.
+ *
+ * If the granule does not exist in the cachefile, the start may be brought
+ * forward to align with the beginning of a granule boundary, and the end may be
+ * moved either way to align also. The extent will be cut off it it would cross
+ * over the boundary between what's cached and what's not.
+ *
+ * If the starting granule does exist in the cachefile, the extent will be
+ * shortened, if necessary, so that it doesn't cross over into a region that is
+ * not present.
+ *
+ * If the granule does not exist and we cannot cache it for lack of space, the
+ * requested extent is left unaltered.
+ */
+unsigned int cachefiles_shape_extent(struct fscache_object *obj,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write)
+{
+ struct cachefiles_object *object =
+ container_of(obj, struct cachefiles_object, fscache);
+ unsigned int ret = 0;
+ pgoff_t start, end, limit, eof, bend;
+ size_t granule;
+
+ if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK)
+ return cachefiles_shape_single(obj, extent, i_size, for_write);
+
+ start = extent->start;
+ end = extent->block_end;
+ limit = extent->limit;
+ _enter("{%lx,%lx,%lx},%llx,%d", start, end, limit, i_size, for_write);
+
+ granule = start / CACHEFILES_GRAN_PAGES;
+
+ /* If the content map didn't get expanded for some reason - simply
+ * ignore this granule.
+ */
+ if (granule / 8 >= object->content_map_size)
+ return 0;
+
+ if (cachefiles_granule_is_present(object, granule)) {
+ /* The start of the requested extent is present in the cache -
+ * restrict the returned extent to the maximum length of what's
+ * available.
+ */
+ bend = round_up(start + 1, CACHEFILES_GRAN_PAGES);
+ while (bend < end) {
+ pgoff_t i = round_up(bend + 1, CACHEFILES_GRAN_PAGES);
+ granule = i / CACHEFILES_GRAN_PAGES;
+ if (!cachefiles_granule_is_present(object, granule))
+ break;
+ bend = i;
+ }
+
+ if (bend > end)
+ bend = end;
+ end = bend;
+ ret = FSCACHE_READ_FROM_CACHE;
+ } else {
+ /* Otherwise expand the extent in both directions to cover what
+ * we want for caching purposes.
+ */
+ start = round_down(start, CACHEFILES_GRAN_PAGES);
+ end = round_up(end, CACHEFILES_GRAN_PAGES);
+
+ if (limit != ULONG_MAX) {
+ limit = round_down(limit, CACHEFILES_GRAN_PAGES);
+ if (end > limit) {
+ end = limit;
+ if (end <= start) {
+ _leave(" = don't");
+ return 0;
+ }
+ }
+ }
+
+ /* But trim to the end of the file and the starting page */
+ eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ if (eof <= extent->start)
+ eof = extent->start + 1;
+ if (end > eof)
+ end = eof;
+
+ if ((start << PAGE_SHIFT) >= object->fscache.cookie->zero_point) {
+ /* The start of the requested extent is beyond the
+ * original EOF of the file on the server - therefore
+ * it's not going to be found on the server.
+ */
+ bend = round_up(start + 1, CACHEFILES_GRAN_PAGES);
+ end = bend;
+ ret = FSCACHE_FILL_WITH_ZERO;
+ } else {
+ bend = start + CACHEFILES_GRAN_PAGES;
+ if (bend > eof)
+ bend = eof;
+ ret = FSCACHE_WRITE_TO_CACHE;
+ }
+
+ /* TODO: Check we have space in the cache */
+ }
+
+ extent->start = start;
+ extent->block_end = bend;
+ extent->limit = end;
+ extent->dio_block_size = CACHEFILES_DIO_BLOCK_SIZE;
+
+ _leave(" = %u {%lx,%lx,%lx}", ret, start, bend, end);
+ return ret;
+}
+
/*
* Mark the content map to indicate stored granule.
*/
@@ -74,23 +250,14 @@ void cachefiles_mark_content_map(struct fscache_io_request *req)
/*
* Expand the content map to a larger file size.
*/
-void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t size)
+void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t i_size)
{
+ size_t size;
u8 *map, *zap;
- /* Determine the size. There's one bit per granule. We size it in
- * terms of 8-byte chunks, where a 64-bit span * 256KiB bytes granules
- * covers 16MiB of file space. At that, 512B will cover 1GiB.
- */
- if (size > 0) {
- size += CACHEFILES_GRAN_SIZE - 1;
- size /= CACHEFILES_GRAN_SIZE;
- size += 8 - 1;
- size /= 8;
- size = roundup_pow_of_two(size);
- } else {
- size = 8;
- }
+ size = cachefiles_map_size(i_size);
+
+ _enter("%llx,%lx,%x", i_size, size, object->content_map_size);
if (size <= object->content_map_size)
return;
@@ -122,7 +289,7 @@ void cachefiles_shorten_content_map(struct cachefiles_object *object,
loff_t new_size)
{
struct fscache_cookie *cookie = object->fscache.cookie;
- loff_t granule, o_granule;
+ size_t granule, tmp, bytes;
if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK)
return;
@@ -137,12 +304,16 @@ void cachefiles_shorten_content_map(struct cachefiles_object *object,
granule += CACHEFILES_GRAN_SIZE - 1;
granule /= CACHEFILES_GRAN_SIZE;
- o_granule = cookie->object_size;
- o_granule += CACHEFILES_GRAN_SIZE - 1;
- o_granule /= CACHEFILES_GRAN_SIZE;
+ tmp = granule;
+ tmp = round_up(granule, 64);
+ bytes = tmp / 8;
+ if (bytes < object->content_map_size)
+ memset(object->content_map + bytes, 0,
+ object->content_map_size - bytes);
- for (; o_granule > granule; o_granule--)
- clear_bit_le(o_granule, object->content_map);
+ if (tmp > granule)
+ for (tmp--; tmp > granule; tmp--)
+ clear_bit_le(tmp, object->content_map);
}
write_unlock_bh(&object->content_map_lock);
@@ -157,7 +328,7 @@ bool cachefiles_load_content_map(struct cachefiles_object *object)
struct cachefiles_cache, cache);
const struct cred *saved_cred;
ssize_t got;
- loff_t size;
+ size_t size;
u8 *map = NULL;
_enter("c=%08x,%llx",
@@ -176,19 +347,7 @@ bool cachefiles_load_content_map(struct cachefiles_object *object)
* bytes granules covers 16MiB of file space. At that, 512B
* will cover 1GiB.
*/
- size = object->fscache.cookie->object_size;
- if (size > 0) {
- size += CACHEFILES_GRAN_SIZE - 1;
- size /= CACHEFILES_GRAN_SIZE;
- size += 8 - 1;
- size /= 8;
- if (size < 8)
- size = 8;
- size = roundup_pow_of_two(size);
- } else {
- size = 8;
- }
-
+ size = cachefiles_map_size(object->fscache.cookie->object_size);
map = kzalloc(size, GFP_KERNEL);
if (!map)
return false;
@@ -212,7 +371,7 @@ bool cachefiles_load_content_map(struct cachefiles_object *object)
object->content_map = map;
object->content_map_size = size;
object->content_info = CACHEFILES_CONTENT_MAP;
- _leave(" = t [%zd/%llu %*phN]", got, size, (int)size, map);
+ _leave(" = t [%zd/%zu %*phN]", got, size, (int)size, map);
}
return true;
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index c7a2a3442061..43f8e71136dd 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -125,6 +125,9 @@ extern void cachefiles_daemon_unbind(struct cachefiles_cache *cache);
/*
* content-map.c
*/
+extern unsigned int cachefiles_shape_extent(struct fscache_object *object,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write);
extern void cachefiles_mark_content_map(struct fscache_io_request *req);
extern void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t size);
extern void cachefiles_shorten_content_map(struct cachefiles_object *object, loff_t new_size);
@@ -149,9 +152,6 @@ extern struct fscache_object *cachefiles_grab_object(struct fscache_object *_obj
/*
* io.c
*/
-extern unsigned int cachefiles_shape_extent(struct fscache_object *object,
- struct fscache_extent *extent,
- loff_t i_size, bool for_write);
extern int cachefiles_read(struct fscache_object *object,
struct fscache_io_request *req,
struct iov_iter *iter);
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 642c3fd34809..ddb44ec5a199 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -12,17 +12,6 @@
#include <linux/xattr.h>
#include "internal.h"
-/*
- * Determine the size of a data extent in a cache object. This must be written
- * as a whole unit, but can be read piecemeal.
- */
-unsigned int cachefiles_shape_extent(struct fscache_object *object,
- struct fscache_extent *extent,
- loff_t i_size, bool for_write)
-{
- return 0;
-}
-
/*
* Initiate a read from the cache.
*/
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 35/61] cachefiles: Round the cachefile size up to DIO block size
2020-05-04 17:07 ` David Howells
` (34 preceding siblings ...)
(?)
@ 2020-05-04 17:12 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Round the size of a cachefile up to DIO block size so that we can always
read back the last partial page of a file using direct I/O.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index d820051a9960..aa063857272a 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -184,6 +184,17 @@ static void cachefiles_update_object(struct fscache_object *_object)
cachefiles_remove_object_xattr(cache, object->dentry);
goto out;
}
+
+ object_size = round_up(object_size, CACHEFILES_DIO_BLOCK_SIZE);
+ _debug("trunc %llx -> %llx", i_size_read(d_inode(object->dentry)), object_size);
+ if (i_size_read(d_inode(object->dentry)) < object_size) {
+ ret = vfs_truncate(&path, object_size);
+ if (ret < 0) {
+ cachefiles_io_error_obj(object, "Trunc-to-dio-size failed");
+ cachefiles_remove_object_xattr(cache, object->dentry);
+ goto out;
+ }
+ }
}
cachefiles_set_object_xattr(object, XATTR_REPLACE);
@@ -358,6 +369,7 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
int ret;
ni_size = object->fscache.cookie->object_size;
+ ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
_enter("{OBJ%x},[%llu]",
object->fscache.debug_id, (unsigned long long) ni_size);
@@ -426,6 +438,7 @@ static void cachefiles_invalidate_object(struct fscache_object *_object)
struct cachefiles_cache, cache);
ni_size = object->fscache.cookie->object_size;
+ ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
_enter("{OBJ%x},[%llu]",
object->fscache.debug_id, (unsigned long long)ni_size);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 36/61] cachefiles: Implement read and write parts of new I/O API
2020-05-04 17:07 ` David Howells
` (35 preceding siblings ...)
(?)
@ 2020-05-04 17:12 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:12 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Implement writing into the cache and reading back from the cache inside
cachefiles using asynchronous direct I/O from the specified iterator. The
size and position of the request should be aligned to the reported
dio_block_size.
Errors and completion are reported by callback.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/io.c | 208 +++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 202 insertions(+), 6 deletions(-)
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index ddb44ec5a199..42e0d620d778 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -12,30 +12,226 @@
#include <linux/xattr.h>
#include "internal.h"
+struct cachefiles_kiocb {
+ struct kiocb iocb;
+ struct fscache_io_request *req;
+ refcount_t ki_refcnt;
+};
+
+static inline void cachefiles_put_kiocb(struct cachefiles_kiocb *ki)
+{
+ if (refcount_dec_and_test(&ki->ki_refcnt)) {
+ fscache_put_io_request(ki->req);
+ fput(ki->iocb.ki_filp);
+ kfree(ki);
+ }
+}
+
+/*
+ * Handle completion of a read from the cache.
+ */
+static void cachefiles_read_complete(struct kiocb *iocb, long ret, long ret2)
+{
+ struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
+ struct fscache_io_request *req = ki->req;
+
+ _enter("%llx,%ld,%ld", req->len, ret, ret2);
+
+ fscache_end_io_operation(req->cookie);
+
+ if (ret < 0) {
+ req->error = ret;
+ } else if (ret != req->len) {
+ req->error = -ENODATA;
+ } else {
+ req->transferred = ret;
+ set_bit(FSCACHE_IO_DATA_FROM_CACHE, &req->flags);
+ }
+ if (req->io_done)
+ req->io_done(req);
+ cachefiles_put_kiocb(ki);
+}
+
/*
* Initiate a read from the cache.
*/
-int cachefiles_read(struct fscache_object *object,
+int cachefiles_read(struct fscache_object *obj,
struct fscache_io_request *req,
struct iov_iter *iter)
{
- req->error = -ENODATA;
+ struct cachefiles_object *object =
+ container_of(obj, struct cachefiles_object, fscache);
+ struct cachefiles_kiocb *ki;
+ struct file *file = object->backing_file;
+ ssize_t ret = -ENOBUFS;
+
+ _enter("%pD,%li,%llx,%llx/%llx",
+ file, file_inode(file)->i_ino, req->pos, req->len, i_size_read(file->f_inode));
+
+ ki = kzalloc(sizeof(struct cachefiles_kiocb), GFP_KERNEL);
+ if (!ki)
+ goto presubmission_error;
+
+ refcount_set(&ki->ki_refcnt, 2);
+ ki->iocb.ki_filp = get_file(file);
+ ki->iocb.ki_pos = req->pos;
+ ki->iocb.ki_flags = IOCB_DIRECT;
+ ki->iocb.ki_hint = ki_hint_validate(file_write_hint(file));
+ ki->iocb.ki_ioprio = get_current_ioprio();
+ ki->req = req;
+
+ if (req->io_done)
+ ki->iocb.ki_complete = cachefiles_read_complete;
+
+ ret = rw_verify_area(READ, file, &ki->iocb.ki_pos, iov_iter_count(iter));
+ if (ret < 0)
+ goto presubmission_error_free;
+
+ fscache_get_io_request(req);
+ ret = call_read_iter(file, &ki->iocb, iter);
+ switch (ret) {
+ case -EIOCBQUEUED:
+ goto in_progress;
+
+ case -ERESTARTSYS:
+ case -ERESTARTNOINTR:
+ case -ERESTARTNOHAND:
+ case -ERESTART_RESTARTBLOCK:
+ /* There's no easy way to restart the syscall since other AIO's
+ * may be already running. Just fail this IO with EINTR.
+ */
+ ret = -EINTR;
+ /* Fall through */
+ default:
+ cachefiles_read_complete(&ki->iocb, ret, 0);
+ if (ret > 0)
+ ret = 0;
+ break;
+ }
+
+in_progress:
+ cachefiles_put_kiocb(ki);
+ _leave(" = %zd", ret);
+ return ret;
+
+presubmission_error_free:
+ fput(file);
+ kfree(ki);
+presubmission_error:
+ req->error = -ENOMEM;
+ if (req->io_done)
+ req->io_done(req);
+ return -ENOMEM;
+}
+
+/*
+ * Handle completion of a write to the cache.
+ */
+static void cachefiles_write_complete(struct kiocb *iocb, long ret, long ret2)
+{
+ struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
+ struct fscache_io_request *req = ki->req;
+ struct inode *inode = file_inode(ki->iocb.ki_filp);
+
+ _enter("%llx,%ld,%ld", req->len, ret, ret2);
+
+ /* Tell lockdep we inherited freeze protection from submission thread */
+ __sb_writers_acquired(inode->i_sb, SB_FREEZE_WRITE);
+ __sb_end_write(inode->i_sb, SB_FREEZE_WRITE);
+
+ fscache_end_io_operation(req->cookie);
+
+ if (ret < 0)
+ req->error = ret;
+ else if (ret != req->len)
+ req->error = -ENOBUFS;
+ else
+ cachefiles_mark_content_map(req);
if (req->io_done)
req->io_done(req);
- return -ENODATA;
+ cachefiles_put_kiocb(ki);
}
/*
* Initiate a write to the cache.
*/
-int cachefiles_write(struct fscache_object *object,
+int cachefiles_write(struct fscache_object *obj,
struct fscache_io_request *req,
struct iov_iter *iter)
{
- req->error = -ENOBUFS;
+ struct cachefiles_object *object =
+ container_of(obj, struct cachefiles_object, fscache);
+ struct cachefiles_kiocb *ki;
+ struct inode *inode;
+ struct file *file = object->backing_file;
+ ssize_t ret = -ENOBUFS;
+
+ _enter("%pD,%li,%llx,%llx/%llx",
+ file, file_inode(file)->i_ino, req->pos, req->len, i_size_read(file->f_inode));
+
+ ki = kzalloc(sizeof(struct cachefiles_kiocb), GFP_KERNEL);
+ if (!ki)
+ goto presubmission_error;
+
+ refcount_set(&ki->ki_refcnt, 2);
+ ki->iocb.ki_filp = get_file(file);
+ ki->iocb.ki_pos = req->pos;
+ ki->iocb.ki_flags = IOCB_DIRECT | IOCB_WRITE;
+ ki->iocb.ki_hint = ki_hint_validate(file_write_hint(file));
+ ki->iocb.ki_ioprio = get_current_ioprio();
+ ki->req = req;
+
+ if (req->io_done)
+ ki->iocb.ki_complete = cachefiles_write_complete;
+
+ ret = rw_verify_area(WRITE, file, &ki->iocb.ki_pos, iov_iter_count(iter));
+ if (ret < 0)
+ goto presubmission_error_free;
+
+ /* Open-code file_start_write here to grab freeze protection, which
+ * will be released by another thread in aio_complete_rw(). Fool
+ * lockdep by telling it the lock got released so that it doesn't
+ * complain about the held lock when we return to userspace.
+ */
+ inode = file_inode(file);
+ __sb_start_write(inode->i_sb, SB_FREEZE_WRITE, true);
+ __sb_writers_release(inode->i_sb, SB_FREEZE_WRITE);
+
+ fscache_get_io_request(req);
+ ret = call_write_iter(file, &ki->iocb, iter);
+ switch (ret) {
+ case -EIOCBQUEUED:
+ goto in_progress;
+
+ case -ERESTARTSYS:
+ case -ERESTARTNOINTR:
+ case -ERESTARTNOHAND:
+ case -ERESTART_RESTARTBLOCK:
+ /* There's no easy way to restart the syscall since other AIO's
+ * may be already running. Just fail this IO with EINTR.
+ */
+ ret = -EINTR;
+ /* Fall through */
+ default:
+ cachefiles_write_complete(&ki->iocb, ret, 0);
+ if (ret > 0)
+ ret = 0;
+ break;
+ }
+
+in_progress:
+ cachefiles_put_kiocb(ki);
+ _leave(" = %zd", ret);
+ return ret;
+
+presubmission_error_free:
+ fput(file);
+ kfree(ki);
+presubmission_error:
+ req->error = -ENOMEM;
if (req->io_done)
req->io_done(req);
- return -ENOBUFS;
+ return -ENOMEM;
}
/*
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 37/61] cachefiles: Add I/O tracepoints
2020-05-04 17:07 ` David Howells
` (36 preceding siblings ...)
(?)
@ 2020-05-04 17:13 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
---
fs/cachefiles/interface.c | 16 +++--
fs/cachefiles/io.c | 2 +
include/trace/events/cachefiles.h | 123 +++++++++++++++++++++++++++++++++++++
3 files changed, 136 insertions(+), 5 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index aa063857272a..feff39dff0f5 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -160,7 +160,8 @@ static void cachefiles_update_object(struct fscache_object *_object)
struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
- loff_t object_size;
+ struct inode *inode;
+ loff_t object_size, i_size;
int ret;
_enter("{OBJ%x}", _object->debug_id);
@@ -172,12 +173,15 @@ static void cachefiles_update_object(struct fscache_object *_object)
cachefiles_begin_secure(cache, &saved_cred);
object_size = object->fscache.cookie->object_size;
- if (i_size_read(d_inode(object->dentry)) > object_size) {
+ inode = d_inode(object->dentry);
+ i_size = i_size_read(inode);
+ if (i_size > object_size) {
struct path path = {
.mnt = cache->mnt,
.dentry = object->dentry
};
- _debug("trunc %llx -> %llx", i_size_read(d_inode(object->dentry)), object_size);
+ _debug("trunc %llx -> %llx", i_size, object_size);
+ trace_cachefiles_trunc(object, inode, i_size, object_size);
ret = vfs_truncate(&path, object_size);
if (ret < 0) {
cachefiles_io_error_obj(object, "Trunc-to-size failed");
@@ -186,8 +190,10 @@ static void cachefiles_update_object(struct fscache_object *_object)
}
object_size = round_up(object_size, CACHEFILES_DIO_BLOCK_SIZE);
- _debug("trunc %llx -> %llx", i_size_read(d_inode(object->dentry)), object_size);
- if (i_size_read(d_inode(object->dentry)) < object_size) {
+ i_size = i_size_read(inode);
+ _debug("trunc %llx -> %llx", i_size, object_size);
+ if (i_size < object_size) {
+ trace_cachefiles_trunc(object, inode, i_size, object_size);
ret = vfs_truncate(&path, object_size);
if (ret < 0) {
cachefiles_io_error_obj(object, "Trunc-to-dio-size failed");
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 42e0d620d778..268e6f69ba9c 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -88,6 +88,7 @@ int cachefiles_read(struct fscache_object *obj,
goto presubmission_error_free;
fscache_get_io_request(req);
+ trace_cachefiles_read(object, file_inode(file), req);
ret = call_read_iter(file, &ki->iocb, iter);
switch (ret) {
case -EIOCBQUEUED:
@@ -198,6 +199,7 @@ int cachefiles_write(struct fscache_object *obj,
__sb_writers_release(inode->i_sb, SB_FREEZE_WRITE);
fscache_get_io_request(req);
+ trace_cachefiles_write(object, inode, req);
ret = call_write_iter(file, &ki->iocb, iter);
switch (ret) {
case -EIOCBQUEUED:
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index e7af1d683009..d83568e8fee8 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -351,6 +351,129 @@ TRACE_EVENT(cachefiles_coherency,
__entry->content)
);
+TRACE_EVENT(cachefiles_read,
+ TP_PROTO(struct cachefiles_object *obj,
+ struct inode *backer,
+ struct fscache_io_request *req),
+
+ TP_ARGS(obj, backer, req),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(unsigned int, backer )
+ __field(unsigned int, len )
+ __field(loff_t, pos )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->backer = backer->i_ino;
+ __entry->pos = req->pos;
+ __entry->len = req->len;
+ ),
+
+ TP_printk("o=%08x b=%08x p=%llx l=%x",
+ __entry->obj,
+ __entry->backer,
+ __entry->pos,
+ __entry->len)
+ );
+
+TRACE_EVENT(cachefiles_write,
+ TP_PROTO(struct cachefiles_object *obj,
+ struct inode *backer,
+ struct fscache_io_request *req),
+
+ TP_ARGS(obj, backer, req),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(unsigned int, backer )
+ __field(unsigned int, len )
+ __field(loff_t, pos )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->backer = backer->i_ino;
+ __entry->pos = req->pos;
+ __entry->len = req->len;
+ ),
+
+ TP_printk("o=%08x b=%08x p=%llx l=%x",
+ __entry->obj,
+ __entry->backer,
+ __entry->pos,
+ __entry->len)
+ );
+
+TRACE_EVENT(cachefiles_trunc,
+ TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
+ loff_t from, loff_t to),
+
+ TP_ARGS(obj, backer, from, to),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(unsigned int, backer )
+ __field(loff_t, from )
+ __field(loff_t, to )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->backer = backer->i_ino;
+ __entry->from = from;
+ __entry->to = to;
+ ),
+
+ TP_printk("o=%08x b=%08x l=%llx->%llx",
+ __entry->obj,
+ __entry->backer,
+ __entry->from,
+ __entry->to)
+ );
+
+TRACE_EVENT(cachefiles_tmpfile,
+ TP_PROTO(struct cachefiles_object *obj, struct inode *backer),
+
+ TP_ARGS(obj, backer),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(unsigned int, backer )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->backer = backer->i_ino;
+ ),
+
+ TP_printk("o=%08x b=%08x",
+ __entry->obj,
+ __entry->backer)
+ );
+
+TRACE_EVENT(cachefiles_link,
+ TP_PROTO(struct cachefiles_object *obj, struct inode *backer),
+
+ TP_ARGS(obj, backer),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, obj )
+ __field(unsigned int, backer )
+ ),
+
+ TP_fast_assign(
+ __entry->obj = obj->fscache.debug_id;
+ __entry->backer = backer->i_ino;
+ ),
+
+ TP_printk("o=%08x b=%08x",
+ __entry->obj,
+ __entry->backer)
+ );
+
#endif /* _TRACE_CACHEFILES_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 38/61] fscache: Add read helper
@ 2020-05-04 17:13 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Add a helper function, fscache_read_helper(), to do the work of shaping
read requests, attempting to read from the cache, issuing or reissuing
requests to the filesystem to pass to the server and writing back to the
filesystem.
The filesystem passes in a prepared request descriptor with the fscache
descriptor embedded in it to fscache_read_helper(). The caller also
indicates which pages it is particularly interested in and provides some
operations to issue reads and manage the request descriptor.
Also add a tracepoint to track calls. A set of 'notes' are taken to record
the path through the function and this is dumped into the trace.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/Makefile | 3
fs/fscache/read_helper.c | 548 ++++++++++++++++++++++++++++++++++++++++
include/linux/fscache.h | 22 ++
include/trace/events/fscache.h | 44 +++
4 files changed, 616 insertions(+), 1 deletion(-)
create mode 100644 fs/fscache/read_helper.c
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 3caf66810e7b..e9a46544756e 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -12,7 +12,8 @@ fscache-y := \
main.o \
netfs.o \
obj.o \
- object_bits.o
+ object_bits.o \
+ read_helper.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/read_helper.c b/fs/fscache/read_helper.c
new file mode 100644
index 000000000000..9ed6e76ef255
--- /dev/null
+++ b/fs/fscache/read_helper.c
@@ -0,0 +1,548 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Read helper.
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/export.h>
+#include <linux/slab.h>
+#include <linux/uio.h>
+#include <linux/task_io_accounting_ops.h>
+#include <linux/fscache-cache.h>
+#include "internal.h"
+
+static void fscache_read_from_server(struct fscache_io_request *req)
+{
+ req->ops->issue_op(req);
+}
+
+/*
+ * Deal with the completion of writing the data to the cache. We have to clear
+ * the PG_fscache bits on the pages involved and releases the caller's ref.
+ */
+static void fscache_read_copy_done(struct fscache_io_request *req)
+{
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->mapping->i_pages, index);
+
+ _enter("%lx,%x,%llx", index, req->nr_pages, req->transferred);
+
+ /* Clear PG_fscache on the pages that were being written out. */
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ unlock_page_fscache(page);
+ }
+ rcu_read_unlock();
+}
+
+/*
+ * Write a completed read request to the cache.
+ */
+static void fscache_do_read_copy_to_cache(struct work_struct *work)
+{
+ struct fscache_io_request *req =
+ container_of(work, struct fscache_io_request, work);
+ struct iov_iter iter;
+
+ _enter("");
+
+ iov_iter_mapping(&iter, WRITE, req->mapping, req->pos,
+ round_up(req->len, req->dio_block_size));
+
+ req->io_done = fscache_read_copy_done;
+ fscache_write(req, &iter);
+ fscache_put_io_request(req);
+}
+
+static void fscache_read_copy_to_cache(struct fscache_io_request *req)
+{
+ fscache_get_io_request(req);
+
+ if (!in_softirq())
+ return fscache_do_read_copy_to_cache(&req->work);
+
+ BUG_ON(work_pending(&req->work));
+ INIT_WORK(&req->work, fscache_do_read_copy_to_cache);
+ if (!queue_work(fscache_op_wq, &req->work))
+ BUG();
+}
+
+/*
+ * Clear the unread part of the file on a short read.
+ */
+static void fscache_clear_unread(struct fscache_io_request *req)
+{
+ struct iov_iter iter;
+
+ iov_iter_mapping(&iter, WRITE, req->mapping,
+ req->pos + req->transferred,
+ req->len - req->transferred);
+
+ _debug("clear %zx @%llx", iov_iter_count(&iter), iter.mapping_start);
+
+ iov_iter_zero(iov_iter_count(&iter), &iter);
+}
+
+/*
+ * Handle completion of a read operation. This may be called in softirq
+ * context.
+ */
+static void fscache_read_done(struct fscache_io_request *req)
+{
+ struct page *page;
+ pgoff_t start = req->pos >> PAGE_SHIFT;
+ pgoff_t last = start + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->mapping->i_pages, start);
+
+ _enter("%lx,%x,%llx,%d",
+ start, req->nr_pages, req->transferred, req->error);
+
+ if (req->transferred < req->len)
+ fscache_clear_unread(req);
+
+ if (!test_bit(FSCACHE_IO_DONT_UNLOCK_PAGES, &req->flags)) {
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (req->write_to_cache)
+ SetPageFsCache(page);
+ if (page == req->no_unlock_page)
+ SetPageUptodate(page);
+ else
+ page_endio(page, false, 0);
+ put_page(page);
+ }
+ rcu_read_unlock();
+ }
+
+ task_io_account_read(req->transferred);
+ req->ops->done(req);
+
+ if (req->write_to_cache)
+ fscache_read_copy_to_cache(req);
+}
+
+/*
+ * Reissue the read against the server.
+ */
+static void fscache_reissue_read(struct work_struct *work)
+{
+ struct fscache_io_request *req =
+ container_of(work, struct fscache_io_request, work);
+
+ _debug("DOWNLOAD: %llu", req->len);
+
+ req->io_done = fscache_read_done;
+ fscache_read_from_server(req);
+ fscache_put_io_request(req);
+}
+
+/*
+ * Handle completion of a read from cache operation. If the read failed, we
+ * need to reissue the request against the server. We might, however, be
+ * called in softirq mode and need to punt.
+ */
+static void fscache_file_read_maybe_reissue(struct fscache_io_request *req)
+{
+ _enter("%d", req->error);
+
+ if (req->error == 0) {
+ fscache_read_done(req);
+ } else {
+ INIT_WORK(&req->work, fscache_reissue_read);
+ fscache_get_io_request(req);
+ queue_work(fscache_op_wq, &req->work);
+ }
+}
+
+/*
+ * Issue a read against the cache.
+ */
+static void fscache_read_from_cache(struct fscache_io_request *req)
+{
+ struct iov_iter iter;
+
+ iov_iter_mapping(&iter, READ, req->mapping, req->pos, req->len);
+ fscache_read(req, &iter);
+}
+
+/*
+ * Discard the locks and page refs that we obtained on a sequence of pages.
+ */
+static void fscache_ignore_pages(struct address_space *mapping,
+ pgoff_t start, pgoff_t end)
+{
+ struct page *page;
+
+ _enter("%lx,%lx", start, end);
+
+ if (end > start) {
+ XA_STATE(xas, &mapping->i_pages, start);
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, end - 1) {
+ _debug("- ignore %lx", page->index);
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ unlock_page(page);
+ put_page(page);
+ }
+ rcu_read_unlock();
+ }
+}
+
+#define FSCACHE_RHLP_NOTE_READ_FROM_CACHE FSCACHE_READ_FROM_CACHE
+#define FSCACHE_RHLP_NOTE_WRITE_TO_CACHE FSCACHE_WRITE_TO_CACHE
+#define FSCACHE_RHLP_NOTE_FILL_WITH_ZERO FSCACHE_FILL_WITH_ZERO
+#define FSCACHE_RHLP_NOTE_READ_FOR_WRITE 0x000100 /* Type: FSCACHE_READ_FOR_WRITE */
+#define FSCACHE_RHLP_NOTE_READ_LOCKED_PAGE 0x000200 /* Type: FSCACHE_READ_LOCKED_PAGE */
+#define FSCACHE_RHLP_NOTE_READ_PAGE_LIST 0x000300 /* Type: FSCACHE_READ_PAGE_LIST */
+#define FSCACHE_RHLP_NOTE_LIST_NOTCONTIG 0x001000 /* Page list: not contiguous */
+#define FSCACHE_RHLP_NOTE_LIST_NOMEM 0x002000 /* Page list: ENOMEM */
+#define FSCACHE_RHLP_NOTE_LIST_U2D 0x004000 /* Page list: page uptodate */
+#define FSCACHE_RHLP_NOTE_LIST_ERROR 0x008000 /* Page list: add error */
+#define FSCACHE_RHLP_NOTE_TRAILER_ADD 0x010000 /* Trailer: Creating */
+#define FSCACHE_RHLP_NOTE_TRAILER_NOMEM 0x020000 /* Trailer: ENOMEM */
+#define FSCACHE_RHLP_NOTE_TRAILER_U2D 0x040000 /* Trailer: Uptodate */
+#define FSCACHE_RHLP_NOTE_U2D_IN_PREFACE 0x100000 /* Uptodate page in preface */
+#define FSCACHE_RHLP_NOTE_UNDERSIZED 0x200000 /* Undersized block */
+#define FSCACHE_RHLP_NOTE_AFTER_EOF 0x800000 /* After EOF */
+
+/**
+ * fscache_read_helper - Helper to manage a read request
+ * @req: The initialised request structure to use
+ * @extent: The extent of the pages to access
+ * @requested_page: Singular page to include
+ * @pages: Unattached pages to include (readpages)
+ * @type: FSCACHE_READ_*
+ * @aop_flags: AOP_FLAG_*
+ *
+ * Read a sequence of pages appropriately sized for an fscache allocation
+ * block. Pages are added at both ends and to fill in the gaps as appropriate
+ * to make it the right size.
+ *
+ * req->mapping should indicate the mapping to which the pages will be attached.
+ *
+ * The operations pointed to by req->ops will be used to issue or reissue a
+ * read against the server in case the cache is unavailable, incomplete or
+ * generates an error. req->iter will be set up to point to the iterator
+ * representing the buffer to be filled in.
+ *
+ * A ref on @req is consumed eventually by this function or one of its
+ * eventually-dispatched callees.
+ */
+int fscache_read_helper(struct fscache_io_request *req,
+ struct fscache_extent *extent,
+ struct page **requested_page,
+ struct list_head *pages,
+ enum fscache_read_type type,
+ unsigned int aop_flags)
+{
+ struct address_space *mapping = req->mapping;
+ struct page *page;
+ enum fscache_read_helper_trace what;
+ unsigned int notes;
+ pgoff_t eof, cursor, start, first_index, trailer = ULONG_MAX;
+ loff_t i_size;
+ int ret;
+
+ first_index = extent->start;
+ _enter("{%lx,%lx}", first_index, extent->limit);
+
+ ASSERTIFCMP(requested_page && *requested_page,
+ (*requested_page)->index, ==, first_index);
+ ASSERTIF(type == FSCACHE_READ_LOCKED_PAGE ||
+ type == FSCACHE_READ_FOR_WRITE,
+ pages == NULL);
+ ASSERTIFCMP(pages && !list_empty(pages),
+ first_index, ==, lru_to_page(pages)->index);
+
+ i_size = i_size_read(mapping->host);
+ if (type == FSCACHE_READ_FOR_WRITE) {
+ loff_t new_size = extent->block_end;
+ new_size <<= PAGE_SHIFT;
+ if (new_size > i_size)
+ i_size = new_size;
+ }
+ eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ notes = fscache_shape_extent(req->cookie, extent, i_size, false);
+ req->dio_block_size = extent->dio_block_size;
+
+ start = cursor = extent->start;
+
+ /* Add pages to the pagecache. We keep the pages ref'd and locked
+ * until the read is complete. We may also need to add pages to both
+ * sides of the request to make it up to the cache allocation granule
+ * alignment and size.
+ *
+ * Note that it's possible for the file size to change whilst we're
+ * doing this, but we rely on the server returning less than we asked
+ * for if the file shrank. We also rely on this to deal with a partial
+ * page at the end of the file.
+ *
+ * If we're going to end up loading from the server and writing to the
+ * cache, we start by inserting blank pages before the first page being
+ * examined. If we can fetch from the cache or we're not going to
+ * write to the cache, it's unnecessary.
+ */
+ if (notes & FSCACHE_RHLP_NOTE_WRITE_TO_CACHE) {
+ req->write_to_cache = true;
+ while (cursor < first_index) {
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page)
+ goto nomem;
+ if (!PageUptodate(page)) {
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ continue;
+ }
+
+ /* There's an up-to-date page in the preface - just
+ * fetch the requested pages and skip saving to the
+ * cache.
+ */
+ notes |= FSCACHE_RHLP_NOTE_U2D_IN_PREFACE;
+ fscache_ignore_pages(mapping, start, cursor + 1);
+ req->write_to_cache = false;
+ start = cursor = first_index;
+ req->nr_pages = 0;
+ break;
+ }
+ page = NULL;
+ } else {
+ req->write_to_cache = false;
+ start = cursor = first_index;
+ req->nr_pages = 0;
+ }
+
+ switch (type) {
+ case FSCACHE_READ_FOR_WRITE:
+ /* We're doing a prefetch for a write on a single page. We get
+ * or create the requested page if we weren't given it and lock
+ * it.
+ */
+ notes |= FSCACHE_RHLP_NOTE_READ_FOR_WRITE;
+ if (*requested_page) {
+ _debug("prewrite req %lx", cursor);
+ page = *requested_page;
+ ret = -ERESTARTSYS;
+ if (lock_page_killable(page) < 0)
+ goto dont;
+ } else {
+ _debug("prewrite new %lx %lx", cursor, eof);
+ page = grab_cache_page_write_begin(mapping, first_index,
+ aop_flags);
+ if (!page)
+ goto nomem;
+ *requested_page = page;
+ }
+ get_page(page);
+ req->no_unlock_page = page;
+ req->nr_pages++;
+ cursor++;
+ page = NULL;
+ ret = 0;
+ break;
+
+ case FSCACHE_READ_LOCKED_PAGE:
+ /* We've got a single page preattached to the inode and locked.
+ * Get our own ref on it.
+ */
+ _debug("locked");
+ notes |= FSCACHE_RHLP_NOTE_READ_LOCKED_PAGE;
+ get_page(*requested_page);
+ req->nr_pages++;
+ cursor++;
+ ret = 0;
+ break;
+
+ case FSCACHE_READ_PAGE_LIST:
+ /* We've been given a contiguous list of pages to add. */
+ notes |= FSCACHE_RHLP_NOTE_READ_PAGE_LIST;
+ do {
+ _debug("given %lx", cursor);
+
+ page = lru_to_page(pages);
+ if (page->index != cursor) {
+ notes |= FSCACHE_RHLP_NOTE_LIST_NOTCONTIG;
+ break;
+ }
+
+ list_del(&page->lru);
+
+ ret = add_to_page_cache_lru(page, mapping, cursor,
+ readahead_gfp_mask(mapping));
+ switch (ret) {
+ case 0:
+ /* Add to the reading list */
+ req->nr_pages++;
+ cursor++;
+ page = NULL;
+ break;
+
+ case -EEXIST:
+ put_page(page);
+
+ _debug("conflict %lx %d", cursor, ret);
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page) {
+ notes |= FSCACHE_RHLP_NOTE_LIST_NOMEM;
+ goto stop;
+ }
+
+ if (PageUptodate(page)) {
+ unlock_page(page);
+ put_page(page); /* Avoid overwriting */
+ ret = 0;
+ notes |= FSCACHE_RHLP_NOTE_LIST_U2D;
+ goto stop;
+ }
+
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ break;
+
+ default:
+ _debug("add fail %lx %d", cursor, ret);
+ put_page(page);
+ page = NULL;
+ notes |= FSCACHE_RHLP_NOTE_LIST_ERROR;
+ goto stop;
+ }
+
+ /* Trim the fetch to the cache granularity so we don't
+ * get a chain-failure of blocks being unable to be
+ * used because the previous uncached read spilt over.
+ */
+ if ((notes & FSCACHE_RHLP_NOTE_U2D_IN_PREFACE) &&
+ cursor == extent->block_end)
+ break;
+
+ } while (!list_empty(pages) && cursor < extent->limit);
+ ret = 0;
+ break;
+
+ default:
+ BUG();
+ }
+
+ /* If we're going to be writing to the cache, insert pages after the
+ * requested block to make up the numbers.
+ */
+ if (req->write_to_cache) {
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_ADD;
+ trailer = cursor;
+ while (cursor < extent->limit) {
+ _debug("after %lx", cursor);
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page) {
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_NOMEM;
+ goto stop;
+ }
+ if (PageUptodate(page)) {
+ unlock_page(page);
+ put_page(page); /* Avoid overwriting */
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_U2D;
+ goto stop;
+ }
+
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ }
+ }
+
+stop:
+ _debug("have %u", req->nr_pages);
+ if (req->nr_pages == 0)
+ goto dont;
+
+ if (cursor <= first_index) {
+ _debug("v.short");
+ goto nomem_unlock; /* We wouldn't've included the first page */
+ }
+
+submit_anyway:
+ if (cursor < extent->block_end) {
+ /* The request is short of what we need to be able to cache the
+ * minimum cache block so discard the trailer.
+ */
+ _debug("short");
+ notes |= FSCACHE_RHLP_NOTE_UNDERSIZED;
+ req->write_to_cache = false;
+ if (trailer != ULONG_MAX) {
+ fscache_ignore_pages(mapping, trailer, cursor);
+ req->nr_pages -= cursor - trailer;
+ }
+ }
+
+ req->len = req->nr_pages * PAGE_SIZE;
+ req->pos = start;
+ req->pos <<= PAGE_SHIFT;
+
+ if (start >= eof) {
+ notes |= FSCACHE_RHLP_NOTE_AFTER_EOF;
+ what = fscache_read_helper_skip;
+ } else if (notes & FSCACHE_RHLP_NOTE_FILL_WITH_ZERO) {
+ what = fscache_read_helper_zero;
+ } else if (notes & FSCACHE_RHLP_NOTE_READ_FROM_CACHE) {
+ what = fscache_read_helper_read;
+ } else {
+ what = fscache_read_helper_download;
+ }
+
+ trace_fscache_read_helper(req->cookie, start, start + req->nr_pages,
+ notes, what);
+
+ switch (what) {
+ case fscache_read_helper_skip:
+ /* The read is entirely beyond the end of the file, so skip the
+ * actual operation and let the done handler deal with clearing
+ * the pages.
+ */
+ _debug("SKIP READ: %llu", req->len);
+ fscache_read_done(req);
+ break;
+ case fscache_read_helper_zero:
+ _debug("ZERO READ: %llu", req->len);
+ fscache_read_done(req);
+ break;
+ case fscache_read_helper_read:
+ req->io_done = fscache_file_read_maybe_reissue;
+ fscache_read_from_cache(req);
+ break;
+ case fscache_read_helper_download:
+ _debug("DOWNLOAD: %llu", req->len);
+ req->io_done = fscache_read_done;
+ fscache_read_from_server(req);
+ break;
+ default:
+ BUG();
+ }
+
+ _leave(" = 0");
+ return 0;
+
+nomem:
+ if (cursor > first_index)
+ goto submit_anyway;
+nomem_unlock:
+ fscache_ignore_pages(mapping, start, cursor);
+ ret = -ENOMEM;
+dont:
+ _leave(" = %d", ret);
+ return ret;
+}
+EXPORT_SYMBOL(fscache_read_helper);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index b46df865911c..ad44fa4cd844 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -152,7 +152,16 @@ struct fscache_io_request {
unsigned long flags;
#define FSCACHE_IO_DATA_FROM_SERVER 0 /* Set if data was read from server */
#define FSCACHE_IO_DATA_FROM_CACHE 1 /* Set if data was read from the cache */
+#define FSCACHE_IO_DONT_UNLOCK_PAGES 2 /* Don't unlock the pages on completion */
void (*io_done)(struct fscache_io_request *);
+ struct work_struct work;
+
+ /* Bits for readpages helper */
+ struct address_space *mapping; /* The mapping being accessed */
+ unsigned int nr_pages; /* Number of pages involved in the I/O */
+ unsigned int dio_block_size; /* Rounding for direct I/O in the cache */
+ bool write_to_cache; /* T if should write to the cache */
+ struct page *no_unlock_page; /* Don't unlock this page after read */
};
struct fscache_io_request_ops {
@@ -571,4 +580,17 @@ int fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
return -ENOBUFS;
}
+enum fscache_read_type {
+ FSCACHE_READ_PAGE_LIST, /* Read the list of pages (readpages) */
+ FSCACHE_READ_LOCKED_PAGE, /* requested_page is added and locked */
+ FSCACHE_READ_FOR_WRITE, /* This read is a prelude to write_begin */
+};
+
+extern int fscache_read_helper(struct fscache_io_request *,
+ struct fscache_extent *,
+ struct page **,
+ struct list_head *,
+ enum fscache_read_type,
+ unsigned int);
+
#endif /* _LINUX_FSCACHE_H */
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 4ce81f3f4c5b..e78a3fa9b633 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -39,6 +39,13 @@ enum fscache_cookie_trace {
fscache_cookie_see_discard,
};
+enum fscache_read_helper_trace {
+ fscache_read_helper_download,
+ fscache_read_helper_read,
+ fscache_read_helper_skip,
+ fscache_read_helper_zero,
+};
+
#endif
/*
@@ -63,6 +70,13 @@ enum fscache_cookie_trace {
EM(fscache_cookie_put_work, "PUT work ") \
E_(fscache_cookie_see_discard, "SEE discd")
+#define fscache_read_helper_traces \
+ EM(fscache_read_helper_download, "DOWN") \
+ EM(fscache_read_helper_read, "READ") \
+ EM(fscache_read_helper_skip, "SKIP") \
+ E_(fscache_read_helper_zero, "ZERO")
+
+
/*
* Export enum symbols via userspace.
*/
@@ -72,6 +86,7 @@ enum fscache_cookie_trace {
#define E_(a, b) TRACE_DEFINE_ENUM(a);
fscache_cookie_traces;
+fscache_read_helper_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -198,6 +213,35 @@ TRACE_EVENT(fscache_relinquish,
__entry->flags, __entry->retire)
);
+TRACE_EVENT(fscache_read_helper,
+ TP_PROTO(struct fscache_cookie *cookie, pgoff_t start, pgoff_t end,
+ unsigned int notes, enum fscache_read_helper_trace what),
+
+ TP_ARGS(cookie, start, end, notes, what),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, cookie )
+ __field(pgoff_t, start )
+ __field(pgoff_t, end )
+ __field(unsigned int, notes )
+ __field(enum fscache_read_helper_trace, what )
+ ),
+
+ TP_fast_assign(
+ __entry->cookie = cookie->debug_id;
+ __entry->start = start;
+ __entry->end = end;
+ __entry->what = what;
+ __entry->notes = notes;
+ ),
+
+ TP_printk("c=%08x %s n=%08x p=%lx-%lx",
+ __entry->cookie,
+ __print_symbolic(__entry->what, fscache_read_helper_traces),
+ __entry->notes,
+ __entry->start, __entry->end)
+ );
+
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 38/61] fscache: Add read helper
@ 2020-05-04 17:13 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Add a helper function, fscache_read_helper(), to do the work of shaping
read requests, attempting to read from the cache, issuing or reissuing
requests to the filesystem to pass to the server and writing back to the
filesystem.
The filesystem passes in a prepared request descriptor with the fscache
descriptor embedded in it to fscache_read_helper(). The caller also
indicates which pages it is particularly interested in and provides some
operations to issue reads and manage the request descriptor.
Also add a tracepoint to track calls. A set of 'notes' are taken to record
the path through the function and this is dumped into the trace.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/fscache/Makefile | 3
fs/fscache/read_helper.c | 548 ++++++++++++++++++++++++++++++++++++++++
include/linux/fscache.h | 22 ++
include/trace/events/fscache.h | 44 +++
4 files changed, 616 insertions(+), 1 deletion(-)
create mode 100644 fs/fscache/read_helper.c
diff --git a/fs/fscache/Makefile b/fs/fscache/Makefile
index 3caf66810e7b..e9a46544756e 100644
--- a/fs/fscache/Makefile
+++ b/fs/fscache/Makefile
@@ -12,7 +12,8 @@ fscache-y := \
main.o \
netfs.o \
obj.o \
- object_bits.o
+ object_bits.o \
+ read_helper.o
fscache-$(CONFIG_PROC_FS) += proc.o
fscache-$(CONFIG_FSCACHE_STATS) += stats.o
diff --git a/fs/fscache/read_helper.c b/fs/fscache/read_helper.c
new file mode 100644
index 000000000000..9ed6e76ef255
--- /dev/null
+++ b/fs/fscache/read_helper.c
@@ -0,0 +1,548 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Read helper.
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
+ */
+
+#define FSCACHE_DEBUG_LEVEL OPERATION
+#include <linux/export.h>
+#include <linux/slab.h>
+#include <linux/uio.h>
+#include <linux/task_io_accounting_ops.h>
+#include <linux/fscache-cache.h>
+#include "internal.h"
+
+static void fscache_read_from_server(struct fscache_io_request *req)
+{
+ req->ops->issue_op(req);
+}
+
+/*
+ * Deal with the completion of writing the data to the cache. We have to clear
+ * the PG_fscache bits on the pages involved and releases the caller's ref.
+ */
+static void fscache_read_copy_done(struct fscache_io_request *req)
+{
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->mapping->i_pages, index);
+
+ _enter("%lx,%x,%llx", index, req->nr_pages, req->transferred);
+
+ /* Clear PG_fscache on the pages that were being written out. */
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ unlock_page_fscache(page);
+ }
+ rcu_read_unlock();
+}
+
+/*
+ * Write a completed read request to the cache.
+ */
+static void fscache_do_read_copy_to_cache(struct work_struct *work)
+{
+ struct fscache_io_request *req =
+ container_of(work, struct fscache_io_request, work);
+ struct iov_iter iter;
+
+ _enter("");
+
+ iov_iter_mapping(&iter, WRITE, req->mapping, req->pos,
+ round_up(req->len, req->dio_block_size));
+
+ req->io_done = fscache_read_copy_done;
+ fscache_write(req, &iter);
+ fscache_put_io_request(req);
+}
+
+static void fscache_read_copy_to_cache(struct fscache_io_request *req)
+{
+ fscache_get_io_request(req);
+
+ if (!in_softirq())
+ return fscache_do_read_copy_to_cache(&req->work);
+
+ BUG_ON(work_pending(&req->work));
+ INIT_WORK(&req->work, fscache_do_read_copy_to_cache);
+ if (!queue_work(fscache_op_wq, &req->work))
+ BUG();
+}
+
+/*
+ * Clear the unread part of the file on a short read.
+ */
+static void fscache_clear_unread(struct fscache_io_request *req)
+{
+ struct iov_iter iter;
+
+ iov_iter_mapping(&iter, WRITE, req->mapping,
+ req->pos + req->transferred,
+ req->len - req->transferred);
+
+ _debug("clear %zx @%llx", iov_iter_count(&iter), iter.mapping_start);
+
+ iov_iter_zero(iov_iter_count(&iter), &iter);
+}
+
+/*
+ * Handle completion of a read operation. This may be called in softirq
+ * context.
+ */
+static void fscache_read_done(struct fscache_io_request *req)
+{
+ struct page *page;
+ pgoff_t start = req->pos >> PAGE_SHIFT;
+ pgoff_t last = start + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->mapping->i_pages, start);
+
+ _enter("%lx,%x,%llx,%d",
+ start, req->nr_pages, req->transferred, req->error);
+
+ if (req->transferred < req->len)
+ fscache_clear_unread(req);
+
+ if (!test_bit(FSCACHE_IO_DONT_UNLOCK_PAGES, &req->flags)) {
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (req->write_to_cache)
+ SetPageFsCache(page);
+ if (page == req->no_unlock_page)
+ SetPageUptodate(page);
+ else
+ page_endio(page, false, 0);
+ put_page(page);
+ }
+ rcu_read_unlock();
+ }
+
+ task_io_account_read(req->transferred);
+ req->ops->done(req);
+
+ if (req->write_to_cache)
+ fscache_read_copy_to_cache(req);
+}
+
+/*
+ * Reissue the read against the server.
+ */
+static void fscache_reissue_read(struct work_struct *work)
+{
+ struct fscache_io_request *req =
+ container_of(work, struct fscache_io_request, work);
+
+ _debug("DOWNLOAD: %llu", req->len);
+
+ req->io_done = fscache_read_done;
+ fscache_read_from_server(req);
+ fscache_put_io_request(req);
+}
+
+/*
+ * Handle completion of a read from cache operation. If the read failed, we
+ * need to reissue the request against the server. We might, however, be
+ * called in softirq mode and need to punt.
+ */
+static void fscache_file_read_maybe_reissue(struct fscache_io_request *req)
+{
+ _enter("%d", req->error);
+
+ if (req->error == 0) {
+ fscache_read_done(req);
+ } else {
+ INIT_WORK(&req->work, fscache_reissue_read);
+ fscache_get_io_request(req);
+ queue_work(fscache_op_wq, &req->work);
+ }
+}
+
+/*
+ * Issue a read against the cache.
+ */
+static void fscache_read_from_cache(struct fscache_io_request *req)
+{
+ struct iov_iter iter;
+
+ iov_iter_mapping(&iter, READ, req->mapping, req->pos, req->len);
+ fscache_read(req, &iter);
+}
+
+/*
+ * Discard the locks and page refs that we obtained on a sequence of pages.
+ */
+static void fscache_ignore_pages(struct address_space *mapping,
+ pgoff_t start, pgoff_t end)
+{
+ struct page *page;
+
+ _enter("%lx,%lx", start, end);
+
+ if (end > start) {
+ XA_STATE(xas, &mapping->i_pages, start);
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, end - 1) {
+ _debug("- ignore %lx", page->index);
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ unlock_page(page);
+ put_page(page);
+ }
+ rcu_read_unlock();
+ }
+}
+
+#define FSCACHE_RHLP_NOTE_READ_FROM_CACHE FSCACHE_READ_FROM_CACHE
+#define FSCACHE_RHLP_NOTE_WRITE_TO_CACHE FSCACHE_WRITE_TO_CACHE
+#define FSCACHE_RHLP_NOTE_FILL_WITH_ZERO FSCACHE_FILL_WITH_ZERO
+#define FSCACHE_RHLP_NOTE_READ_FOR_WRITE 0x000100 /* Type: FSCACHE_READ_FOR_WRITE */
+#define FSCACHE_RHLP_NOTE_READ_LOCKED_PAGE 0x000200 /* Type: FSCACHE_READ_LOCKED_PAGE */
+#define FSCACHE_RHLP_NOTE_READ_PAGE_LIST 0x000300 /* Type: FSCACHE_READ_PAGE_LIST */
+#define FSCACHE_RHLP_NOTE_LIST_NOTCONTIG 0x001000 /* Page list: not contiguous */
+#define FSCACHE_RHLP_NOTE_LIST_NOMEM 0x002000 /* Page list: ENOMEM */
+#define FSCACHE_RHLP_NOTE_LIST_U2D 0x004000 /* Page list: page uptodate */
+#define FSCACHE_RHLP_NOTE_LIST_ERROR 0x008000 /* Page list: add error */
+#define FSCACHE_RHLP_NOTE_TRAILER_ADD 0x010000 /* Trailer: Creating */
+#define FSCACHE_RHLP_NOTE_TRAILER_NOMEM 0x020000 /* Trailer: ENOMEM */
+#define FSCACHE_RHLP_NOTE_TRAILER_U2D 0x040000 /* Trailer: Uptodate */
+#define FSCACHE_RHLP_NOTE_U2D_IN_PREFACE 0x100000 /* Uptodate page in preface */
+#define FSCACHE_RHLP_NOTE_UNDERSIZED 0x200000 /* Undersized block */
+#define FSCACHE_RHLP_NOTE_AFTER_EOF 0x800000 /* After EOF */
+
+/**
+ * fscache_read_helper - Helper to manage a read request
+ * @req: The initialised request structure to use
+ * @extent: The extent of the pages to access
+ * @requested_page: Singular page to include
+ * @pages: Unattached pages to include (readpages)
+ * @type: FSCACHE_READ_*
+ * @aop_flags: AOP_FLAG_*
+ *
+ * Read a sequence of pages appropriately sized for an fscache allocation
+ * block. Pages are added at both ends and to fill in the gaps as appropriate
+ * to make it the right size.
+ *
+ * req->mapping should indicate the mapping to which the pages will be attached.
+ *
+ * The operations pointed to by req->ops will be used to issue or reissue a
+ * read against the server in case the cache is unavailable, incomplete or
+ * generates an error. req->iter will be set up to point to the iterator
+ * representing the buffer to be filled in.
+ *
+ * A ref on @req is consumed eventually by this function or one of its
+ * eventually-dispatched callees.
+ */
+int fscache_read_helper(struct fscache_io_request *req,
+ struct fscache_extent *extent,
+ struct page **requested_page,
+ struct list_head *pages,
+ enum fscache_read_type type,
+ unsigned int aop_flags)
+{
+ struct address_space *mapping = req->mapping;
+ struct page *page;
+ enum fscache_read_helper_trace what;
+ unsigned int notes;
+ pgoff_t eof, cursor, start, first_index, trailer = ULONG_MAX;
+ loff_t i_size;
+ int ret;
+
+ first_index = extent->start;
+ _enter("{%lx,%lx}", first_index, extent->limit);
+
+ ASSERTIFCMP(requested_page && *requested_page,
+ (*requested_page)->index, ==, first_index);
+ ASSERTIF(type == FSCACHE_READ_LOCKED_PAGE ||
+ type == FSCACHE_READ_FOR_WRITE,
+ pages == NULL);
+ ASSERTIFCMP(pages && !list_empty(pages),
+ first_index, ==, lru_to_page(pages)->index);
+
+ i_size = i_size_read(mapping->host);
+ if (type == FSCACHE_READ_FOR_WRITE) {
+ loff_t new_size = extent->block_end;
+ new_size <<= PAGE_SHIFT;
+ if (new_size > i_size)
+ i_size = new_size;
+ }
+ eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ notes = fscache_shape_extent(req->cookie, extent, i_size, false);
+ req->dio_block_size = extent->dio_block_size;
+
+ start = cursor = extent->start;
+
+ /* Add pages to the pagecache. We keep the pages ref'd and locked
+ * until the read is complete. We may also need to add pages to both
+ * sides of the request to make it up to the cache allocation granule
+ * alignment and size.
+ *
+ * Note that it's possible for the file size to change whilst we're
+ * doing this, but we rely on the server returning less than we asked
+ * for if the file shrank. We also rely on this to deal with a partial
+ * page at the end of the file.
+ *
+ * If we're going to end up loading from the server and writing to the
+ * cache, we start by inserting blank pages before the first page being
+ * examined. If we can fetch from the cache or we're not going to
+ * write to the cache, it's unnecessary.
+ */
+ if (notes & FSCACHE_RHLP_NOTE_WRITE_TO_CACHE) {
+ req->write_to_cache = true;
+ while (cursor < first_index) {
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page)
+ goto nomem;
+ if (!PageUptodate(page)) {
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ continue;
+ }
+
+ /* There's an up-to-date page in the preface - just
+ * fetch the requested pages and skip saving to the
+ * cache.
+ */
+ notes |= FSCACHE_RHLP_NOTE_U2D_IN_PREFACE;
+ fscache_ignore_pages(mapping, start, cursor + 1);
+ req->write_to_cache = false;
+ start = cursor = first_index;
+ req->nr_pages = 0;
+ break;
+ }
+ page = NULL;
+ } else {
+ req->write_to_cache = false;
+ start = cursor = first_index;
+ req->nr_pages = 0;
+ }
+
+ switch (type) {
+ case FSCACHE_READ_FOR_WRITE:
+ /* We're doing a prefetch for a write on a single page. We get
+ * or create the requested page if we weren't given it and lock
+ * it.
+ */
+ notes |= FSCACHE_RHLP_NOTE_READ_FOR_WRITE;
+ if (*requested_page) {
+ _debug("prewrite req %lx", cursor);
+ page = *requested_page;
+ ret = -ERESTARTSYS;
+ if (lock_page_killable(page) < 0)
+ goto dont;
+ } else {
+ _debug("prewrite new %lx %lx", cursor, eof);
+ page = grab_cache_page_write_begin(mapping, first_index,
+ aop_flags);
+ if (!page)
+ goto nomem;
+ *requested_page = page;
+ }
+ get_page(page);
+ req->no_unlock_page = page;
+ req->nr_pages++;
+ cursor++;
+ page = NULL;
+ ret = 0;
+ break;
+
+ case FSCACHE_READ_LOCKED_PAGE:
+ /* We've got a single page preattached to the inode and locked.
+ * Get our own ref on it.
+ */
+ _debug("locked");
+ notes |= FSCACHE_RHLP_NOTE_READ_LOCKED_PAGE;
+ get_page(*requested_page);
+ req->nr_pages++;
+ cursor++;
+ ret = 0;
+ break;
+
+ case FSCACHE_READ_PAGE_LIST:
+ /* We've been given a contiguous list of pages to add. */
+ notes |= FSCACHE_RHLP_NOTE_READ_PAGE_LIST;
+ do {
+ _debug("given %lx", cursor);
+
+ page = lru_to_page(pages);
+ if (page->index != cursor) {
+ notes |= FSCACHE_RHLP_NOTE_LIST_NOTCONTIG;
+ break;
+ }
+
+ list_del(&page->lru);
+
+ ret = add_to_page_cache_lru(page, mapping, cursor,
+ readahead_gfp_mask(mapping));
+ switch (ret) {
+ case 0:
+ /* Add to the reading list */
+ req->nr_pages++;
+ cursor++;
+ page = NULL;
+ break;
+
+ case -EEXIST:
+ put_page(page);
+
+ _debug("conflict %lx %d", cursor, ret);
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page) {
+ notes |= FSCACHE_RHLP_NOTE_LIST_NOMEM;
+ goto stop;
+ }
+
+ if (PageUptodate(page)) {
+ unlock_page(page);
+ put_page(page); /* Avoid overwriting */
+ ret = 0;
+ notes |= FSCACHE_RHLP_NOTE_LIST_U2D;
+ goto stop;
+ }
+
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ break;
+
+ default:
+ _debug("add fail %lx %d", cursor, ret);
+ put_page(page);
+ page = NULL;
+ notes |= FSCACHE_RHLP_NOTE_LIST_ERROR;
+ goto stop;
+ }
+
+ /* Trim the fetch to the cache granularity so we don't
+ * get a chain-failure of blocks being unable to be
+ * used because the previous uncached read spilt over.
+ */
+ if ((notes & FSCACHE_RHLP_NOTE_U2D_IN_PREFACE) &&
+ cursor == extent->block_end)
+ break;
+
+ } while (!list_empty(pages) && cursor < extent->limit);
+ ret = 0;
+ break;
+
+ default:
+ BUG();
+ }
+
+ /* If we're going to be writing to the cache, insert pages after the
+ * requested block to make up the numbers.
+ */
+ if (req->write_to_cache) {
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_ADD;
+ trailer = cursor;
+ while (cursor < extent->limit) {
+ _debug("after %lx", cursor);
+ page = find_or_create_page(mapping, cursor,
+ readahead_gfp_mask(mapping));
+ if (!page) {
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_NOMEM;
+ goto stop;
+ }
+ if (PageUptodate(page)) {
+ unlock_page(page);
+ put_page(page); /* Avoid overwriting */
+ notes |= FSCACHE_RHLP_NOTE_TRAILER_U2D;
+ goto stop;
+ }
+
+ req->nr_pages++; /* Add to the reading list */
+ cursor++;
+ }
+ }
+
+stop:
+ _debug("have %u", req->nr_pages);
+ if (req->nr_pages == 0)
+ goto dont;
+
+ if (cursor <= first_index) {
+ _debug("v.short");
+ goto nomem_unlock; /* We wouldn't've included the first page */
+ }
+
+submit_anyway:
+ if (cursor < extent->block_end) {
+ /* The request is short of what we need to be able to cache the
+ * minimum cache block so discard the trailer.
+ */
+ _debug("short");
+ notes |= FSCACHE_RHLP_NOTE_UNDERSIZED;
+ req->write_to_cache = false;
+ if (trailer != ULONG_MAX) {
+ fscache_ignore_pages(mapping, trailer, cursor);
+ req->nr_pages -= cursor - trailer;
+ }
+ }
+
+ req->len = req->nr_pages * PAGE_SIZE;
+ req->pos = start;
+ req->pos <<= PAGE_SHIFT;
+
+ if (start >= eof) {
+ notes |= FSCACHE_RHLP_NOTE_AFTER_EOF;
+ what = fscache_read_helper_skip;
+ } else if (notes & FSCACHE_RHLP_NOTE_FILL_WITH_ZERO) {
+ what = fscache_read_helper_zero;
+ } else if (notes & FSCACHE_RHLP_NOTE_READ_FROM_CACHE) {
+ what = fscache_read_helper_read;
+ } else {
+ what = fscache_read_helper_download;
+ }
+
+ trace_fscache_read_helper(req->cookie, start, start + req->nr_pages,
+ notes, what);
+
+ switch (what) {
+ case fscache_read_helper_skip:
+ /* The read is entirely beyond the end of the file, so skip the
+ * actual operation and let the done handler deal with clearing
+ * the pages.
+ */
+ _debug("SKIP READ: %llu", req->len);
+ fscache_read_done(req);
+ break;
+ case fscache_read_helper_zero:
+ _debug("ZERO READ: %llu", req->len);
+ fscache_read_done(req);
+ break;
+ case fscache_read_helper_read:
+ req->io_done = fscache_file_read_maybe_reissue;
+ fscache_read_from_cache(req);
+ break;
+ case fscache_read_helper_download:
+ _debug("DOWNLOAD: %llu", req->len);
+ req->io_done = fscache_read_done;
+ fscache_read_from_server(req);
+ break;
+ default:
+ BUG();
+ }
+
+ _leave(" = 0");
+ return 0;
+
+nomem:
+ if (cursor > first_index)
+ goto submit_anyway;
+nomem_unlock:
+ fscache_ignore_pages(mapping, start, cursor);
+ ret = -ENOMEM;
+dont:
+ _leave(" = %d", ret);
+ return ret;
+}
+EXPORT_SYMBOL(fscache_read_helper);
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index b46df865911c..ad44fa4cd844 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -152,7 +152,16 @@ struct fscache_io_request {
unsigned long flags;
#define FSCACHE_IO_DATA_FROM_SERVER 0 /* Set if data was read from server */
#define FSCACHE_IO_DATA_FROM_CACHE 1 /* Set if data was read from the cache */
+#define FSCACHE_IO_DONT_UNLOCK_PAGES 2 /* Don't unlock the pages on completion */
void (*io_done)(struct fscache_io_request *);
+ struct work_struct work;
+
+ /* Bits for readpages helper */
+ struct address_space *mapping; /* The mapping being accessed */
+ unsigned int nr_pages; /* Number of pages involved in the I/O */
+ unsigned int dio_block_size; /* Rounding for direct I/O in the cache */
+ bool write_to_cache; /* T if should write to the cache */
+ struct page *no_unlock_page; /* Don't unlock this page after read */
};
struct fscache_io_request_ops {
@@ -571,4 +580,17 @@ int fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
return -ENOBUFS;
}
+enum fscache_read_type {
+ FSCACHE_READ_PAGE_LIST, /* Read the list of pages (readpages) */
+ FSCACHE_READ_LOCKED_PAGE, /* requested_page is added and locked */
+ FSCACHE_READ_FOR_WRITE, /* This read is a prelude to write_begin */
+};
+
+extern int fscache_read_helper(struct fscache_io_request *,
+ struct fscache_extent *,
+ struct page **,
+ struct list_head *,
+ enum fscache_read_type,
+ unsigned int);
+
#endif /* _LINUX_FSCACHE_H */
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 4ce81f3f4c5b..e78a3fa9b633 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -39,6 +39,13 @@ enum fscache_cookie_trace {
fscache_cookie_see_discard,
};
+enum fscache_read_helper_trace {
+ fscache_read_helper_download,
+ fscache_read_helper_read,
+ fscache_read_helper_skip,
+ fscache_read_helper_zero,
+};
+
#endif
/*
@@ -63,6 +70,13 @@ enum fscache_cookie_trace {
EM(fscache_cookie_put_work, "PUT work ") \
E_(fscache_cookie_see_discard, "SEE discd")
+#define fscache_read_helper_traces \
+ EM(fscache_read_helper_download, "DOWN") \
+ EM(fscache_read_helper_read, "READ") \
+ EM(fscache_read_helper_skip, "SKIP") \
+ E_(fscache_read_helper_zero, "ZERO")
+
+
/*
* Export enum symbols via userspace.
*/
@@ -72,6 +86,7 @@ enum fscache_cookie_trace {
#define E_(a, b) TRACE_DEFINE_ENUM(a);
fscache_cookie_traces;
+fscache_read_helper_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -198,6 +213,35 @@ TRACE_EVENT(fscache_relinquish,
__entry->flags, __entry->retire)
);
+TRACE_EVENT(fscache_read_helper,
+ TP_PROTO(struct fscache_cookie *cookie, pgoff_t start, pgoff_t end,
+ unsigned int notes, enum fscache_read_helper_trace what),
+
+ TP_ARGS(cookie, start, end, notes, what),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, cookie )
+ __field(pgoff_t, start )
+ __field(pgoff_t, end )
+ __field(unsigned int, notes )
+ __field(enum fscache_read_helper_trace, what )
+ ),
+
+ TP_fast_assign(
+ __entry->cookie = cookie->debug_id;
+ __entry->start = start;
+ __entry->end = end;
+ __entry->what = what;
+ __entry->notes = notes;
+ ),
+
+ TP_printk("c=%08x %s n=%08x p=%lx-%lx",
+ __entry->cookie,
+ __print_symbolic(__entry->what, fscache_read_helper_traces),
+ __entry->notes,
+ __entry->start, __entry->end)
+ );
+
#endif /* _TRACE_FSCACHE_H */
/* This part must be outside protection */
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 39/61] fscache: Display cache-specific data in /proc/fs/fscache/objects
@ 2020-05-04 17:13 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Allow the cache to add information in /proc/fs/fscache/objects instead of
displaying cookie key and aux data - which can be seen in the cookies file.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/content-map.c | 41 +++++++++++++++++++++++++++++++++++++++++
fs/cachefiles/interface.c | 1 +
fs/cachefiles/internal.h | 1 +
fs/fscache/object-list.c | 33 +++------------------------------
include/linux/fscache-cache.h | 4 ++++
5 files changed, 50 insertions(+), 30 deletions(-)
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
index dea28948f006..02236b24f914 100644
--- a/fs/cachefiles/content-map.c
+++ b/fs/cachefiles/content-map.c
@@ -408,3 +408,44 @@ void cachefiles_save_content_map(struct cachefiles_object *object)
_leave(" = %zd", ret);
}
+
+/*
+ * Display object information in proc.
+ */
+int cachefiles_display_object(struct seq_file *m, struct fscache_object *_object)
+{
+ struct cachefiles_object *object =
+ container_of(_object, struct cachefiles_object, fscache);
+
+ if (object->fscache.cookie->type == FSCACHE_COOKIE_TYPE_INDEX) {
+ if (object->content_info != CACHEFILES_CONTENT_NO_DATA)
+ seq_printf(m, " ???%u???", object->content_info);
+ } else {
+ switch (object->content_info) {
+ case CACHEFILES_CONTENT_NO_DATA:
+ seq_puts(m, " <n>");
+ break;
+ case CACHEFILES_CONTENT_SINGLE:
+ seq_puts(m, " <s>");
+ break;
+ case CACHEFILES_CONTENT_ALL:
+ seq_puts(m, " <a>");
+ break;
+ case CACHEFILES_CONTENT_MAP:
+ read_lock_bh(&object->content_map_lock);
+ if (object->content_map) {
+ seq_printf(m, " %*phN",
+ object->content_map_size,
+ object->content_map);
+ }
+ read_unlock_bh(&object->content_map_lock);
+ break;
+ default:
+ seq_printf(m, " <%u>", object->content_info);
+ break;
+ }
+ }
+
+ seq_putc(m, '\n');
+ return 0;
+}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index feff39dff0f5..4fcbd788d3b2 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -495,4 +495,5 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.shape_extent = cachefiles_shape_extent,
.read = cachefiles_read,
.write = cachefiles_write,
+ .display_object = cachefiles_display_object,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 43f8e71136dd..32cb55319a7d 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -133,6 +133,7 @@ extern void cachefiles_expand_content_map(struct cachefiles_object *object, loff
extern void cachefiles_shorten_content_map(struct cachefiles_object *object, loff_t new_size);
extern bool cachefiles_load_content_map(struct cachefiles_object *object);
extern void cachefiles_save_content_map(struct cachefiles_object *object);
+extern int cachefiles_display_object(struct seq_file *m, struct fscache_object *object);
/*
* daemon.c
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 5777f909d31a..361610e124bd 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -155,7 +155,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
struct fscache_cookie *cookie;
unsigned long config = data->config;
char _type[3], *type;
- u8 *p;
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT USE CHLDN OPS FL S"
@@ -201,8 +200,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
obj->stage);
if (obj->cookie) {
- uint16_t keylen = 0, auxlen = 0;
-
switch (cookie->type) {
case 0:
type = "IX";
@@ -211,8 +208,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
type = "DT";
break;
default:
- snprintf(_type, sizeof(_type), "%02u",
- cookie->type);
+ snprintf(_type, sizeof(_type), "%02x", cookie->type);
type = _type;
break;
}
@@ -223,34 +219,11 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
type,
cookie->stage,
cookie->flags);
-
- if (config & FSCACHE_OBJLIST_CONFIG_KEY)
- keylen = cookie->key_len;
-
- if (config & FSCACHE_OBJLIST_CONFIG_AUX)
- auxlen = cookie->aux_len;
-
- if (keylen > 0 || auxlen > 0) {
- seq_puts(m, " ");
- p = keylen <= sizeof(cookie->inline_key) ?
- cookie->inline_key : cookie->key;
- for (; keylen > 0; keylen--)
- seq_printf(m, "%02x", *p++);
- if (auxlen > 0) {
- if (config & FSCACHE_OBJLIST_CONFIG_KEY)
- seq_puts(m, ", ");
- p = auxlen <= sizeof(cookie->inline_aux) ?
- cookie->inline_aux : cookie->aux;
- for (; auxlen > 0; auxlen--)
- seq_printf(m, "%02x", *p++);
- }
- }
-
- seq_puts(m, "\n");
} else {
seq_puts(m, "<no_netfs>\n");
}
- return 0;
+
+ return obj->cache->ops->display_object(m, obj);
}
static const struct seq_operations fscache_objlist_ops = {
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 1d235072239d..f82c998917e0 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -19,6 +19,7 @@
#define NR_MAXCACHES BITS_PER_LONG
+struct seq_file;
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
@@ -152,6 +153,9 @@ struct fscache_cache_ops {
int (*write)(struct fscache_object *object,
struct fscache_io_request *req,
struct iov_iter *iter);
+
+ /* Display object info in /proc/fs/fscache/objects */
+ int (*display_object)(struct seq_file *m, struct fscache_object *object);
};
extern struct fscache_cookie fscache_fsdef_index;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 39/61] fscache: Display cache-specific data in /proc/fs/fscache/objects
@ 2020-05-04 17:13 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Allow the cache to add information in /proc/fs/fscache/objects instead of
displaying cookie key and aux data - which can be seen in the cookies file.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/content-map.c | 41 +++++++++++++++++++++++++++++++++++++++++
fs/cachefiles/interface.c | 1 +
fs/cachefiles/internal.h | 1 +
fs/fscache/object-list.c | 33 +++------------------------------
include/linux/fscache-cache.h | 4 ++++
5 files changed, 50 insertions(+), 30 deletions(-)
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
index dea28948f006..02236b24f914 100644
--- a/fs/cachefiles/content-map.c
+++ b/fs/cachefiles/content-map.c
@@ -408,3 +408,44 @@ void cachefiles_save_content_map(struct cachefiles_object *object)
_leave(" = %zd", ret);
}
+
+/*
+ * Display object information in proc.
+ */
+int cachefiles_display_object(struct seq_file *m, struct fscache_object *_object)
+{
+ struct cachefiles_object *object =
+ container_of(_object, struct cachefiles_object, fscache);
+
+ if (object->fscache.cookie->type == FSCACHE_COOKIE_TYPE_INDEX) {
+ if (object->content_info != CACHEFILES_CONTENT_NO_DATA)
+ seq_printf(m, " ???%u???", object->content_info);
+ } else {
+ switch (object->content_info) {
+ case CACHEFILES_CONTENT_NO_DATA:
+ seq_puts(m, " <n>");
+ break;
+ case CACHEFILES_CONTENT_SINGLE:
+ seq_puts(m, " <s>");
+ break;
+ case CACHEFILES_CONTENT_ALL:
+ seq_puts(m, " <a>");
+ break;
+ case CACHEFILES_CONTENT_MAP:
+ read_lock_bh(&object->content_map_lock);
+ if (object->content_map) {
+ seq_printf(m, " %*phN",
+ object->content_map_size,
+ object->content_map);
+ }
+ read_unlock_bh(&object->content_map_lock);
+ break;
+ default:
+ seq_printf(m, " <%u>", object->content_info);
+ break;
+ }
+ }
+
+ seq_putc(m, '\n');
+ return 0;
+}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index feff39dff0f5..4fcbd788d3b2 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -495,4 +495,5 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.shape_extent = cachefiles_shape_extent,
.read = cachefiles_read,
.write = cachefiles_write,
+ .display_object = cachefiles_display_object,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 43f8e71136dd..32cb55319a7d 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -133,6 +133,7 @@ extern void cachefiles_expand_content_map(struct cachefiles_object *object, loff
extern void cachefiles_shorten_content_map(struct cachefiles_object *object, loff_t new_size);
extern bool cachefiles_load_content_map(struct cachefiles_object *object);
extern void cachefiles_save_content_map(struct cachefiles_object *object);
+extern int cachefiles_display_object(struct seq_file *m, struct fscache_object *object);
/*
* daemon.c
diff --git a/fs/fscache/object-list.c b/fs/fscache/object-list.c
index 5777f909d31a..361610e124bd 100644
--- a/fs/fscache/object-list.c
+++ b/fs/fscache/object-list.c
@@ -155,7 +155,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
struct fscache_cookie *cookie;
unsigned long config = data->config;
char _type[3], *type;
- u8 *p;
if ((unsigned long) v == 1) {
seq_puts(m, "OBJECT PARENT USE CHLDN OPS FL S"
@@ -201,8 +200,6 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
obj->stage);
if (obj->cookie) {
- uint16_t keylen = 0, auxlen = 0;
-
switch (cookie->type) {
case 0:
type = "IX";
@@ -211,8 +208,7 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
type = "DT";
break;
default:
- snprintf(_type, sizeof(_type), "%02u",
- cookie->type);
+ snprintf(_type, sizeof(_type), "%02x", cookie->type);
type = _type;
break;
}
@@ -223,34 +219,11 @@ static int fscache_objlist_show(struct seq_file *m, void *v)
type,
cookie->stage,
cookie->flags);
-
- if (config & FSCACHE_OBJLIST_CONFIG_KEY)
- keylen = cookie->key_len;
-
- if (config & FSCACHE_OBJLIST_CONFIG_AUX)
- auxlen = cookie->aux_len;
-
- if (keylen > 0 || auxlen > 0) {
- seq_puts(m, " ");
- p = keylen <= sizeof(cookie->inline_key) ?
- cookie->inline_key : cookie->key;
- for (; keylen > 0; keylen--)
- seq_printf(m, "%02x", *p++);
- if (auxlen > 0) {
- if (config & FSCACHE_OBJLIST_CONFIG_KEY)
- seq_puts(m, ", ");
- p = auxlen <= sizeof(cookie->inline_aux) ?
- cookie->inline_aux : cookie->aux;
- for (; auxlen > 0; auxlen--)
- seq_printf(m, "%02x", *p++);
- }
- }
-
- seq_puts(m, "\n");
} else {
seq_puts(m, "<no_netfs>\n");
}
- return 0;
+
+ return obj->cache->ops->display_object(m, obj);
}
static const struct seq_operations fscache_objlist_ops = {
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 1d235072239d..f82c998917e0 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -19,6 +19,7 @@
#define NR_MAXCACHES BITS_PER_LONG
+struct seq_file;
struct fscache_cache;
struct fscache_cache_ops;
struct fscache_object;
@@ -152,6 +153,9 @@ struct fscache_cache_ops {
int (*write)(struct fscache_object *object,
struct fscache_io_request *req,
struct iov_iter *iter);
+
+ /* Display object info in /proc/fs/fscache/objects */
+ int (*display_object)(struct seq_file *m, struct fscache_object *object);
};
extern struct fscache_cookie fscache_fsdef_index;
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 40/61] fscache: Remove more obsolete stats
2020-05-04 17:07 ` David Howells
` (39 preceding siblings ...)
(?)
@ 2020-05-04 17:13 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove some more stats that have become obsolete.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/internal.h | 18 ++----------------
fs/fscache/obj.c | 6 +++---
fs/fscache/stats.c | 50 +++++++++----------------------------------------
3 files changed, 14 insertions(+), 60 deletions(-)
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index a70c1a612309..d168e37011af 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -163,25 +163,13 @@ extern void fscache_proc_cleanup(void);
* stats.c
*/
#ifdef CONFIG_FSCACHE_STATS
-extern atomic_t fscache_n_op_pend;
-extern atomic_t fscache_n_op_run;
-extern atomic_t fscache_n_op_enqueue;
-extern atomic_t fscache_n_op_deferred_release;
-extern atomic_t fscache_n_op_initialised;
-extern atomic_t fscache_n_op_release;
-extern atomic_t fscache_n_op_gc;
-extern atomic_t fscache_n_op_cancelled;
-extern atomic_t fscache_n_op_rejected;
-
extern atomic_t fscache_n_acquires;
extern atomic_t fscache_n_acquires_null;
extern atomic_t fscache_n_acquires_no_cache;
extern atomic_t fscache_n_acquires_ok;
-extern atomic_t fscache_n_acquires_nobufs;
extern atomic_t fscache_n_acquires_oom;
extern atomic_t fscache_n_invalidates;
-extern atomic_t fscache_n_invalidates_run;
extern atomic_t fscache_n_updates;
extern atomic_t fscache_n_updates_null;
@@ -200,15 +188,13 @@ extern atomic_t fscache_n_object_no_alloc;
extern atomic_t fscache_n_object_lookups;
extern atomic_t fscache_n_object_lookups_negative;
extern atomic_t fscache_n_object_lookups_positive;
-extern atomic_t fscache_n_object_lookups_timed_out;
-extern atomic_t fscache_n_object_created;
+extern atomic_t fscache_n_object_creates;
extern atomic_t fscache_n_object_avail;
extern atomic_t fscache_n_object_dead;
extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
-extern atomic_t fscache_n_cop_lookup_complete;
-extern atomic_t fscache_n_cop_grab_object;
+extern atomic_t fscache_n_cop_create_object;
extern atomic_t fscache_n_cop_invalidate_object;
extern atomic_t fscache_n_cop_update_object;
extern atomic_t fscache_n_cop_drop_object;
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
index 1bf2bee00d18..7d8297f4d6a2 100644
--- a/fs/fscache/obj.c
+++ b/fs/fscache/obj.c
@@ -47,10 +47,10 @@ static int fscache_do_lookup_object(struct fscache_object *object, void *data)
static int fscache_do_create_object(struct fscache_object *object, void *data)
{
int ret;
- fscache_stat(&fscache_n_object_lookups);
- fscache_stat(&fscache_n_cop_lookup_object);
+ fscache_stat(&fscache_n_object_creates);
+ fscache_stat(&fscache_n_cop_create_object);
ret = object->cache->ops->create_object(object, data);
- fscache_stat_d(&fscache_n_cop_lookup_object);
+ fscache_stat_d(&fscache_n_cop_create_object);
return ret;
}
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 583817f4f113..ccca0016fd26 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -14,25 +14,13 @@
/*
* operation counters
*/
-atomic_t fscache_n_op_pend;
-atomic_t fscache_n_op_run;
-atomic_t fscache_n_op_enqueue;
-atomic_t fscache_n_op_deferred_release;
-atomic_t fscache_n_op_initialised;
-atomic_t fscache_n_op_release;
-atomic_t fscache_n_op_gc;
-atomic_t fscache_n_op_cancelled;
-atomic_t fscache_n_op_rejected;
-
atomic_t fscache_n_acquires;
atomic_t fscache_n_acquires_null;
atomic_t fscache_n_acquires_no_cache;
atomic_t fscache_n_acquires_ok;
-atomic_t fscache_n_acquires_nobufs;
atomic_t fscache_n_acquires_oom;
atomic_t fscache_n_invalidates;
-atomic_t fscache_n_invalidates_run;
atomic_t fscache_n_updates;
atomic_t fscache_n_updates_null;
@@ -51,15 +39,13 @@ atomic_t fscache_n_object_no_alloc;
atomic_t fscache_n_object_lookups;
atomic_t fscache_n_object_lookups_negative;
atomic_t fscache_n_object_lookups_positive;
-atomic_t fscache_n_object_lookups_timed_out;
-atomic_t fscache_n_object_created;
+atomic_t fscache_n_object_creates;
atomic_t fscache_n_object_avail;
atomic_t fscache_n_object_dead;
atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
-atomic_t fscache_n_cop_lookup_complete;
-atomic_t fscache_n_cop_grab_object;
+atomic_t fscache_n_cop_create_object;
atomic_t fscache_n_cop_invalidate_object;
atomic_t fscache_n_cop_update_object;
atomic_t fscache_n_cop_drop_object;
@@ -90,25 +76,21 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_object_avail),
atomic_read(&fscache_n_object_dead));
- seq_printf(m, "Acquire: n=%u nul=%u noc=%u ok=%u nbf=%u"
- " oom=%u\n",
+ seq_printf(m, "Acquire: n=%u nul=%u noc=%u ok=%u oom=%u\n",
atomic_read(&fscache_n_acquires),
atomic_read(&fscache_n_acquires_null),
atomic_read(&fscache_n_acquires_no_cache),
atomic_read(&fscache_n_acquires_ok),
- atomic_read(&fscache_n_acquires_nobufs),
atomic_read(&fscache_n_acquires_oom));
- seq_printf(m, "Lookups: n=%u neg=%u pos=%u crt=%u tmo=%u\n",
+ seq_printf(m, "Lookups: n=%u neg=%u pos=%u crt=%u\n",
atomic_read(&fscache_n_object_lookups),
atomic_read(&fscache_n_object_lookups_negative),
atomic_read(&fscache_n_object_lookups_positive),
- atomic_read(&fscache_n_object_created),
- atomic_read(&fscache_n_object_lookups_timed_out));
+ atomic_read(&fscache_n_object_creates));
- seq_printf(m, "Invals : n=%u run=%u\n",
- atomic_read(&fscache_n_invalidates),
- atomic_read(&fscache_n_invalidates_run));
+ seq_printf(m, "Invals : n=%u\n",
+ atomic_read(&fscache_n_invalidates));
seq_printf(m, "Updates: n=%u nul=%u run=%u\n",
atomic_read(&fscache_n_updates),
@@ -120,23 +102,9 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_relinquishes_null),
atomic_read(&fscache_n_relinquishes_retire));
- seq_printf(m, "Ops : pend=%u run=%u enq=%u can=%u rej=%u\n",
- atomic_read(&fscache_n_op_pend),
- atomic_read(&fscache_n_op_run),
- atomic_read(&fscache_n_op_enqueue),
- atomic_read(&fscache_n_op_cancelled),
- atomic_read(&fscache_n_op_rejected));
- seq_printf(m, "Ops : ini=%u dfr=%u rel=%u gc=%u\n",
- atomic_read(&fscache_n_op_initialised),
- atomic_read(&fscache_n_op_deferred_release),
- atomic_read(&fscache_n_op_release),
- atomic_read(&fscache_n_op_gc));
-
- seq_printf(m, "CacheOp: alo=%d luo=%d luc=%d gro=%d\n",
+ seq_printf(m, "CacheOp: alo=%d luo=%d\n",
atomic_read(&fscache_n_cop_alloc_object),
- atomic_read(&fscache_n_cop_lookup_object),
- atomic_read(&fscache_n_cop_lookup_complete),
- atomic_read(&fscache_n_cop_grab_object));
+ atomic_read(&fscache_n_cop_lookup_object));
seq_printf(m, "CacheOp: inv=%d upo=%d dro=%d pto=%d atc=%d syn=%d\n",
atomic_read(&fscache_n_cop_invalidate_object),
atomic_read(&fscache_n_cop_update_object),
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 41/61] fscache: New stats
2020-05-04 17:07 ` David Howells
` (40 preceding siblings ...)
(?)
@ 2020-05-04 17:13 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Create some new stat counters appropriate to the new routines and display them
in /proc/fs/fscache/stats.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/fscache/dispatcher.c | 6 +++++
fs/fscache/internal.h | 25 +++++++++++++++++++++
fs/fscache/io.c | 2 ++
fs/fscache/read_helper.c | 38 +++++++++++++++++++++++++++++---
fs/fscache/stats.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 123 insertions(+), 3 deletions(-)
diff --git a/fs/fscache/dispatcher.c b/fs/fscache/dispatcher.c
index fba71b99c951..489b8ab8cccd 100644
--- a/fs/fscache/dispatcher.c
+++ b/fs/fscache/dispatcher.c
@@ -41,6 +41,8 @@ void fscache_dispatch(struct fscache_cookie *cookie,
struct fscache_work *work;
bool queued = false;
+ fscache_stat(&fscache_n_dispatch_count);
+
work = kzalloc(sizeof(struct fscache_work), GFP_KERNEL);
if (work) {
work->cookie = cookie;
@@ -57,10 +59,13 @@ void fscache_dispatch(struct fscache_cookie *cookie,
queued = true;
}
spin_unlock(&fscache_work_lock);
+ if (queued)
+ fscache_stat(&fscache_n_dispatch_deferred);
}
if (!queued) {
kfree(work);
+ fscache_stat(&fscache_n_dispatch_inline);
func(cookie, object, param);
}
}
@@ -86,6 +91,7 @@ static int fscache_dispatcher(void *data)
if (work) {
work->func(work->cookie, work->object, work->param);
+ fscache_stat(&fscache_n_dispatch_in_pool);
fscache_cookie_put(work->cookie, fscache_cookie_put_work);
kfree(work);
}
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index d168e37011af..b9cad60e3c4e 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -207,6 +207,31 @@ extern atomic_t fscache_n_cache_stale_objects;
extern atomic_t fscache_n_cache_retired_objects;
extern atomic_t fscache_n_cache_culled_objects;
+extern atomic_t fscache_n_dispatch_count;
+extern atomic_t fscache_n_dispatch_deferred;
+extern atomic_t fscache_n_dispatch_inline;
+extern atomic_t fscache_n_dispatch_in_pool;
+
+extern atomic_t fscache_n_read;
+extern atomic_t fscache_n_write;
+
+extern atomic_t fscache_n_read_helper;
+extern atomic_t fscache_n_read_helper_stop_nomem;
+extern atomic_t fscache_n_read_helper_stop_noncontig;
+extern atomic_t fscache_n_read_helper_stop_uptodate;
+extern atomic_t fscache_n_read_helper_stop_exist;
+extern atomic_t fscache_n_read_helper_stop_kill;
+extern atomic_t fscache_n_read_helper_read;
+extern atomic_t fscache_n_read_helper_download;
+extern atomic_t fscache_n_read_helper_zero;
+extern atomic_t fscache_n_read_helper_beyond_eof;
+extern atomic_t fscache_n_read_helper_reissue;
+extern atomic_t fscache_n_read_helper_read_done;
+extern atomic_t fscache_n_read_helper_read_failed;
+extern atomic_t fscache_n_read_helper_copy;
+extern atomic_t fscache_n_read_helper_copy_done;
+extern atomic_t fscache_n_read_helper_copy_failed;
+
static inline void fscache_stat(atomic_t *stat)
{
atomic_inc(stat);
diff --git a/fs/fscache/io.c b/fs/fscache/io.c
index 0cea98bbb8ad..66005c9d2d99 100644
--- a/fs/fscache/io.c
+++ b/fs/fscache/io.c
@@ -141,6 +141,7 @@ int __fscache_read(struct fscache_io_request *req, struct iov_iter *iter)
fscache_begin_io_operation(req->cookie, FSCACHE_WANT_READ, req);
if (!IS_ERR(object)) {
+ fscache_stat(&fscache_n_read);
req->object = object;
return object->cache->ops->read(object, req, iter);
} else {
@@ -161,6 +162,7 @@ int __fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
fscache_begin_io_operation(req->cookie, FSCACHE_WANT_WRITE, req);
if (!IS_ERR(object)) {
+ fscache_stat(&fscache_n_write);
req->object = object;
return object->cache->ops->write(object, req, iter);
} else {
diff --git a/fs/fscache/read_helper.c b/fs/fscache/read_helper.c
index 9ed6e76ef255..b36ad2a0b749 100644
--- a/fs/fscache/read_helper.c
+++ b/fs/fscache/read_helper.c
@@ -32,6 +32,11 @@ static void fscache_read_copy_done(struct fscache_io_request *req)
_enter("%lx,%x,%llx", index, req->nr_pages, req->transferred);
+ if (req->error == 0)
+ fscache_stat(&fscache_n_read_helper_copy_done);
+ else
+ fscache_stat(&fscache_n_read_helper_copy_failed);
+
/* Clear PG_fscache on the pages that were being written out. */
rcu_read_lock();
xas_for_each(&xas, page, last) {
@@ -54,6 +59,8 @@ static void fscache_do_read_copy_to_cache(struct work_struct *work)
_enter("");
+ fscache_stat(&fscache_n_read_helper_copy);
+
iov_iter_mapping(&iter, WRITE, req->mapping, req->pos,
round_up(req->len, req->dio_block_size));
@@ -106,6 +113,11 @@ static void fscache_read_done(struct fscache_io_request *req)
_enter("%lx,%x,%llx,%d",
start, req->nr_pages, req->transferred, req->error);
+ if (req->error == 0)
+ fscache_stat(&fscache_n_read_helper_read_done);
+ else
+ fscache_stat(&fscache_n_read_helper_read_failed);
+
if (req->transferred < req->len)
fscache_clear_unread(req);
@@ -157,6 +169,7 @@ static void fscache_file_read_maybe_reissue(struct fscache_io_request *req)
if (req->error == 0) {
fscache_read_done(req);
} else {
+ fscache_stat(&fscache_n_read_helper_reissue);
INIT_WORK(&req->work, fscache_reissue_read);
fscache_get_io_request(req);
queue_work(fscache_op_wq, &req->work);
@@ -255,6 +268,8 @@ int fscache_read_helper(struct fscache_io_request *req,
loff_t i_size;
int ret;
+ fscache_stat(&fscache_n_read_helper);
+
first_index = extent->start;
_enter("{%lx,%lx}", first_index, extent->limit);
@@ -300,8 +315,10 @@ int fscache_read_helper(struct fscache_io_request *req,
while (cursor < first_index) {
page = find_or_create_page(mapping, cursor,
readahead_gfp_mask(mapping));
- if (!page)
+ if (!page) {
+ fscache_stat(&fscache_n_read_helper_stop_nomem);
goto nomem;
+ }
if (!PageUptodate(page)) {
req->nr_pages++; /* Add to the reading list */
cursor++;
@@ -313,6 +330,7 @@ int fscache_read_helper(struct fscache_io_request *req,
* cache.
*/
notes |= FSCACHE_RHLP_NOTE_U2D_IN_PREFACE;
+ fscache_stat(&fscache_n_read_helper_stop_uptodate);
fscache_ignore_pages(mapping, start, cursor + 1);
req->write_to_cache = false;
start = cursor = first_index;
@@ -337,14 +355,18 @@ int fscache_read_helper(struct fscache_io_request *req,
_debug("prewrite req %lx", cursor);
page = *requested_page;
ret = -ERESTARTSYS;
- if (lock_page_killable(page) < 0)
+ if (lock_page_killable(page) < 0) {
+ fscache_stat(&fscache_n_read_helper_stop_kill);
goto dont;
+ }
} else {
_debug("prewrite new %lx %lx", cursor, eof);
page = grab_cache_page_write_begin(mapping, first_index,
aop_flags);
- if (!page)
+ if (!page) {
+ fscache_stat(&fscache_n_read_helper_stop_nomem);
goto nomem;
+ }
*requested_page = page;
}
get_page(page);
@@ -376,6 +398,7 @@ int fscache_read_helper(struct fscache_io_request *req,
page = lru_to_page(pages);
if (page->index != cursor) {
notes |= FSCACHE_RHLP_NOTE_LIST_NOTCONTIG;
+ fscache_stat(&fscache_n_read_helper_stop_noncontig);
break;
}
@@ -399,12 +422,14 @@ int fscache_read_helper(struct fscache_io_request *req,
readahead_gfp_mask(mapping));
if (!page) {
notes |= FSCACHE_RHLP_NOTE_LIST_NOMEM;
+ fscache_stat(&fscache_n_read_helper_stop_nomem);
goto stop;
}
if (PageUptodate(page)) {
unlock_page(page);
put_page(page); /* Avoid overwriting */
+ fscache_stat(&fscache_n_read_helper_stop_exist);
ret = 0;
notes |= FSCACHE_RHLP_NOTE_LIST_U2D;
goto stop;
@@ -417,6 +442,7 @@ int fscache_read_helper(struct fscache_io_request *req,
default:
_debug("add fail %lx %d", cursor, ret);
put_page(page);
+ fscache_stat(&fscache_n_read_helper_stop_nomem);
page = NULL;
notes |= FSCACHE_RHLP_NOTE_LIST_ERROR;
goto stop;
@@ -450,12 +476,14 @@ int fscache_read_helper(struct fscache_io_request *req,
readahead_gfp_mask(mapping));
if (!page) {
notes |= FSCACHE_RHLP_NOTE_TRAILER_NOMEM;
+ fscache_stat(&fscache_n_read_helper_stop_nomem);
goto stop;
}
if (PageUptodate(page)) {
unlock_page(page);
put_page(page); /* Avoid overwriting */
notes |= FSCACHE_RHLP_NOTE_TRAILER_U2D;
+ fscache_stat(&fscache_n_read_helper_stop_uptodate);
goto stop;
}
@@ -513,18 +541,22 @@ int fscache_read_helper(struct fscache_io_request *req,
* the pages.
*/
_debug("SKIP READ: %llu", req->len);
+ fscache_stat(&fscache_n_read_helper_beyond_eof);
fscache_read_done(req);
break;
case fscache_read_helper_zero:
_debug("ZERO READ: %llu", req->len);
+ fscache_stat(&fscache_n_read_helper_zero);
fscache_read_done(req);
break;
case fscache_read_helper_read:
+ fscache_stat(&fscache_n_read_helper_read);
req->io_done = fscache_file_read_maybe_reissue;
fscache_read_from_cache(req);
break;
case fscache_read_helper_download:
_debug("DOWNLOAD: %llu", req->len);
+ fscache_stat(&fscache_n_read_helper_download);
req->io_done = fscache_read_done;
fscache_read_from_server(req);
break;
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index ccca0016fd26..fdea31ff8e69 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -58,6 +58,31 @@ atomic_t fscache_n_cache_stale_objects;
atomic_t fscache_n_cache_retired_objects;
atomic_t fscache_n_cache_culled_objects;
+atomic_t fscache_n_dispatch_count;
+atomic_t fscache_n_dispatch_deferred;
+atomic_t fscache_n_dispatch_inline;
+atomic_t fscache_n_dispatch_in_pool;
+
+atomic_t fscache_n_read;
+atomic_t fscache_n_write;
+
+atomic_t fscache_n_read_helper;
+atomic_t fscache_n_read_helper_stop_nomem;
+atomic_t fscache_n_read_helper_stop_noncontig;
+atomic_t fscache_n_read_helper_stop_uptodate;
+atomic_t fscache_n_read_helper_stop_exist;
+atomic_t fscache_n_read_helper_stop_kill;
+atomic_t fscache_n_read_helper_read;
+atomic_t fscache_n_read_helper_download;
+atomic_t fscache_n_read_helper_zero;
+atomic_t fscache_n_read_helper_beyond_eof;
+atomic_t fscache_n_read_helper_reissue;
+atomic_t fscache_n_read_helper_read_done;
+atomic_t fscache_n_read_helper_read_failed;
+atomic_t fscache_n_read_helper_copy;
+atomic_t fscache_n_read_helper_copy_done;
+atomic_t fscache_n_read_helper_copy_failed;
+
/*
* display the general statistics
*/
@@ -117,5 +142,35 @@ int fscache_stats_show(struct seq_file *m, void *v)
atomic_read(&fscache_n_cache_stale_objects),
atomic_read(&fscache_n_cache_retired_objects),
atomic_read(&fscache_n_cache_culled_objects));
+
+ seq_printf(m, "Disp : n=%u il=%u df=%u pl=%u\n",
+ atomic_read(&fscache_n_dispatch_count),
+ atomic_read(&fscache_n_dispatch_inline),
+ atomic_read(&fscache_n_dispatch_deferred),
+ atomic_read(&fscache_n_dispatch_in_pool));
+
+ seq_printf(m, "IO : rd=%u wr=%u\n",
+ atomic_read(&fscache_n_read),
+ atomic_read(&fscache_n_write));
+
+ seq_printf(m, "RdHelp : nm=%u nc=%u ud=%u ex=%u kl=%u\n",
+ atomic_read(&fscache_n_read_helper_stop_nomem),
+ atomic_read(&fscache_n_read_helper_stop_noncontig),
+ atomic_read(&fscache_n_read_helper_stop_uptodate),
+ atomic_read(&fscache_n_read_helper_stop_exist),
+ atomic_read(&fscache_n_read_helper_stop_kill));
+ seq_printf(m, "RdHelp : n=%u rd=%u dl=%u zr=%u eo=%u\n",
+ atomic_read(&fscache_n_read_helper),
+ atomic_read(&fscache_n_read_helper_read),
+ atomic_read(&fscache_n_read_helper_download),
+ atomic_read(&fscache_n_read_helper_zero),
+ atomic_read(&fscache_n_read_helper_beyond_eof));
+ seq_printf(m, "RdHelp : ri=%u dn=%u fl=%u cp=%u cd=%u cf=%u\n",
+ atomic_read(&fscache_n_read_helper_reissue),
+ atomic_read(&fscache_n_read_helper_read_done),
+ atomic_read(&fscache_n_read_helper_read_failed),
+ atomic_read(&fscache_n_read_helper_copy),
+ atomic_read(&fscache_n_read_helper_copy_done),
+ atomic_read(&fscache_n_read_helper_copy_failed));
return 0;
}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 42/61] fscache, cachefiles: Rewrite invalidation
2020-05-04 17:07 ` David Howells
` (41 preceding siblings ...)
(?)
@ 2020-05-04 17:13 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Rewrite the cache object invalidation code in fscache and cachefiles. The
following changes are made to fscache:
(1) Invalidation is now ignored or allowed to proceed depending on the
'stage' a non-index cookie is in with respect to the backing object.
(2) If invalidation is proceeds, it pins the object and holds an operation
count for the duration.
(3) The fscache_object struct is given an invalidation counter that is
incremented any time fscache_invalidate() is called, even if the
cookie is at a stage in which it cannot be applied. The counter,
however, can be noted and applied retroactively later.
(4) The invalidation counter is noted in the operation struct when a cache
operation is begun and can be checked on operation completion to find
out if any consequent metadata changes should be dropped.
(5) New operations aren't allowed to proceed if the object is being
invalidated.
and to cachefiles:
(1) If an open object is invalidated, the open backing file is replaced
with a tmpfile (as if opened O_TMPFILE). This is held unlinked until
the object released from memory, at which point the file is simply
abandoned if it was retired or the old file is unlinked and the new
one linked into its place.
Note: This would be easier if linkat() could be given a flag to
indicate the destination should be overwritten or if RENAME_EXCHANGE
could be applied to tmpfiles, effectively unlinking the destination.
(2) Upon invalidation, the content map is replaced with a blank one.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/inode.c | 8 ++-
fs/cachefiles/content-map.c | 32 ++++++++++
fs/cachefiles/interface.c | 130 +++++++++++++++++++++++++++++++++--------
fs/cachefiles/internal.h | 9 ++-
fs/cachefiles/namei.c | 69 ++++++++++++++++++++--
fs/cachefiles/xattr.c | 6 +-
fs/fscache/cookie.c | 46 ++++++++++++---
fs/fscache/io.c | 2 +
fs/fscache/obj.c | 31 +++-------
include/linux/fscache-cache.h | 5 +-
include/linux/fscache.h | 13 +++-
11 files changed, 281 insertions(+), 70 deletions(-)
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 1f248181d47d..59dc179b40cb 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -577,7 +577,13 @@ static void afs_zap_data(struct afs_vnode *vnode)
_enter("{%llx:%llu}", vnode->fid.vid, vnode->fid.vnode);
#ifdef CONFIG_AFS_FSCACHE
- fscache_invalidate(vnode->cache, i_size_read(&vnode->vfs_inode));
+ {
+ struct afs_vnode_cache_aux aux = {
+ .data_version = vnode->status.data_version,
+ };
+ fscache_invalidate(afs_vnode_cache(vnode), &aux,
+ i_size_read(&vnode->vfs_inode), 0);
+ }
#endif
/* nuke all the non-dirty pages that aren't locked, mapped or being
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
index 02236b24f914..13632c097248 100644
--- a/fs/cachefiles/content-map.c
+++ b/fs/cachefiles/content-map.c
@@ -204,6 +204,34 @@ unsigned int cachefiles_shape_extent(struct fscache_object *obj,
return ret;
}
+/*
+ * Allocate a new content map.
+ */
+u8 *cachefiles_new_content_map(struct cachefiles_object *object,
+ unsigned int *_size)
+{
+ size_t size;
+ u8 *map = NULL;
+
+ _enter("");
+
+ if (!(object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK)) {
+ /* Single-chunk object. The presence or absence of the content
+ * map xattr is sufficient indication.
+ */
+ *_size = 0;
+ return NULL;
+ }
+
+ /* Granular object. */
+ size = cachefiles_map_size(object->fscache.cookie->object_size);
+ map = kzalloc(size, GFP_KERNEL);
+ if (!map)
+ return ERR_PTR(-ENOMEM);
+ *_size = size;
+ return map;
+}
+
/*
* Mark the content map to indicate stored granule.
*/
@@ -217,7 +245,9 @@ void cachefiles_mark_content_map(struct fscache_io_request *req)
read_lock_bh(&object->content_map_lock);
- if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK) {
+ if (req->inval_counter != object->fscache.inval_counter) {
+ _debug("inval mark");
+ } else if (object->fscache.cookie->advice & FSCACHE_ADV_SINGLE_CHUNK) {
if (pos == 0) {
object->content_info = CACHEFILES_CONTENT_SINGLE;
set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->fscache.flags);
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 4fcbd788d3b2..b312a04f672b 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -203,7 +203,7 @@ static void cachefiles_update_object(struct fscache_object *_object)
}
}
- cachefiles_set_object_xattr(object, XATTR_REPLACE);
+ cachefiles_set_object_xattr(object);
out:
cachefiles_end_secure(cache, saved_cred);
@@ -213,11 +213,15 @@ static void cachefiles_update_object(struct fscache_object *_object)
/*
* Commit changes to the object as we drop it.
*/
-static void cachefiles_commit_object(struct cachefiles_object *object,
+static bool cachefiles_commit_object(struct cachefiles_object *object,
struct cachefiles_cache *cache)
{
if (object->content_map_changed)
cachefiles_save_content_map(object);
+
+ if (test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags))
+ return cachefiles_commit_tmpfile(cache, object);
+ return true;
}
/*
@@ -428,47 +432,125 @@ static int cachefiles_attr_changed(struct cachefiles_object *object)
}
/*
- * Invalidate an object
+ * Create a temporary file and leave it unattached and un-xattr'd until the
+ * time comes to discard the object from memory.
*/
-static void cachefiles_invalidate_object(struct fscache_object *_object)
+static struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
{
- struct cachefiles_object *object;
struct cachefiles_cache *cache;
const struct cred *saved_cred;
+ struct file *file;
struct path path;
uint64_t ni_size;
- int ret;
+ long ret;
- object = container_of(_object, struct cachefiles_object, fscache);
cache = container_of(object->fscache.cache,
struct cachefiles_cache, cache);
ni_size = object->fscache.cookie->object_size;
ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
- _enter("{OBJ%x},[%llu]",
- object->fscache.debug_id, (unsigned long long)ni_size);
-
- if (object->dentry) {
- ASSERT(d_is_reg(object->dentry));
+ cachefiles_begin_secure(cache, &saved_cred);
- path.dentry = object->dentry;
- path.mnt = cache->mnt;
+ path.mnt = cache->mnt;
+ path.dentry = vfs_tmpfile(cache->graveyard, S_IFREG, O_RDWR);
+ if (IS_ERR(path.dentry)) {
+ if (PTR_ERR(path.dentry) == -EIO)
+ cachefiles_io_error_obj(object, "Failed to create tmpfile");
+ file = ERR_CAST(path.dentry);
+ goto out;
+ }
- cachefiles_begin_secure(cache, &saved_cred);
- ret = vfs_truncate(&path, 0);
- if (ret == 0)
- ret = vfs_truncate(&path, ni_size);
- cachefiles_end_secure(cache, saved_cred);
+ trace_cachefiles_tmpfile(object, d_inode(path.dentry));
- if (ret != 0) {
- if (ret == -EIO)
- cachefiles_io_error_obj(object,
- "Invalidate failed");
+ if (ni_size > 0) {
+ trace_cachefiles_trunc(object, d_inode(path.dentry), 0, ni_size);
+ ret = vfs_truncate(&path, ni_size);
+ if (ret < 0) {
+ file = ERR_PTR(ret);
+ goto out_dput;
}
}
- _leave("");
+ file = open_with_fake_path(&path,
+ O_RDWR | O_LARGEFILE | O_DIRECT,
+ d_backing_inode(path.dentry),
+ cache->cache_cred);
+out_dput:
+ dput(path.dentry);
+out:
+ cachefiles_end_secure(cache, saved_cred);
+ return file;
+}
+
+/*
+ * Invalidate an object
+ */
+static bool cachefiles_invalidate_object(struct fscache_object *_object,
+ unsigned int flags)
+{
+ struct cachefiles_object *object;
+ struct file *file, *old_file;
+ u8 *map, *old_map;
+ unsigned int map_size;
+
+ object = container_of(_object, struct cachefiles_object, fscache);
+
+ _enter("{OBJ%x},[%llu]",
+ object->fscache.debug_id, _object->cookie->object_size);
+
+ if ((flags & FSCACHE_INVAL_LIGHT) &&
+ test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags)) {
+ _leave(" = t [light]");
+ return true;
+ }
+
+ if (object->dentry) {
+ ASSERT(d_is_reg(object->dentry));
+ ASSERT(!object->old);
+
+ file = cachefiles_create_tmpfile(object);
+ if (IS_ERR(file))
+ goto failed;
+
+ map = cachefiles_new_content_map(object, &map_size);
+ if (IS_ERR(map))
+ goto failed_fput;
+
+ /* Substitute the VFS target */
+ _debug("sub");
+ dget(file->f_path.dentry); /* Do outside of content_map_lock */
+ spin_lock(&object->fscache.lock);
+ write_lock_bh(&object->content_map_lock);
+
+ object->old = object->dentry;
+ old_file = object->backing_file;
+ old_map = object->content_map;
+ object->backing_file = file;
+ object->dentry = file->f_path.dentry;
+ object->content_info = CACHEFILES_CONTENT_NO_DATA;
+ object->content_map = map;
+ object->content_map_size = map_size;
+ object->content_map_changed = true;
+ set_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags);
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->fscache.flags);
+
+ write_unlock_bh(&object->content_map_lock);
+ spin_unlock(&object->fscache.lock);
+ _debug("subbed");
+
+ kfree(old_map);
+ fput(old_file);
+ }
+
+ _leave(" = t [tmpfile]");
+ return true;
+
+failed_fput:
+ fput(file);
+failed:
+ _leave(" = f");
+ return false;
}
static unsigned int cachefiles_get_object_usage(const struct fscache_object *_object)
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 32cb55319a7d..ccc8cd2b8250 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -54,6 +54,8 @@ struct cachefiles_object {
struct file *backing_file; /* File open on backing storage */
loff_t i_size; /* object size */
atomic_t usage; /* object usage count */
+ unsigned long flags;
+#define CACHEFILES_OBJECT_USING_TMPFILE 0 /* Object has a tmpfile that need linking */
uint8_t type; /* object type */
bool new; /* T if object new */
@@ -128,6 +130,7 @@ extern void cachefiles_daemon_unbind(struct cachefiles_cache *cache);
extern unsigned int cachefiles_shape_extent(struct fscache_object *object,
struct fscache_extent *extent,
loff_t i_size, bool for_write);
+extern u8 *cachefiles_new_content_map(struct cachefiles_object *object, unsigned int *_size);
extern void cachefiles_mark_content_map(struct fscache_io_request *req);
extern void cachefiles_expand_content_map(struct cachefiles_object *object, loff_t size);
extern void cachefiles_shorten_content_map(struct cachefiles_object *object, loff_t new_size);
@@ -186,6 +189,9 @@ extern int cachefiles_cull(struct cachefiles_cache *cache, struct dentry *dir,
extern int cachefiles_check_in_use(struct cachefiles_cache *cache,
struct dentry *dir, char *filename);
+extern bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
+ struct cachefiles_object *object);
+
/*
* proc.c
*/
@@ -238,8 +244,7 @@ static inline void cachefiles_end_secure(struct cachefiles_cache *cache,
* xattr.c
*/
extern int cachefiles_check_object_type(struct cachefiles_object *object);
-extern int cachefiles_set_object_xattr(struct cachefiles_object *object,
- unsigned int xattr_flags);
+extern int cachefiles_set_object_xattr(struct cachefiles_object *object);
extern int cachefiles_check_auxdata(struct cachefiles_object *object);
extern int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct dentry *dentry);
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index d697a74436c0..ef615e20f159 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -468,7 +468,7 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
if (object->new) {
/* attach data to a newly constructed terminal object */
- ret = cachefiles_set_object_xattr(object, XATTR_CREATE);
+ ret = cachefiles_set_object_xattr(object);
if (ret < 0)
goto check_error;
} else {
@@ -487,8 +487,6 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
pr_warn("cachefiles: Block size too large\n");
goto check_error;
}
-
- object->old = dget(object->dentry);
} else {
BUG(); // TODO: open file in data-class subdir
}
@@ -523,9 +521,7 @@ bool cachefiles_walk_to_object(struct cachefiles_object *parent,
cachefiles_unmark_inode_in_use(object, object->dentry);
cachefiles_mark_object_inactive(cache, object);
dput(object->dentry);
- dput(object->old);
object->dentry = NULL;
- object->old = NULL;
goto error_out;
lookup_error:
@@ -811,3 +807,66 @@ int cachefiles_check_in_use(struct cachefiles_cache *cache, struct dentry *dir,
//_leave(" = 0");
return ret;
}
+
+/*
+ * Attempt to link a temporary file into its rightful place in the cache.
+ */
+bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
+ struct cachefiles_object *object)
+{
+ struct dentry *dir, *dentry, *old;
+ char *name;
+ unsigned int namelen;
+ bool success = false;
+ int ret;
+
+ _enter(",%pd", object->old);
+
+ namelen = object->old->d_name.len;
+ name = kmemdup_nul(object->old->d_name.name, namelen, GFP_KERNEL);
+ if (!name)
+ goto out;
+
+ dir = dget_parent(object->old);
+
+ inode_lock_nested(d_inode(dir), I_MUTEX_PARENT);
+ ret = cachefiles_bury_object(cache, object, dir, object->old,
+ FSCACHE_OBJECT_IS_STALE);
+ dput(object->old);
+ object->old = NULL;
+ if (ret < 0 && ret != -ENOENT) {
+ _debug("bury fail %d", ret);
+ goto out_name;
+ }
+
+ inode_lock_nested(d_inode(dir), I_MUTEX_PARENT);
+ dentry = lookup_one_len(name, dir, namelen);
+ if (IS_ERR(dentry)) {
+ _debug("lookup fail %ld", PTR_ERR(dentry));
+ goto out_unlock;
+ }
+
+ ret = vfs_link(object->dentry, d_inode(dir), dentry, NULL);
+ if (ret < 0) {
+ _debug("link fail %d", ret);
+ dput(dentry);
+ } else {
+ trace_cachefiles_link(object, d_inode(object->dentry));
+ spin_lock(&object->fscache.lock);
+ old = object->dentry;
+ object->dentry = dentry;
+ success = true;
+ clear_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags);
+ spin_unlock(&object->fscache.lock);
+ dput(old);
+ }
+
+out_unlock:
+ inode_unlock(d_inode(dir));
+out_name:
+ kfree(name);
+ dput(dir);
+out:
+ _leave(" = %u", success);
+ return success;
+}
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index a1d4a3d1db69..22c56ca2fd0b 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -104,8 +104,7 @@ int cachefiles_check_object_type(struct cachefiles_object *object)
/*
* set the state xattr on a cache file
*/
-int cachefiles_set_object_xattr(struct cachefiles_object *object,
- unsigned int xattr_flags)
+int cachefiles_set_object_xattr(struct cachefiles_object *object)
{
struct cachefiles_xattr *buf;
struct dentry *dentry = object->dentry;
@@ -129,8 +128,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object,
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
ret = vfs_setxattr(dentry, cachefiles_xattr_cache,
- buf, sizeof(struct cachefiles_xattr) + len,
- xattr_flags);
+ buf, sizeof(struct cachefiles_xattr) + len, 0);
if (ret < 0) {
trace_cachefiles_coherency(object, d_inode(dentry)->i_ino,
buf->content,
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index f02ed34beb0c..c0d93dc7f69f 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -472,10 +472,14 @@ void fscache_set_cookie_stage(struct fscache_cookie *cookie,
}
/*
- * Invalidate an object. Callable with spinlocks held.
+ * Invalidate an object.
*/
-void __fscache_invalidate(struct fscache_cookie *cookie, loff_t new_size)
+void __fscache_invalidate(struct fscache_cookie *cookie,
+ const void *aux_data, loff_t new_size,
+ unsigned int flags)
{
+ struct fscache_object *object = NULL;
+
_enter("{%s}", cookie->type_name);
fscache_stat(&fscache_n_invalidates);
@@ -488,13 +492,41 @@ void __fscache_invalidate(struct fscache_cookie *cookie, loff_t new_size)
ASSERTCMP(cookie->type, !=, FSCACHE_COOKIE_TYPE_INDEX);
spin_lock(&cookie->lock);
- cookie->object_size = new_size;
+ fscache_update_aux(cookie, aux_data, &new_size);
cookie->zero_point = new_size;
- spin_unlock(&cookie->lock);
- if (!hlist_empty(&cookie->backing_objects) &&
- test_and_set_bit(FSCACHE_COOKIE_INVALIDATING, &cookie->flags))
- fscache_dispatch(cookie, NULL, 0, fscache_invalidate_object);
+ if (!hlist_empty(&cookie->backing_objects)) {
+ object = hlist_entry(cookie->backing_objects.first,
+ struct fscache_object, cookie_link);
+ object->inval_counter++;
+ }
+
+ switch (cookie->stage) {
+ case FSCACHE_COOKIE_STAGE_QUIESCENT:
+ case FSCACHE_COOKIE_STAGE_DEAD:
+ case FSCACHE_COOKIE_STAGE_INITIALISING: /* Assume later checks will catch it */
+ case FSCACHE_COOKIE_STAGE_INVALIDATING: /* is_still_valid will catch it */
+ spin_unlock(&cookie->lock);
+ _leave(" [no %u]", cookie->stage);
+ return;
+
+ case FSCACHE_COOKIE_STAGE_LOOKING_UP:
+ _leave(" [look %x]", object->inval_counter);
+ return;
+
+ case FSCACHE_COOKIE_STAGE_NO_DATA_YET:
+ case FSCACHE_COOKIE_STAGE_ACTIVE:
+ cookie->stage = FSCACHE_COOKIE_STAGE_INVALIDATING;
+ wake_up_var(&cookie->stage);
+
+ atomic_inc(&cookie->n_ops);
+ object->cache->ops->grab_object(object, fscache_obj_get_inval);
+ spin_unlock(&cookie->lock);
+
+ fscache_dispatch(cookie, object, flags, fscache_invalidate_object);
+ _leave(" [inv]");
+ return;
+ }
}
EXPORT_SYMBOL(__fscache_invalidate);
diff --git a/fs/fscache/io.c b/fs/fscache/io.c
index 66005c9d2d99..b6ec15588334 100644
--- a/fs/fscache/io.c
+++ b/fs/fscache/io.c
@@ -84,6 +84,8 @@ static struct fscache_object *fscache_begin_io_operation(
goto not_live;
object->cache->ops->grab_object(object, fscache_obj_get_ioreq);
+ if (req)
+ req->inval_counter = object->inval_counter;
atomic_inc(&cookie->n_ops);
spin_unlock(&cookie->lock);
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
index 7d8297f4d6a2..aea4239641a9 100644
--- a/fs/fscache/obj.c
+++ b/fs/fscache/obj.c
@@ -237,32 +237,21 @@ void fscache_lookup_object(struct fscache_cookie *cookie,
}
/*
- * Invalidate an object
+ * Invalidate an object. param passes the invalidation flags.
*/
void fscache_invalidate_object(struct fscache_cookie *cookie,
- struct fscache_object *unused, int param)
+ struct fscache_object *object, int flags)
{
- struct fscache_object *object = NULL;
+ bool success;
- spin_lock(&cookie->lock);
-
- if (!hlist_empty(&cookie->backing_objects)) {
- object = hlist_entry(cookie->backing_objects.first,
- struct fscache_object,
- cookie_link);
- object = object->cache->ops->grab_object(object,
- fscache_obj_get_inval);
- }
-
- spin_unlock(&cookie->lock);
-
- if (object) {
- object->cache->ops->invalidate_object(object);
- fscache_do_put_object(object, fscache_obj_put_inval);
- }
+ success = object->cache->ops->invalidate_object(object, flags);
+ fscache_do_put_object(object, fscache_obj_put_inval);
- clear_bit_unlock(FSCACHE_COOKIE_INVALIDATING, &cookie->flags);
- wake_up_bit(&cookie->flags, FSCACHE_COOKIE_INVALIDATING);
+ if (success)
+ fscache_set_cookie_stage(cookie, FSCACHE_COOKIE_STAGE_NO_DATA_YET);
+ else
+ fscache_set_cookie_stage(cookie, FSCACHE_COOKIE_STAGE_DEAD);
+ fscache_end_io_operation(cookie);
}
/*
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index f82c998917e0..4fb63d8a60cd 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -120,7 +120,8 @@ struct fscache_cache_ops {
void (*update_object)(struct fscache_object *object);
/* Invalidate an object */
- void (*invalidate_object)(struct fscache_object *object);
+ bool (*invalidate_object)(struct fscache_object *object,
+ unsigned int flags);
/* discard the resources pinned by an object and effect retirement if
* necessary */
@@ -177,10 +178,12 @@ enum fscache_object_stage {
struct fscache_object {
int debug_id; /* debugging ID */
int n_children; /* number of child objects */
+ unsigned int inval_counter; /* Number of invalidations applied */
enum fscache_object_stage stage; /* Stage of object's lifecycle */
spinlock_t lock; /* state and operations lock */
unsigned long flags;
+#define FSCACHE_OBJECT_NEEDS_INVAL 8 /* T if object needs invalidation */
#define FSCACHE_OBJECT_NEEDS_UPDATE 9 /* T if object attrs need writing to disk */
struct list_head cache_link; /* link in cache->object_list */
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index ad44fa4cd844..2390af45d751 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -57,6 +57,8 @@ enum fscache_cookie_type {
#define FSCACHE_ADV_WRITE_CACHE 0x00 /* Do cache if written to locally */
#define FSCACHE_ADV_WRITE_NOCACHE 0x02 /* Don't cache if written to locally */
+#define FSCACHE_INVAL_LIGHT 0x01 /* Don't re-invalidate if temp object */
+
/*
* fscache cached network filesystem type
* - name, version and ops must be filled in before registration
@@ -105,7 +107,6 @@ struct fscache_cookie {
loff_t zero_point; /* Size after which no data on server */
unsigned long flags;
-#define FSCACHE_COOKIE_INVALIDATING 4 /* T if cookie is being invalidated */
#define FSCACHE_COOKIE_ACQUIRED 5 /* T if cookie is in use */
#define FSCACHE_COOKIE_RELINQUISHED 6 /* T if cookie has been relinquished */
@@ -149,6 +150,7 @@ struct fscache_io_request {
loff_t len; /* Size of the I/O */
loff_t transferred; /* Amount of data transferred */
short error; /* 0 or error that occurred */
+ unsigned int inval_counter; /* object->inval_counter at begin_op */
unsigned long flags;
#define FSCACHE_IO_DATA_FROM_SERVER 0 /* Set if data was read from server */
#define FSCACHE_IO_DATA_FROM_CACHE 1 /* Set if data was read from the cache */
@@ -197,7 +199,7 @@ extern void __fscache_use_cookie(struct fscache_cookie *, bool);
extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
-extern void __fscache_invalidate(struct fscache_cookie *, loff_t);
+extern void __fscache_invalidate(struct fscache_cookie *, const void *, loff_t, unsigned int);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
loff_t, bool);
@@ -431,7 +433,9 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
/**
* fscache_invalidate - Notify cache that an object needs invalidation
* @cookie: The cookie representing the cache object
+ * @aux_data: The updated auxiliary data for the cookie (may be NULL)
* @size: The revised size of the object.
+ * @flags: Invalidation flags (FSCACHE_INVAL_*)
*
* Notify the cache that an object is needs to be invalidated and that it
* should abort any retrievals or stores it is doing on the cache. The object
@@ -443,10 +447,11 @@ void fscache_unpin_cookie(struct fscache_cookie *cookie)
* description.
*/
static inline
-void fscache_invalidate(struct fscache_cookie *cookie, loff_t size)
+void fscache_invalidate(struct fscache_cookie *cookie,
+ const void *aux_data, loff_t size, unsigned int flags)
{
if (fscache_cookie_valid(cookie))
- __fscache_invalidate(cookie, size);
+ __fscache_invalidate(cookie, aux_data, size, flags);
}
/**
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 43/61] fscache: Implement "will_modify" parameter on fscache_use_cookie()
2020-05-04 17:07 ` David Howells
` (42 preceding siblings ...)
(?)
@ 2020-05-04 17:13 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:13 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Implement the "will_modify" parameter passed to fscache_use_cookie().
Setting this to true will henceforth cause the affected object to be marked
as dirty on disk, subject to conflict resolution in the event that power
failure or a crash occurs or the filesystem operates in disconnected mode.
The dirty flag is removed when the fscache_object is discarded from memory.
A cache hook is provided to prepare for writing - and this can be used to
mark the object on disk.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 65 +++++++++++++++++++++++++++++++++++++++++
fs/cachefiles/internal.h | 2 +
fs/cachefiles/xattr.c | 21 +++++++++++++
fs/fscache/cookie.c | 17 +++++++++--
fs/fscache/internal.h | 1 +
fs/fscache/obj.c | 29 +++++++++++++++---
include/linux/fscache-cache.h | 4 +++
7 files changed, 130 insertions(+), 9 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index b312a04f672b..457adfca5931 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -210,14 +210,78 @@ static void cachefiles_update_object(struct fscache_object *_object)
_leave("");
}
+/*
+ * Shorten the backing object to discard any dirty data and free up
+ * any unused granules.
+ */
+static bool cachefiles_shorten_object(struct cachefiles_object *object, loff_t new_size)
+{
+ struct cachefiles_cache *cache;
+ struct inode *inode;
+ struct path path;
+ loff_t i_size;
+
+ cache = container_of(object->fscache.cache,
+ struct cachefiles_cache, cache);
+ path.mnt = cache->mnt;
+ path.dentry = object->dentry;
+
+ inode = d_inode(object->dentry);
+ trace_cachefiles_trunc(object, inode, i_size_read(inode), new_size);
+ if (vfs_truncate(&path, new_size) < 0) {
+ cachefiles_io_error_obj(object, "Trunc-to-size failed");
+ cachefiles_remove_object_xattr(cache, object->dentry);
+ return false;
+ }
+
+ new_size = round_up(new_size, CACHEFILES_DIO_BLOCK_SIZE);
+ i_size = i_size_read(inode);
+ if (i_size < new_size) {
+ trace_cachefiles_trunc(object, inode, i_size, new_size);
+ if (vfs_truncate(&path, new_size) < 0) {
+ cachefiles_io_error_obj(object, "Trunc-to-dio-size failed");
+ cachefiles_remove_object_xattr(cache, object->dentry);
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/*
+ * Trim excess stored data off of an object.
+ */
+static bool cachefiles_trim_object(struct cachefiles_object *object)
+{
+ loff_t object_size;
+
+ _enter("{OBJ%x}", object->fscache.debug_id);
+
+ object_size = object->fscache.cookie->object_size;
+ if (i_size_read(d_inode(object->dentry)) <= object_size)
+ return true;
+
+ return cachefiles_shorten_object(object, object_size);
+}
+
/*
* Commit changes to the object as we drop it.
*/
static bool cachefiles_commit_object(struct cachefiles_object *object,
struct cachefiles_cache *cache)
{
+ bool update = false;
+
if (object->content_map_changed)
cachefiles_save_content_map(object);
+ if (test_and_clear_bit(FSCACHE_OBJECT_LOCAL_WRITE, &object->fscache.flags))
+ update = true;
+ if (test_and_clear_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->fscache.flags))
+ update = true;
+ if (update) {
+ if (cachefiles_trim_object(object))
+ cachefiles_set_object_xattr(object);
+ }
if (test_bit(CACHEFILES_OBJECT_USING_TMPFILE, &object->flags))
return cachefiles_commit_tmpfile(cache, object);
@@ -577,5 +641,6 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.shape_extent = cachefiles_shape_extent,
.read = cachefiles_read,
.write = cachefiles_write,
+ .prepare_to_write = cachefiles_prepare_to_write,
.display_object = cachefiles_display_object,
};
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index ccc8cd2b8250..487a434ddb75 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -248,7 +248,7 @@ extern int cachefiles_set_object_xattr(struct cachefiles_object *object);
extern int cachefiles_check_auxdata(struct cachefiles_object *object);
extern int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
struct dentry *dentry);
-
+extern int cachefiles_prepare_to_write(struct fscache_object *object);
/*
* error handling
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 22c56ca2fd0b..456301b7abb0 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -124,6 +124,8 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
buf->zero_point = cpu_to_be64(object->fscache.cookie->zero_point);
buf->type = object->fscache.cookie->type;
buf->content = object->content_info;
+ if (test_bit(FSCACHE_OBJECT_LOCAL_WRITE, &object->fscache.flags))
+ buf->content = CACHEFILES_CONTENT_DIRTY;
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->fscache.cookie), len);
@@ -184,10 +186,16 @@ int cachefiles_check_auxdata(struct cachefiles_object *object)
why = cachefiles_coherency_check_aux;
} else if (be64_to_cpu(buf->object_size) != object->fscache.cookie->object_size) {
why = cachefiles_coherency_check_objsize;
+ } else if (buf->content == CACHEFILES_CONTENT_DIRTY) {
+ // TODO: Begin conflict resolution
+ pr_warn("Dirty object in cache\n");
+ why = cachefiles_coherency_check_dirty;
} else {
object->fscache.cookie->zero_point = be64_to_cpu(buf->zero_point);
object->content_info = buf->content;
why = cachefiles_coherency_check_ok;
+ object->fscache.cookie->zero_point = be64_to_cpu(buf->zero_point);
+ object->content_info = buf->content;
ret = 0;
}
@@ -219,3 +227,16 @@ int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
_leave(" = %d", ret);
return ret;
}
+
+/*
+ * Stick a marker on the cache object to indicate that it's dirty.
+ */
+int cachefiles_prepare_to_write(struct fscache_object *_object)
+{
+ struct cachefiles_object *object =
+ container_of(_object, struct cachefiles_object, fscache);
+
+ _enter("c=%08x", object->fscache.cookie->debug_id);
+
+ return cachefiles_set_object_xattr(object);
+}
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c
index c0d93dc7f69f..c96cd23bcdb2 100644
--- a/fs/fscache/cookie.c
+++ b/fs/fscache/cookie.c
@@ -342,6 +342,8 @@ EXPORT_SYMBOL(__fscache_acquire_cookie);
void __fscache_use_cookie(struct fscache_cookie *cookie, bool will_modify)
{
enum fscache_cookie_stage stage;
+ struct fscache_object *object;
+ bool write_set;
_enter("c=%08x", cookie->debug_id);
@@ -360,7 +362,7 @@ void __fscache_use_cookie(struct fscache_cookie *cookie, bool will_modify)
/* The lookup job holds its own active increment */
atomic_inc(&cookie->n_active);
- fscache_dispatch(cookie, NULL, 0, fscache_lookup_object);
+ fscache_dispatch(cookie, NULL, will_modify, fscache_lookup_object);
break;
case FSCACHE_COOKIE_STAGE_INITIALISING:
@@ -373,8 +375,17 @@ void __fscache_use_cookie(struct fscache_cookie *cookie, bool will_modify)
case FSCACHE_COOKIE_STAGE_NO_DATA_YET:
case FSCACHE_COOKIE_STAGE_ACTIVE:
case FSCACHE_COOKIE_STAGE_INVALIDATING:
- // TODO: Handle will_modify
- spin_unlock(&cookie->lock);
+ if (will_modify) {
+ object = hlist_entry(cookie->backing_objects.first,
+ struct fscache_object, cookie_link);
+ write_set = test_and_set_bit(FSCACHE_OBJECT_LOCAL_WRITE,
+ &object->flags);
+ spin_unlock(&cookie->lock);
+ if (!write_set)
+ fscache_dispatch(cookie, object, 0, fscache_prepare_to_write);
+ } else {
+ spin_unlock(&cookie->lock);
+ }
break;
case FSCACHE_COOKIE_STAGE_DEAD:
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index b9cad60e3c4e..0b370d059bdf 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -134,6 +134,7 @@ extern void fscache_lookup_object(struct fscache_cookie *, struct fscache_object
extern void fscache_invalidate_object(struct fscache_cookie *, struct fscache_object *, int);
extern void fscache_drop_object(struct fscache_cookie *, struct fscache_object *, bool);
extern void fscache_discard_objects(struct fscache_cookie *, struct fscache_object *, int);
+extern void fscache_prepare_to_write(struct fscache_cookie *, struct fscache_object *, int);
/*
* object-list.c
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
index aea4239641a9..332ba132413d 100644
--- a/fs/fscache/obj.c
+++ b/fs/fscache/obj.c
@@ -117,7 +117,8 @@ static bool fscache_wrangle_object(struct fscache_cookie *cookie,
* Create an object chain, making sure that the index chain is fully created.
*/
static struct fscache_object *fscache_lookup_object_chain(struct fscache_cookie *cookie,
- struct fscache_cache *cache)
+ struct fscache_cache *cache,
+ bool will_modify)
{
struct fscache_object *object = NULL, *parent, *xobject;
@@ -131,7 +132,7 @@ static struct fscache_object *fscache_lookup_object_chain(struct fscache_cookie
spin_unlock(&cookie->lock);
/* Recurse to look up/create the parent index. */
- parent = fscache_lookup_object_chain(cookie->parent, cache);
+ parent = fscache_lookup_object_chain(cookie->parent, cache, false);
if (IS_ERR(parent))
goto error;
@@ -146,9 +147,13 @@ static struct fscache_object *fscache_lookup_object_chain(struct fscache_cookie
if (!object)
goto error;
+ if (will_modify)
+ __set_bit(FSCACHE_OBJECT_LOCAL_WRITE, &object->flags);
+
xobject = fscache_attach_object(cookie, object);
if (xobject != object) {
fscache_do_put_object(object, fscache_obj_put_alloc_dup);
+ object = xobject;
goto object_exists;
}
@@ -199,7 +204,8 @@ static struct fscache_object *fscache_lookup_object_chain(struct fscache_cookie
* - this must make sure the index chain is instantiated and instantiate the
* object representation too
*/
-static void fscache_lookup_object_locked(struct fscache_cookie *cookie)
+static void fscache_lookup_object_locked(struct fscache_cookie *cookie,
+ bool will_modify)
{
struct fscache_object *object;
struct fscache_cache *cache;
@@ -217,12 +223,16 @@ static void fscache_lookup_object_locked(struct fscache_cookie *cookie)
_debug("cache %s", cache->tag->name);
- object = fscache_lookup_object_chain(cookie, cache);
+ object = fscache_lookup_object_chain(cookie, cache, will_modify);
if (!object) {
_leave(" [fail]");
return;
}
+ if (will_modify &&
+ test_and_set_bit(FSCACHE_OBJECT_LOCAL_WRITE, &object->flags))
+ fscache_prepare_to_write(cookie, object, 0);
+
fscache_do_put_object(object, fscache_obj_put);
_leave(" [done]");
}
@@ -231,7 +241,7 @@ void fscache_lookup_object(struct fscache_cookie *cookie,
struct fscache_object *object, int param)
{
down_read(&fscache_addremove_sem);
- fscache_lookup_object_locked(cookie);
+ fscache_lookup_object_locked(cookie, param);
up_read(&fscache_addremove_sem);
__fscache_unuse_cookie(cookie, NULL, NULL);
}
@@ -336,3 +346,12 @@ void fscache_discard_objects(struct fscache_cookie *cookie,
up_read(&fscache_addremove_sem);
_leave("");
}
+
+/*
+ * Prepare a cache object to be written to.
+ */
+void fscache_prepare_to_write(struct fscache_cookie *cookie,
+ struct fscache_object *object, int param)
+{
+ object->cache->ops->prepare_to_write(object);
+}
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 4fb63d8a60cd..848ced13c4ae 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -155,6 +155,9 @@ struct fscache_cache_ops {
struct fscache_io_request *req,
struct iov_iter *iter);
+ /* Prepare to write to a live cache object */
+ int (*prepare_to_write)(struct fscache_object *object);
+
/* Display object info in /proc/fs/fscache/objects */
int (*display_object)(struct seq_file *m, struct fscache_object *object);
};
@@ -183,6 +186,7 @@ struct fscache_object {
spinlock_t lock; /* state and operations lock */
unsigned long flags;
+#define FSCACHE_OBJECT_LOCAL_WRITE 1 /* T if the object is being modified locally */
#define FSCACHE_OBJECT_NEEDS_INVAL 8 /* T if object needs invalidation */
#define FSCACHE_OBJECT_NEEDS_UPDATE 9 /* T if object attrs need writing to disk */
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 44/61] fscache: Provide resize operation
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Provide a cache operation to resize an object. This is intended to be run
synchronously rather than being deferred as it really needs to run inside
the inode lock on the netfs inode from ->setattr() to correctly order with
respect to other truncates and writes.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 24 ++++++++++++++++++++++++
fs/fscache/internal.h | 3 +++
fs/fscache/io.c | 27 +++++++++++++++++++++++++++
fs/fscache/stats.c | 9 +++++++--
include/linux/fscache-cache.h | 2 ++
include/linux/fscache.h | 18 ++++++++++++++++++
6 files changed, 81 insertions(+), 2 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 457adfca5931..f9c2c09e1713 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -248,6 +248,29 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object, loff_t n
return true;
}
+/*
+ * Resize the backing object.
+ */
+static void cachefiles_resize_object(struct fscache_object *_object, loff_t new_size)
+{
+ struct cachefiles_object *object =
+ container_of(_object, struct cachefiles_object, fscache);
+ loff_t old_size = object->fscache.cookie->object_size;
+
+ _enter("%llu->%llu", old_size, new_size);
+
+ if (new_size < old_size) {
+ cachefiles_shorten_content_map(object, new_size);
+ cachefiles_shorten_object(object, new_size);
+ return;
+ }
+
+ /* The file is being expanded. We don't need to do anything
+ * particularly. cookie->initial_size doesn't change and so the point
+ * at which we have to download before doesn't change.
+ */
+}
+
/*
* Trim excess stored data off of an object.
*/
@@ -633,6 +656,7 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.free_lookup_data = cachefiles_free_lookup_data,
.grab_object = cachefiles_grab_object,
.update_object = cachefiles_update_object,
+ .resize_object = cachefiles_resize_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
.put_object = cachefiles_put_object,
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 0b370d059bdf..7a0b36a49457 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -176,6 +176,9 @@ extern atomic_t fscache_n_updates;
extern atomic_t fscache_n_updates_null;
extern atomic_t fscache_n_updates_run;
+extern atomic_t fscache_n_resizes;
+extern atomic_t fscache_n_resizes_null;
+
extern atomic_t fscache_n_relinquishes;
extern atomic_t fscache_n_relinquishes_null;
extern atomic_t fscache_n_relinquishes_retire;
diff --git a/fs/fscache/io.c b/fs/fscache/io.c
index b6ec15588334..8c95679203e7 100644
--- a/fs/fscache/io.c
+++ b/fs/fscache/io.c
@@ -175,3 +175,30 @@ int __fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
}
}
EXPORT_SYMBOL(__fscache_write);
+
+/*
+ * Change the size of a backing object.
+ */
+void __fscache_resize_cookie(struct fscache_cookie *cookie, loff_t new_size)
+{
+ struct fscache_object *object;
+
+ ASSERT(cookie->type != FSCACHE_COOKIE_TYPE_INDEX);
+
+ object = fscache_begin_io_operation(cookie, FSCACHE_WANT_WRITE, NULL);
+ if (!IS_ERR(object)) {
+ fscache_stat(&fscache_n_resizes);
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags);
+
+ /* We cannot defer a resize as we need to do it inside the
+ * netfs's inode lock so that we're serialised with respect to
+ * writes.
+ */
+ object->cache->ops->resize_object(object, new_size);
+ object->cache->ops->put_object(object, fscache_obj_put_ioreq);
+ fscache_end_io_operation(cookie);
+ } else {
+ fscache_stat(&fscache_n_resizes_null);
+ }
+}
+EXPORT_SYMBOL(__fscache_resize_cookie);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index fdea31ff8e69..3b262af57625 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -26,6 +26,9 @@ atomic_t fscache_n_updates;
atomic_t fscache_n_updates_null;
atomic_t fscache_n_updates_run;
+atomic_t fscache_n_resizes;
+atomic_t fscache_n_resizes_null;
+
atomic_t fscache_n_relinquishes;
atomic_t fscache_n_relinquishes_null;
atomic_t fscache_n_relinquishes_retire;
@@ -117,10 +120,12 @@ int fscache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "Invals : n=%u\n",
atomic_read(&fscache_n_invalidates));
- seq_printf(m, "Updates: n=%u nul=%u run=%u\n",
+ seq_printf(m, "Updates: n=%u nul=%u run=%u rsz=%u rsn=%u\n",
atomic_read(&fscache_n_updates),
atomic_read(&fscache_n_updates_null),
- atomic_read(&fscache_n_updates_run));
+ atomic_read(&fscache_n_updates_run),
+ atomic_read(&fscache_n_resizes),
+ atomic_read(&fscache_n_resizes_null));
seq_printf(m, "Relinqs: n=%u nul=%u rtr=%u\n",
atomic_read(&fscache_n_relinquishes),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 848ced13c4ae..adb899448f6e 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -118,6 +118,8 @@ struct fscache_cache_ops {
/* store the updated auxiliary data on an object */
void (*update_object)(struct fscache_object *object);
+ /* Change the size of a data object */
+ void (*resize_object)(struct fscache_object *object, loff_t new_size);
/* Invalidate an object */
bool (*invalidate_object)(struct fscache_object *object,
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 2390af45d751..a5fd1e365fd9 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -199,6 +199,7 @@ extern void __fscache_use_cookie(struct fscache_cookie *, bool);
extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
+extern void __fscache_resize_cookie(struct fscache_cookie *, loff_t);
extern void __fscache_invalidate(struct fscache_cookie *, const void *, loff_t, unsigned int);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
@@ -401,6 +402,23 @@ void fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data,
__fscache_update_cookie(cookie, aux_data, object_size);
}
+/**
+ * fscache_resize_cookie - Request that a cache object be resized
+ * @cookie: The cookie representing the cache object
+ * @new_size: The new size of the object (may be NULL)
+ *
+ * Request that the size of an object be changed.
+ *
+ * See Documentation/filesystems/caching/netfs-api.txt for a complete
+ * description.
+ */
+static inline
+void fscache_resize_cookie(struct fscache_cookie *cookie, loff_t new_size)
+{
+ if (fscache_cookie_valid(cookie))
+ __fscache_resize_cookie(cookie, new_size);
+}
+
/**
* fscache_pin_cookie - Pin a data-storage cache object in its cache
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 44/61] fscache: Provide resize operation
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Provide a cache operation to resize an object. This is intended to be run
synchronously rather than being deferred as it really needs to run inside
the inode lock on the netfs inode from ->setattr() to correctly order with
respect to other truncates and writes.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/interface.c | 24 ++++++++++++++++++++++++
fs/fscache/internal.h | 3 +++
fs/fscache/io.c | 27 +++++++++++++++++++++++++++
fs/fscache/stats.c | 9 +++++++--
include/linux/fscache-cache.h | 2 ++
include/linux/fscache.h | 18 ++++++++++++++++++
6 files changed, 81 insertions(+), 2 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 457adfca5931..f9c2c09e1713 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -248,6 +248,29 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object, loff_t n
return true;
}
+/*
+ * Resize the backing object.
+ */
+static void cachefiles_resize_object(struct fscache_object *_object, loff_t new_size)
+{
+ struct cachefiles_object *object =
+ container_of(_object, struct cachefiles_object, fscache);
+ loff_t old_size = object->fscache.cookie->object_size;
+
+ _enter("%llu->%llu", old_size, new_size);
+
+ if (new_size < old_size) {
+ cachefiles_shorten_content_map(object, new_size);
+ cachefiles_shorten_object(object, new_size);
+ return;
+ }
+
+ /* The file is being expanded. We don't need to do anything
+ * particularly. cookie->initial_size doesn't change and so the point
+ * at which we have to download before doesn't change.
+ */
+}
+
/*
* Trim excess stored data off of an object.
*/
@@ -633,6 +656,7 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.free_lookup_data = cachefiles_free_lookup_data,
.grab_object = cachefiles_grab_object,
.update_object = cachefiles_update_object,
+ .resize_object = cachefiles_resize_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
.put_object = cachefiles_put_object,
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 0b370d059bdf..7a0b36a49457 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -176,6 +176,9 @@ extern atomic_t fscache_n_updates;
extern atomic_t fscache_n_updates_null;
extern atomic_t fscache_n_updates_run;
+extern atomic_t fscache_n_resizes;
+extern atomic_t fscache_n_resizes_null;
+
extern atomic_t fscache_n_relinquishes;
extern atomic_t fscache_n_relinquishes_null;
extern atomic_t fscache_n_relinquishes_retire;
diff --git a/fs/fscache/io.c b/fs/fscache/io.c
index b6ec15588334..8c95679203e7 100644
--- a/fs/fscache/io.c
+++ b/fs/fscache/io.c
@@ -175,3 +175,30 @@ int __fscache_write(struct fscache_io_request *req, struct iov_iter *iter)
}
}
EXPORT_SYMBOL(__fscache_write);
+
+/*
+ * Change the size of a backing object.
+ */
+void __fscache_resize_cookie(struct fscache_cookie *cookie, loff_t new_size)
+{
+ struct fscache_object *object;
+
+ ASSERT(cookie->type != FSCACHE_COOKIE_TYPE_INDEX);
+
+ object = fscache_begin_io_operation(cookie, FSCACHE_WANT_WRITE, NULL);
+ if (!IS_ERR(object)) {
+ fscache_stat(&fscache_n_resizes);
+ set_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags);
+
+ /* We cannot defer a resize as we need to do it inside the
+ * netfs's inode lock so that we're serialised with respect to
+ * writes.
+ */
+ object->cache->ops->resize_object(object, new_size);
+ object->cache->ops->put_object(object, fscache_obj_put_ioreq);
+ fscache_end_io_operation(cookie);
+ } else {
+ fscache_stat(&fscache_n_resizes_null);
+ }
+}
+EXPORT_SYMBOL(__fscache_resize_cookie);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index fdea31ff8e69..3b262af57625 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -26,6 +26,9 @@ atomic_t fscache_n_updates;
atomic_t fscache_n_updates_null;
atomic_t fscache_n_updates_run;
+atomic_t fscache_n_resizes;
+atomic_t fscache_n_resizes_null;
+
atomic_t fscache_n_relinquishes;
atomic_t fscache_n_relinquishes_null;
atomic_t fscache_n_relinquishes_retire;
@@ -117,10 +120,12 @@ int fscache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "Invals : n=%u\n",
atomic_read(&fscache_n_invalidates));
- seq_printf(m, "Updates: n=%u nul=%u run=%u\n",
+ seq_printf(m, "Updates: n=%u nul=%u run=%u rsz=%u rsn=%u\n",
atomic_read(&fscache_n_updates),
atomic_read(&fscache_n_updates_null),
- atomic_read(&fscache_n_updates_run));
+ atomic_read(&fscache_n_updates_run),
+ atomic_read(&fscache_n_resizes),
+ atomic_read(&fscache_n_resizes_null));
seq_printf(m, "Relinqs: n=%u nul=%u rtr=%u\n",
atomic_read(&fscache_n_relinquishes),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index 848ced13c4ae..adb899448f6e 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -118,6 +118,8 @@ struct fscache_cache_ops {
/* store the updated auxiliary data on an object */
void (*update_object)(struct fscache_object *object);
+ /* Change the size of a data object */
+ void (*resize_object)(struct fscache_object *object, loff_t new_size);
/* Invalidate an object */
bool (*invalidate_object)(struct fscache_object *object,
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 2390af45d751..a5fd1e365fd9 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -199,6 +199,7 @@ extern void __fscache_use_cookie(struct fscache_cookie *, bool);
extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *);
extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool);
extern void __fscache_update_cookie(struct fscache_cookie *, const void *, const loff_t *);
+extern void __fscache_resize_cookie(struct fscache_cookie *, loff_t);
extern void __fscache_invalidate(struct fscache_cookie *, const void *, loff_t, unsigned int);
extern unsigned int __fscache_shape_extent(struct fscache_cookie *,
struct fscache_extent *,
@@ -401,6 +402,23 @@ void fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data,
__fscache_update_cookie(cookie, aux_data, object_size);
}
+/**
+ * fscache_resize_cookie - Request that a cache object be resized
+ * @cookie: The cookie representing the cache object
+ * @new_size: The new size of the object (may be NULL)
+ *
+ * Request that the size of an object be changed.
+ *
+ * See Documentation/filesystems/caching/netfs-api.txt for a complete
+ * description.
+ */
+static inline
+void fscache_resize_cookie(struct fscache_cookie *cookie, loff_t new_size)
+{
+ if (fscache_cookie_valid(cookie))
+ __fscache_resize_cookie(cookie, new_size);
+}
+
/**
* fscache_pin_cookie - Pin a data-storage cache object in its cache
* @cookie: The cookie representing the cache object
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 45/61] fscache: Remove the update operation
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the cache-side of the object update operation as it doesn't
serialise with other setattr, O_TRUNC and write operations.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/interface.c | 59 -----------------------------------------
fs/fscache/internal.h | 1 -
fs/fscache/obj.c | 14 ----------
fs/fscache/stats.c | 4 +--
include/linux/fscache-cache.h | 2 -
5 files changed, 1 insertion(+), 79 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index f9c2c09e1713..59910433f2a3 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -152,64 +152,6 @@ struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
return &object->fscache;
}
-/*
- * update the auxiliary data for an object object on disk
- */
-static void cachefiles_update_object(struct fscache_object *_object)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- const struct cred *saved_cred;
- struct inode *inode;
- loff_t object_size, i_size;
- int ret;
-
- _enter("{OBJ%x}", _object->debug_id);
-
- object = container_of(_object, struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache, struct cachefiles_cache,
- cache);
-
- cachefiles_begin_secure(cache, &saved_cred);
-
- object_size = object->fscache.cookie->object_size;
- inode = d_inode(object->dentry);
- i_size = i_size_read(inode);
- if (i_size > object_size) {
- struct path path = {
- .mnt = cache->mnt,
- .dentry = object->dentry
- };
- _debug("trunc %llx -> %llx", i_size, object_size);
- trace_cachefiles_trunc(object, inode, i_size, object_size);
- ret = vfs_truncate(&path, object_size);
- if (ret < 0) {
- cachefiles_io_error_obj(object, "Trunc-to-size failed");
- cachefiles_remove_object_xattr(cache, object->dentry);
- goto out;
- }
-
- object_size = round_up(object_size, CACHEFILES_DIO_BLOCK_SIZE);
- i_size = i_size_read(inode);
- _debug("trunc %llx -> %llx", i_size, object_size);
- if (i_size < object_size) {
- trace_cachefiles_trunc(object, inode, i_size, object_size);
- ret = vfs_truncate(&path, object_size);
- if (ret < 0) {
- cachefiles_io_error_obj(object, "Trunc-to-dio-size failed");
- cachefiles_remove_object_xattr(cache, object->dentry);
- goto out;
- }
- }
- }
-
- cachefiles_set_object_xattr(object);
-
-out:
- cachefiles_end_secure(cache, saved_cred);
- _leave("");
-}
-
/*
* Shorten the backing object to discard any dirty data and free up
* any unused granules.
@@ -655,7 +597,6 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.lookup_object = cachefiles_lookup_object,
.free_lookup_data = cachefiles_free_lookup_data,
.grab_object = cachefiles_grab_object,
- .update_object = cachefiles_update_object,
.resize_object = cachefiles_resize_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 7a0b36a49457..2670c821e1c9 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -200,7 +200,6 @@ extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
extern atomic_t fscache_n_cop_create_object;
extern atomic_t fscache_n_cop_invalidate_object;
-extern atomic_t fscache_n_cop_update_object;
extern atomic_t fscache_n_cop_drop_object;
extern atomic_t fscache_n_cop_put_object;
extern atomic_t fscache_n_cop_sync_cache;
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
index 332ba132413d..b2e7675a80c1 100644
--- a/fs/fscache/obj.c
+++ b/fs/fscache/obj.c
@@ -54,14 +54,6 @@ static int fscache_do_create_object(struct fscache_object *object, void *data)
return ret;
}
-static void fscache_do_update_object(struct fscache_object *object)
-{
- fscache_stat(&fscache_n_updates_run);
- fscache_stat(&fscache_n_cop_update_object);
- object->cache->ops->update_object(object);
- fscache_stat_d(&fscache_n_cop_update_object);
-}
-
static void fscache_do_drop_object(struct fscache_cache *cache,
struct fscache_object *object,
bool invalidate)
@@ -279,12 +271,6 @@ void fscache_drop_object(struct fscache_cookie *cookie,
_enter("{OBJ%x,%d},%u",
object->debug_id, object->n_children, invalidate);
- if (!invalidate &&
- test_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags)) {
- _debug("final update");
- fscache_do_update_object(object);
- }
-
spin_lock(&cache->object_list_lock);
list_del_init(&object->cache_link);
spin_unlock(&cache->object_list_lock);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 3b262af57625..cf71c8b2a22a 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -50,7 +50,6 @@ atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
atomic_t fscache_n_cop_create_object;
atomic_t fscache_n_cop_invalidate_object;
-atomic_t fscache_n_cop_update_object;
atomic_t fscache_n_cop_drop_object;
atomic_t fscache_n_cop_put_object;
atomic_t fscache_n_cop_sync_cache;
@@ -135,9 +134,8 @@ int fscache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "CacheOp: alo=%d luo=%d\n",
atomic_read(&fscache_n_cop_alloc_object),
atomic_read(&fscache_n_cop_lookup_object));
- seq_printf(m, "CacheOp: inv=%d upo=%d dro=%d pto=%d atc=%d syn=%d\n",
+ seq_printf(m, "CacheOp: inv=%d dro=%d pto=%d atc=%d syn=%d\n",
atomic_read(&fscache_n_cop_invalidate_object),
- atomic_read(&fscache_n_cop_update_object),
atomic_read(&fscache_n_cop_drop_object),
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index adb899448f6e..95f7c60b9092 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -116,8 +116,6 @@ struct fscache_cache_ops {
/* unpin an object in the cache */
void (*unpin_object)(struct fscache_object *object);
- /* store the updated auxiliary data on an object */
- void (*update_object)(struct fscache_object *object);
/* Change the size of a data object */
void (*resize_object)(struct fscache_object *object, loff_t new_size);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 45/61] fscache: Remove the update operation
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Remove the cache-side of the object update operation as it doesn't
serialise with other setattr, O_TRUNC and write operations.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/cachefiles/interface.c | 59 -----------------------------------------
fs/fscache/internal.h | 1 -
fs/fscache/obj.c | 14 ----------
fs/fscache/stats.c | 4 +--
include/linux/fscache-cache.h | 2 -
5 files changed, 1 insertion(+), 79 deletions(-)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index f9c2c09e1713..59910433f2a3 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -152,64 +152,6 @@ struct fscache_object *cachefiles_grab_object(struct fscache_object *_object,
return &object->fscache;
}
-/*
- * update the auxiliary data for an object object on disk
- */
-static void cachefiles_update_object(struct fscache_object *_object)
-{
- struct cachefiles_object *object;
- struct cachefiles_cache *cache;
- const struct cred *saved_cred;
- struct inode *inode;
- loff_t object_size, i_size;
- int ret;
-
- _enter("{OBJ%x}", _object->debug_id);
-
- object = container_of(_object, struct cachefiles_object, fscache);
- cache = container_of(object->fscache.cache, struct cachefiles_cache,
- cache);
-
- cachefiles_begin_secure(cache, &saved_cred);
-
- object_size = object->fscache.cookie->object_size;
- inode = d_inode(object->dentry);
- i_size = i_size_read(inode);
- if (i_size > object_size) {
- struct path path = {
- .mnt = cache->mnt,
- .dentry = object->dentry
- };
- _debug("trunc %llx -> %llx", i_size, object_size);
- trace_cachefiles_trunc(object, inode, i_size, object_size);
- ret = vfs_truncate(&path, object_size);
- if (ret < 0) {
- cachefiles_io_error_obj(object, "Trunc-to-size failed");
- cachefiles_remove_object_xattr(cache, object->dentry);
- goto out;
- }
-
- object_size = round_up(object_size, CACHEFILES_DIO_BLOCK_SIZE);
- i_size = i_size_read(inode);
- _debug("trunc %llx -> %llx", i_size, object_size);
- if (i_size < object_size) {
- trace_cachefiles_trunc(object, inode, i_size, object_size);
- ret = vfs_truncate(&path, object_size);
- if (ret < 0) {
- cachefiles_io_error_obj(object, "Trunc-to-dio-size failed");
- cachefiles_remove_object_xattr(cache, object->dentry);
- goto out;
- }
- }
- }
-
- cachefiles_set_object_xattr(object);
-
-out:
- cachefiles_end_secure(cache, saved_cred);
- _leave("");
-}
-
/*
* Shorten the backing object to discard any dirty data and free up
* any unused granules.
@@ -655,7 +597,6 @@ const struct fscache_cache_ops cachefiles_cache_ops = {
.lookup_object = cachefiles_lookup_object,
.free_lookup_data = cachefiles_free_lookup_data,
.grab_object = cachefiles_grab_object,
- .update_object = cachefiles_update_object,
.resize_object = cachefiles_resize_object,
.invalidate_object = cachefiles_invalidate_object,
.drop_object = cachefiles_drop_object,
diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h
index 7a0b36a49457..2670c821e1c9 100644
--- a/fs/fscache/internal.h
+++ b/fs/fscache/internal.h
@@ -200,7 +200,6 @@ extern atomic_t fscache_n_cop_alloc_object;
extern atomic_t fscache_n_cop_lookup_object;
extern atomic_t fscache_n_cop_create_object;
extern atomic_t fscache_n_cop_invalidate_object;
-extern atomic_t fscache_n_cop_update_object;
extern atomic_t fscache_n_cop_drop_object;
extern atomic_t fscache_n_cop_put_object;
extern atomic_t fscache_n_cop_sync_cache;
diff --git a/fs/fscache/obj.c b/fs/fscache/obj.c
index 332ba132413d..b2e7675a80c1 100644
--- a/fs/fscache/obj.c
+++ b/fs/fscache/obj.c
@@ -54,14 +54,6 @@ static int fscache_do_create_object(struct fscache_object *object, void *data)
return ret;
}
-static void fscache_do_update_object(struct fscache_object *object)
-{
- fscache_stat(&fscache_n_updates_run);
- fscache_stat(&fscache_n_cop_update_object);
- object->cache->ops->update_object(object);
- fscache_stat_d(&fscache_n_cop_update_object);
-}
-
static void fscache_do_drop_object(struct fscache_cache *cache,
struct fscache_object *object,
bool invalidate)
@@ -279,12 +271,6 @@ void fscache_drop_object(struct fscache_cookie *cookie,
_enter("{OBJ%x,%d},%u",
object->debug_id, object->n_children, invalidate);
- if (!invalidate &&
- test_bit(FSCACHE_OBJECT_NEEDS_UPDATE, &object->flags)) {
- _debug("final update");
- fscache_do_update_object(object);
- }
-
spin_lock(&cache->object_list_lock);
list_del_init(&object->cache_link);
spin_unlock(&cache->object_list_lock);
diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c
index 3b262af57625..cf71c8b2a22a 100644
--- a/fs/fscache/stats.c
+++ b/fs/fscache/stats.c
@@ -50,7 +50,6 @@ atomic_t fscache_n_cop_alloc_object;
atomic_t fscache_n_cop_lookup_object;
atomic_t fscache_n_cop_create_object;
atomic_t fscache_n_cop_invalidate_object;
-atomic_t fscache_n_cop_update_object;
atomic_t fscache_n_cop_drop_object;
atomic_t fscache_n_cop_put_object;
atomic_t fscache_n_cop_sync_cache;
@@ -135,9 +134,8 @@ int fscache_stats_show(struct seq_file *m, void *v)
seq_printf(m, "CacheOp: alo=%d luo=%d\n",
atomic_read(&fscache_n_cop_alloc_object),
atomic_read(&fscache_n_cop_lookup_object));
- seq_printf(m, "CacheOp: inv=%d upo=%d dro=%d pto=%d atc=%d syn=%d\n",
+ seq_printf(m, "CacheOp: inv=%d dro=%d pto=%d atc=%d syn=%d\n",
atomic_read(&fscache_n_cop_invalidate_object),
- atomic_read(&fscache_n_cop_update_object),
atomic_read(&fscache_n_cop_drop_object),
atomic_read(&fscache_n_cop_put_object),
atomic_read(&fscache_n_cop_attr_changed),
diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h
index adb899448f6e..95f7c60b9092 100644
--- a/include/linux/fscache-cache.h
+++ b/include/linux/fscache-cache.h
@@ -116,8 +116,6 @@ struct fscache_cache_ops {
/* unpin an object in the cache */
void (*unpin_object)(struct fscache_object *object);
- /* store the updated auxiliary data on an object */
- void (*update_object)(struct fscache_object *object);
/* Change the size of a data object */
void (*resize_object)(struct fscache_object *object, loff_t new_size);
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 46/61] cachefiles: Shape write requests
2020-05-04 17:07 ` David Howells
` (45 preceding siblings ...)
(?)
@ 2020-05-04 17:14 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
In cachefiles_shape_extent(), shape a write request to always write to the
cache. The assumption is made that the caller has read the entire cache
granule beforehand if necessary.
Possibly this should be amended so that writes will only take place to
granules that are marked present and granules that lie beyond the EOF.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/cachefiles/content-map.c | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/fs/cachefiles/content-map.c b/fs/cachefiles/content-map.c
index 13632c097248..2c45092465b6 100644
--- a/fs/cachefiles/content-map.c
+++ b/fs/cachefiles/content-map.c
@@ -71,7 +71,8 @@ static unsigned int cachefiles_shape_single(struct fscache_object *obj,
extent->dio_block_size = CACHEFILES_DIO_BLOCK_SIZE;
- if (object->content_info == CACHEFILES_CONTENT_SINGLE) {
+ if (!for_write &&
+ object->content_info == CACHEFILES_CONTENT_SINGLE) {
ret = FSCACHE_READ_FROM_CACHE;
} else {
eof = (i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
@@ -128,13 +129,20 @@ unsigned int cachefiles_shape_extent(struct fscache_object *obj,
granule = start / CACHEFILES_GRAN_PAGES;
- /* If the content map didn't get expanded for some reason - simply
- * ignore this granule.
- */
- if (granule / 8 >= object->content_map_size)
- return 0;
+ if (granule / 8 >= object->content_map_size) {
+ cachefiles_expand_content_map(object, i_size);
+ if (granule / 8 >= object->content_map_size)
+ return 0;
+ }
- if (cachefiles_granule_is_present(object, granule)) {
+ if (for_write) {
+ /* Assume that the preparation to write involved preloading any
+ * bits of the cache that weren't to be written and filling any
+ * gaps that didn't end up being written.
+ */
+ bend = end;
+ ret = FSCACHE_WRITE_TO_CACHE;
+ } else if (cachefiles_granule_is_present(object, granule)) {
/* The start of the requested extent is present in the cache -
* restrict the returned extent to the maximum length of what's
* available.
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 47/61] afs: Remove afs_zero_fid as it's not used
2020-05-04 17:07 ` David Howells
` (46 preceding siblings ...)
(?)
@ 2020-05-04 17:14 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove afs_zero_fid as it's not used.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/yfsclient.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index b5b45c57e1b1..7f419f8bb943 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -15,8 +15,6 @@
#include "xdr_fs.h"
#include "protocol_yfs.h"
-static const struct afs_fid afs_zero_fid;
-
static inline void afs_use_fs_server(struct afs_call *call, struct afs_cb_interest *cbi)
{
call->cbi = afs_get_cb_interest(cbi);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 48/61] afs: Move key to afs_read struct
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Stash the key used to authenticate read operations in the afs_read struct.
This will be necessary to reissue the operation against the server if a
read from the cache fails in upcoming cache changes.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/dir.c | 3 ++-
fs/afs/file.c | 16 +++++++++-------
fs/afs/internal.h | 3 ++-
fs/afs/write.c | 12 ++++++------
4 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index d1e1caa23c8b..9d8504885f6a 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -243,6 +243,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
return ERR_PTR(-ENOMEM);
refcount_set(&req->usage, 1);
+ req->key = key_get(key);
req->nr_pages = nr_pages;
req->actual_len = i_size; /* May change */
req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
@@ -308,7 +309,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
if (!test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
trace_afs_reload_dir(dvnode);
- ret = afs_fetch_data(dvnode, key, req);
+ ret = afs_fetch_data(dvnode, req);
if (ret < 0)
goto error_unlock;
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 39db1e7b186d..834f47c4dc94 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -211,6 +211,7 @@ void afs_put_read(struct afs_read *req)
if (req->pages != req->array)
kfree(req->pages);
}
+ key_put(req->key);
kfree(req);
}
}
@@ -218,7 +219,7 @@ void afs_put_read(struct afs_read *req)
/*
* Fetch file data from the volume.
*/
-int afs_fetch_data(struct afs_vnode *vnode, struct key *key, struct afs_read *req)
+int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
{
struct afs_fs_cursor fc;
struct afs_status_cb *scb;
@@ -229,14 +230,14 @@ int afs_fetch_data(struct afs_vnode *vnode, struct key *key, struct afs_read *re
vnode->fid.vid,
vnode->fid.vnode,
vnode->fid.unique,
- key_serial(key));
+ key_serial(req->key));
scb = kzalloc(sizeof(struct afs_status_cb), GFP_KERNEL);
if (!scb)
return -ENOMEM;
ret = -ERESTARTSYS;
- if (afs_begin_vnode_operation(&fc, vnode, key, true)) {
+ if (afs_begin_vnode_operation(&fc, vnode, req->key, true)) {
afs_dataversion_t data_version = vnode->status.data_version;
while (afs_select_fileserver(&fc)) {
@@ -289,6 +290,7 @@ int afs_page_filler(void *data, struct page *page)
* unmarshalling code will clear the unfilled space.
*/
refcount_set(&req->usage, 1);
+ req->key = key_get(key);
req->pos = (loff_t)page->index << PAGE_SHIFT;
req->len = PAGE_SIZE;
req->nr_pages = 1;
@@ -298,7 +300,7 @@ int afs_page_filler(void *data, struct page *page)
/* read the contents of the file from the server into the
* page */
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
afs_put_read(req);
if (ret < 0) {
@@ -383,7 +385,6 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
struct afs_read *req;
struct list_head *p;
struct page *first, *page;
- struct key *key = afs_file_key(file);
pgoff_t index;
int ret, n, i;
@@ -407,6 +408,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
refcount_set(&req->usage, 1);
req->vnode = vnode;
+ req->key = key_get(afs_file_key(file));
req->page_done = afs_readpages_page_done;
req->pos = first->index;
req->pos <<= PAGE_SHIFT;
@@ -436,11 +438,11 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
} while (req->nr_pages < n);
if (req->nr_pages == 0) {
- kfree(req);
+ afs_put_read(req);
return 0;
}
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
if (ret < 0)
goto error;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 633c0f49bb55..68d3eb0dc367 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -227,6 +227,7 @@ struct afs_read {
loff_t actual_len; /* How much we're actually getting */
loff_t remain; /* Amount remaining */
loff_t file_size; /* File size returned by server */
+ struct key *key; /* The key to use to reissue the read */
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
unsigned int index; /* Which page we're reading into */
@@ -921,7 +922,7 @@ extern int afs_cache_wb_key(struct afs_vnode *, struct afs_file *);
extern void afs_put_wb_key(struct afs_wb_key *);
extern int afs_open(struct inode *, struct file *);
extern int afs_release(struct inode *, struct file *);
-extern int afs_fetch_data(struct afs_vnode *, struct key *, struct afs_read *);
+extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
extern int afs_page_filler(void *, struct page *);
extern void afs_put_read(struct afs_read *);
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 25786237485a..8473f9bc3548 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -25,9 +25,10 @@ int afs_set_page_dirty(struct page *page)
/*
* partly or wholly fill a page that's under preparation for writing
*/
-static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
+static int afs_fill_page(struct file *file,
loff_t pos, unsigned int len, struct page *page)
{
+ struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
struct afs_read *req;
size_t p;
void *data;
@@ -49,6 +50,7 @@ static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
return -ENOMEM;
refcount_set(&req->usage, 1);
+ req->key = afs_file_key(file);
req->pos = pos;
req->len = len;
req->nr_pages = 1;
@@ -56,7 +58,7 @@ static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
req->pages[0] = page;
get_page(page);
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
afs_put_read(req);
if (ret < 0) {
if (ret == -ENOENT) {
@@ -80,7 +82,6 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
{
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
struct page *page;
- struct key *key = afs_file_key(file);
unsigned long priv;
unsigned f, from = pos & (PAGE_SIZE - 1);
unsigned t, to = from + len;
@@ -100,7 +101,7 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
return -ENOMEM;
if (!PageUptodate(page) && len != PAGE_SIZE) {
- ret = afs_fill_page(vnode, key, pos & PAGE_MASK, PAGE_SIZE, page);
+ ret = afs_fill_page(file, pos & PAGE_MASK, PAGE_SIZE, page);
if (ret < 0) {
unlock_page(page);
put_page(page);
@@ -183,7 +184,6 @@ int afs_write_end(struct file *file, struct address_space *mapping,
struct page *page, void *fsdata)
{
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
- struct key *key = afs_file_key(file);
loff_t i_size, maybe_i_size;
int ret;
@@ -207,7 +207,7 @@ int afs_write_end(struct file *file, struct address_space *mapping,
* unmarshalling routine will take care of clearing any
* bits that are beyond the EOF.
*/
- ret = afs_fill_page(vnode, key, pos + copied,
+ ret = afs_fill_page(file, pos + copied,
len - copied, page);
if (ret < 0)
goto out;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 48/61] afs: Move key to afs_read struct
@ 2020-05-04 17:14 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Stash the key used to authenticate read operations in the afs_read struct.
This will be necessary to reissue the operation against the server if a
read from the cache fails in upcoming cache changes.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/afs/dir.c | 3 ++-
fs/afs/file.c | 16 +++++++++-------
fs/afs/internal.h | 3 ++-
fs/afs/write.c | 12 ++++++------
4 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index d1e1caa23c8b..9d8504885f6a 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -243,6 +243,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
return ERR_PTR(-ENOMEM);
refcount_set(&req->usage, 1);
+ req->key = key_get(key);
req->nr_pages = nr_pages;
req->actual_len = i_size; /* May change */
req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
@@ -308,7 +309,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
if (!test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
trace_afs_reload_dir(dvnode);
- ret = afs_fetch_data(dvnode, key, req);
+ ret = afs_fetch_data(dvnode, req);
if (ret < 0)
goto error_unlock;
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 39db1e7b186d..834f47c4dc94 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -211,6 +211,7 @@ void afs_put_read(struct afs_read *req)
if (req->pages != req->array)
kfree(req->pages);
}
+ key_put(req->key);
kfree(req);
}
}
@@ -218,7 +219,7 @@ void afs_put_read(struct afs_read *req)
/*
* Fetch file data from the volume.
*/
-int afs_fetch_data(struct afs_vnode *vnode, struct key *key, struct afs_read *req)
+int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
{
struct afs_fs_cursor fc;
struct afs_status_cb *scb;
@@ -229,14 +230,14 @@ int afs_fetch_data(struct afs_vnode *vnode, struct key *key, struct afs_read *re
vnode->fid.vid,
vnode->fid.vnode,
vnode->fid.unique,
- key_serial(key));
+ key_serial(req->key));
scb = kzalloc(sizeof(struct afs_status_cb), GFP_KERNEL);
if (!scb)
return -ENOMEM;
ret = -ERESTARTSYS;
- if (afs_begin_vnode_operation(&fc, vnode, key, true)) {
+ if (afs_begin_vnode_operation(&fc, vnode, req->key, true)) {
afs_dataversion_t data_version = vnode->status.data_version;
while (afs_select_fileserver(&fc)) {
@@ -289,6 +290,7 @@ int afs_page_filler(void *data, struct page *page)
* unmarshalling code will clear the unfilled space.
*/
refcount_set(&req->usage, 1);
+ req->key = key_get(key);
req->pos = (loff_t)page->index << PAGE_SHIFT;
req->len = PAGE_SIZE;
req->nr_pages = 1;
@@ -298,7 +300,7 @@ int afs_page_filler(void *data, struct page *page)
/* read the contents of the file from the server into the
* page */
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
afs_put_read(req);
if (ret < 0) {
@@ -383,7 +385,6 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
struct afs_read *req;
struct list_head *p;
struct page *first, *page;
- struct key *key = afs_file_key(file);
pgoff_t index;
int ret, n, i;
@@ -407,6 +408,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
refcount_set(&req->usage, 1);
req->vnode = vnode;
+ req->key = key_get(afs_file_key(file));
req->page_done = afs_readpages_page_done;
req->pos = first->index;
req->pos <<= PAGE_SHIFT;
@@ -436,11 +438,11 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
} while (req->nr_pages < n);
if (req->nr_pages == 0) {
- kfree(req);
+ afs_put_read(req);
return 0;
}
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
if (ret < 0)
goto error;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 633c0f49bb55..68d3eb0dc367 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -227,6 +227,7 @@ struct afs_read {
loff_t actual_len; /* How much we're actually getting */
loff_t remain; /* Amount remaining */
loff_t file_size; /* File size returned by server */
+ struct key *key; /* The key to use to reissue the read */
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
unsigned int index; /* Which page we're reading into */
@@ -921,7 +922,7 @@ extern int afs_cache_wb_key(struct afs_vnode *, struct afs_file *);
extern void afs_put_wb_key(struct afs_wb_key *);
extern int afs_open(struct inode *, struct file *);
extern int afs_release(struct inode *, struct file *);
-extern int afs_fetch_data(struct afs_vnode *, struct key *, struct afs_read *);
+extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
extern int afs_page_filler(void *, struct page *);
extern void afs_put_read(struct afs_read *);
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 25786237485a..8473f9bc3548 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -25,9 +25,10 @@ int afs_set_page_dirty(struct page *page)
/*
* partly or wholly fill a page that's under preparation for writing
*/
-static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
+static int afs_fill_page(struct file *file,
loff_t pos, unsigned int len, struct page *page)
{
+ struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
struct afs_read *req;
size_t p;
void *data;
@@ -49,6 +50,7 @@ static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
return -ENOMEM;
refcount_set(&req->usage, 1);
+ req->key = afs_file_key(file);
req->pos = pos;
req->len = len;
req->nr_pages = 1;
@@ -56,7 +58,7 @@ static int afs_fill_page(struct afs_vnode *vnode, struct key *key,
req->pages[0] = page;
get_page(page);
- ret = afs_fetch_data(vnode, key, req);
+ ret = afs_fetch_data(vnode, req);
afs_put_read(req);
if (ret < 0) {
if (ret == -ENOENT) {
@@ -80,7 +82,6 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
{
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
struct page *page;
- struct key *key = afs_file_key(file);
unsigned long priv;
unsigned f, from = pos & (PAGE_SIZE - 1);
unsigned t, to = from + len;
@@ -100,7 +101,7 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
return -ENOMEM;
if (!PageUptodate(page) && len != PAGE_SIZE) {
- ret = afs_fill_page(vnode, key, pos & PAGE_MASK, PAGE_SIZE, page);
+ ret = afs_fill_page(file, pos & PAGE_MASK, PAGE_SIZE, page);
if (ret < 0) {
unlock_page(page);
put_page(page);
@@ -183,7 +184,6 @@ int afs_write_end(struct file *file, struct address_space *mapping,
struct page *page, void *fsdata)
{
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
- struct key *key = afs_file_key(file);
loff_t i_size, maybe_i_size;
int ret;
@@ -207,7 +207,7 @@ int afs_write_end(struct file *file, struct address_space *mapping,
* unmarshalling routine will take care of clearing any
* bits that are beyond the EOF.
*/
- ret = afs_fill_page(vnode, key, pos + copied,
+ ret = afs_fill_page(file, pos + copied,
len - copied, page);
if (ret < 0)
goto out;
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 49/61] afs: Don't truncate iter during data fetch
2020-05-04 17:07 ` David Howells
` (48 preceding siblings ...)
(?)
@ 2020-05-04 17:14 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:14 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Don't truncate the iterator to correspond to the actual data size when
fetching the data from the server - rather, pass the length we want to read
to rxrpc.
This will allow the clear-after-read code in future to simply clear the
remaining iterator capacity rather than having to reinitialise the
iterator.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/fsclient.c | 6 ++++--
fs/afs/internal.h | 6 ++++++
fs/afs/rxrpc.c | 13 +++++++++----
fs/afs/yfsclient.c | 6 ++++--
include/net/af_rxrpc.h | 2 +-
net/rxrpc/recvmsg.c | 9 +++++----
6 files changed, 29 insertions(+), 13 deletions(-)
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 68fc46634346..492fbd964576 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -327,8 +327,9 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
unsigned int size;
int ret;
- _enter("{%u,%zu/%llu}",
- call->unmarshall, iov_iter_count(call->iter), req->actual_len);
+ _enter("{%u,%zu,%zu/%llu}",
+ call->unmarshall, call->iov_len, iov_iter_count(call->iter),
+ req->actual_len);
switch (call->unmarshall) {
case 0:
@@ -365,6 +366,7 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
size = PAGE_SIZE - req->offset;
else
size = req->remain;
+ call->iov_len = size;
call->bvec[0].bv_len = size;
call->bvec[0].bv_offset = req->offset;
call->bvec[0].bv_page = req->pages[req->index];
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 68d3eb0dc367..521a03023112 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -115,6 +115,7 @@ struct afs_call {
struct afs_vnode *lvnode; /* vnode being locked */
void *request; /* request data (first part) */
struct address_space *mapping; /* Pages being written from */
+ size_t iov_len; /* Size of *iter to be used */
struct iov_iter def_iter; /* Default buffer/data iterator */
struct iov_iter *iter; /* Iterator currently in use */
union { /* Convenience for ->def_iter */
@@ -1131,6 +1132,7 @@ static inline void afs_set_fc_call(struct afs_call *call, struct afs_fs_cursor *
static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)
{
+ call->iov_len = size;
call->kvec[0].iov_base = buf;
call->kvec[0].iov_len = size;
iov_iter_kvec(&call->def_iter, READ, call->kvec, 1, size);
@@ -1138,21 +1140,25 @@ static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t si
static inline void afs_extract_to_tmp(struct afs_call *call)
{
+ call->iov_len = sizeof(call->tmp);
afs_extract_begin(call, &call->tmp, sizeof(call->tmp));
}
static inline void afs_extract_to_tmp64(struct afs_call *call)
{
+ call->iov_len = sizeof(call->tmp64);
afs_extract_begin(call, &call->tmp64, sizeof(call->tmp64));
}
static inline void afs_extract_discard(struct afs_call *call, size_t size)
{
+ call->iov_len = size;
iov_iter_discard(&call->def_iter, READ, size);
}
static inline void afs_extract_to_buf(struct afs_call *call, size_t size)
{
+ call->iov_len = size;
afs_extract_begin(call, call->buffer, size);
}
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index 1ecc67da6c1a..b2296feaaff3 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -364,6 +364,7 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
struct rxrpc_call *rxcall;
struct msghdr msg;
struct kvec iov[1];
+ size_t len;
s64 tx_total_len;
int ret;
@@ -465,9 +466,10 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
rxrpc_kernel_abort_call(call->net->socket, rxcall,
RX_USER_ABORT, ret, "KSD");
} else {
+ len = 0;
iov_iter_kvec(&msg.msg_iter, READ, NULL, 0, 0);
rxrpc_kernel_recv_data(call->net->socket, rxcall,
- &msg.msg_iter, false,
+ &msg.msg_iter, &len, false,
&call->abort_code, &call->service_id);
ac->abort_code = call->abort_code;
ac->responded = true;
@@ -503,6 +505,7 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
static void afs_deliver_to_call(struct afs_call *call)
{
enum afs_call_state state;
+ size_t len;
u32 abort_code, remote_abort = 0;
int ret;
@@ -515,10 +518,11 @@ static void afs_deliver_to_call(struct afs_call *call)
state == AFS_CALL_SV_AWAIT_ACK
) {
if (state == AFS_CALL_SV_AWAIT_ACK) {
+ len = 0;
iov_iter_kvec(&call->def_iter, READ, NULL, 0, 0);
ret = rxrpc_kernel_recv_data(call->net->socket,
call->rxcall, &call->def_iter,
- false, &remote_abort,
+ &len, false, &remote_abort,
&call->service_id);
trace_afs_receive_data(call, &call->def_iter, false, ret);
@@ -926,10 +930,11 @@ int afs_extract_data(struct afs_call *call, bool want_more)
u32 remote_abort = 0;
int ret;
- _enter("{%s,%zu},%d", call->type->name, iov_iter_count(iter), want_more);
+ _enter("{%s,%zu,%zu},%d",
+ call->type->name, call->iov_len, iov_iter_count(iter), want_more);
ret = rxrpc_kernel_recv_data(net->socket, call->rxcall, iter,
- want_more, &remote_abort,
+ &call->iov_len, want_more, &remote_abort,
&call->service_id);
if (ret == 0 || ret == -EAGAIN)
return ret;
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 7f419f8bb943..1164f48e308d 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -442,8 +442,9 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
unsigned int size;
int ret;
- _enter("{%u,%zu/%llu}",
- call->unmarshall, iov_iter_count(call->iter), req->actual_len);
+ _enter("{%u,%zu, %zu/%llu}",
+ call->unmarshall, call->iov_len, iov_iter_count(call->iter),
+ req->actual_len);
switch (call->unmarshall) {
case 0:
@@ -475,6 +476,7 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
size = PAGE_SIZE - req->offset;
else
size = req->remain;
+ call->iov_len = size;
call->bvec[0].bv_len = size;
call->bvec[0].bv_offset = req->offset;
call->bvec[0].bv_page = req->pages[req->index];
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index 04e97bab6f28..0f7d13547b77 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -53,7 +53,7 @@ int rxrpc_kernel_send_data(struct socket *, struct rxrpc_call *,
struct msghdr *, size_t,
rxrpc_notify_end_tx_t);
int rxrpc_kernel_recv_data(struct socket *, struct rxrpc_call *,
- struct iov_iter *, bool, u32 *, u16 *);
+ struct iov_iter *, size_t *, bool, u32 *, u16 *);
bool rxrpc_kernel_abort_call(struct socket *, struct rxrpc_call *,
u32, int, const char *);
void rxrpc_kernel_end_call(struct socket *, struct rxrpc_call *);
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 8578c39ec839..802b712f3d79 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -624,6 +624,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
* @sock: The socket that the call exists on
* @call: The call to send data through
* @iter: The buffer to receive into
+ * @_len: The amount of data we want to receive (decreased on return)
* @want_more: True if more data is expected to be read
* @_abort: Where the abort code is stored if -ECONNABORTED is returned
* @_service: Where to store the actual service ID (may be upgraded)
@@ -639,7 +640,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
* *_abort should also be initialised to 0.
*/
int rxrpc_kernel_recv_data(struct socket *sock, struct rxrpc_call *call,
- struct iov_iter *iter,
+ struct iov_iter *iter, size_t *_len,
bool want_more, u32 *_abort, u16 *_service)
{
size_t offset = 0;
@@ -647,7 +648,7 @@ int rxrpc_kernel_recv_data(struct socket *sock, struct rxrpc_call *call,
_enter("{%d,%s},%zu,%d",
call->debug_id, rxrpc_call_states[call->state],
- iov_iter_count(iter), want_more);
+ *_len, want_more);
ASSERTCMP(call->state, !=, RXRPC_CALL_SERVER_ACCEPTING);
@@ -658,8 +659,8 @@ int rxrpc_kernel_recv_data(struct socket *sock, struct rxrpc_call *call,
case RXRPC_CALL_SERVER_RECV_REQUEST:
case RXRPC_CALL_SERVER_ACK_REQUEST:
ret = rxrpc_recvmsg_data(sock, call, NULL, iter,
- iov_iter_count(iter), 0,
- &offset);
+ *_len, 0, &offset);
+ *_len -= offset;
if (ret < 0)
goto out;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 50/61] afs: Set up the iov_iter before calling afs_extract_data()
@ 2020-05-04 17:15 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
afs_extract_data sets up a temporary iov_iter and passes it to AF_RXRPC
each time it is called to describe the remaining buffer to be filled.
Instead:
(1) Put an iterator in the afs_call struct.
(2) Set the iterator for each marshalling stage to load data into the
appropriate places. A number of convenience functions are provided to
this end (eg. afs_extract_to_buf()).
This iterator is then passed to afs_extract_data().
(3) Use the new ITER_MAPPING iterator when reading data to load directly
into the inode's pages without needing to create a list of them. This
comes with a page-done callback that can be used to unlock pages as
they are filled.
This will allow O_DIRECT calls to be supported in future patches.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/dir.c | 223 +++++++++++++++++++++++++++++++++++-----------------
fs/afs/file.c | 188 +++++++++++++++++++++++++-------------------
fs/afs/fsclient.c | 54 +++----------
fs/afs/internal.h | 16 ++--
fs/afs/write.c | 8 +-
fs/afs/yfsclient.c | 53 +++---------
6 files changed, 296 insertions(+), 246 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 9d8504885f6a..a10bcf632e0c 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -104,6 +104,35 @@ struct afs_lookup_cookie {
struct afs_fid fids[50];
};
+/*
+ * Drop the refs that we're holding on the pages we were reading into. We've
+ * got refs on the first nr_pages pages.
+ */
+static void afs_dir_read_cleanup(struct afs_read *req)
+{
+ struct address_space *mapping = req->iter->mapping;
+ struct page *page;
+ pgoff_t last = req->nr_pages - 1;
+
+ XA_STATE(xas, &mapping->i_pages, 0);
+
+ if (unlikely(!req->nr_pages))
+ return;
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+ ASSERTCMP(page->mapping, ==, mapping);
+
+ put_page(page);
+ }
+
+ rcu_read_unlock();
+}
+
/*
* check that a directory page is valid
*/
@@ -129,7 +158,7 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
qty /= sizeof(union afs_xdr_dir_block);
/* check them */
- dbuf = kmap(page);
+ dbuf = kmap_atomic(page);
for (tmp = 0; tmp < qty; tmp++) {
if (dbuf->blocks[tmp].hdr.magic != AFS_DIR_MAGIC) {
printk("kAFS: %s(%lx): bad magic %d/%d is %04hx\n",
@@ -148,7 +177,7 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
((u8 *)&dbuf->blocks[tmp])[AFS_DIR_BLOCK_SIZE - 1] = 0;
}
- kunmap(page);
+ kunmap_atomic(dbuf);
checked:
afs_stat_v(dvnode, n_read_dir);
@@ -159,35 +188,74 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
}
/*
- * Check the contents of a directory that we've just read.
+ * Dump the contents of a directory.
*/
-static bool afs_dir_check_pages(struct afs_vnode *dvnode, struct afs_read *req)
+static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
{
struct afs_xdr_dir_page *dbuf;
- unsigned int i, j, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
+ struct address_space *mapping = dvnode->vfs_inode.i_mapping;
+ struct page *page;
+ unsigned int i, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
+ pgoff_t last = req->nr_pages - 1;
- for (i = 0; i < req->nr_pages; i++)
- if (!afs_dir_check_page(dvnode, req->pages[i], req->actual_len))
- goto bad;
- return true;
+ XA_STATE(xas, &mapping->i_pages, 0);
-bad:
- pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx r=%llx\n",
+ pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode,
- req->file_size, req->len, req->actual_len, req->remain);
- pr_warn("DIR %llx %x %x %x\n",
- req->pos, req->index, req->nr_pages, req->offset);
+ req->file_size, req->len, req->actual_len);
+ pr_warn("DIR %llx %x %zx %zx\n",
+ req->pos, req->nr_pages,
+ req->iter->iov_offset, iov_iter_count(req->iter));
+
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
- for (i = 0; i < req->nr_pages; i++) {
- dbuf = kmap(req->pages[i]);
- for (j = 0; j < qty; j++) {
- union afs_xdr_dir_block *block = &dbuf->blocks[j];
+ BUG_ON(PageCompound(page));
+ BUG_ON(page->mapping != mapping);
- pr_warn("[%02x] %32phN\n", i * qty + j, block);
+ dbuf = kmap_atomic(page);
+ for (i = 0; i < qty; i++) {
+ union afs_xdr_dir_block *block = &dbuf->blocks[i];
+
+ pr_warn("[%02lx] %32phN\n", page->index * qty + i, block);
}
- kunmap(req->pages[i]);
+ kunmap_atomic(dbuf);
}
- return false;
+}
+
+/*
+ * Check all the pages in a directory. All the pages are held pinned.
+ */
+static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
+{
+ struct address_space *mapping = dvnode->vfs_inode.i_mapping;
+ struct page *page;
+ pgoff_t last = req->nr_pages - 1;
+ int ret = 0;
+
+ XA_STATE(xas, &mapping->i_pages, 0);
+
+ if (unlikely(!req->nr_pages))
+ return 0;
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
+
+ BUG_ON(PageCompound(page));
+ BUG_ON(page->mapping != mapping);
+
+ ret = afs_dir_check_page(dvnode, page, req->file_size);
+ if (ret < 0) {
+ afs_dir_dump(dvnode, req);
+ break;
+ }
+ }
+
+ rcu_read_unlock();
+ return ret;
}
/*
@@ -216,58 +284,56 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
{
struct afs_read *req;
loff_t i_size;
- int nr_pages, nr_inline, i, n;
- int ret = -ENOMEM;
+ int nr_pages, i, n;
+ int ret;
-retry:
+ _enter("");
+
+ req = kzalloc(sizeof(*req), GFP_KERNEL);
+ if (!req)
+ return ERR_PTR(-ENOMEM);
+
+ refcount_set(&req->usage, 1);
+ req->key = key_get(key);
+ req->cleanup = afs_dir_read_cleanup;
+
+expand:
i_size = i_size_read(&dvnode->vfs_inode);
- if (i_size < 2048)
- return ERR_PTR(afs_bad(dvnode, afs_file_error_dir_small));
+ if (i_size < 2048) {
+ ret = afs_bad(dvnode, afs_file_error_dir_small);
+ goto error;
+ }
if (i_size > 2048 * 1024) {
trace_afs_file_error(dvnode, -EFBIG, afs_file_error_dir_big);
- return ERR_PTR(-EFBIG);
+ ret = -EFBIG;
+ goto error;
}
_enter("%llu", i_size);
- /* Get a request record to hold the page list. We want to hold it
- * inline if we can, but we don't want to make an order 1 allocation.
- */
nr_pages = (i_size + PAGE_SIZE - 1) / PAGE_SIZE;
- nr_inline = nr_pages;
- if (nr_inline > (PAGE_SIZE - sizeof(*req)) / sizeof(struct page *))
- nr_inline = 0;
- req = kzalloc(struct_size(req, array, nr_inline), GFP_KERNEL);
- if (!req)
- return ERR_PTR(-ENOMEM);
-
- refcount_set(&req->usage, 1);
- req->key = key_get(key);
- req->nr_pages = nr_pages;
req->actual_len = i_size; /* May change */
req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
req->data_version = dvnode->status.data_version; /* May change */
- if (nr_inline > 0) {
- req->pages = req->array;
- } else {
- req->pages = kcalloc(nr_pages, sizeof(struct page *),
- GFP_KERNEL);
- if (!req->pages)
- goto error;
- }
+ iov_iter_mapping(&req->def_iter, READ, dvnode->vfs_inode.i_mapping,
+ 0, i_size);
+ req->iter = &req->def_iter;
- /* Get a list of all the pages that hold or will hold the directory
- * content. We need to fill in any gaps that we might find where the
- * memory reclaimer has been at work. If there are any gaps, we will
+ /* Fill in any gaps that we might find where the memory reclaimer has
+ * been at work and pin all the pages. If there are any gaps, we will
* need to reread the entire directory contents.
*/
- i = 0;
- do {
+ i = req->nr_pages;
+ while (i < nr_pages) {
+ struct page *pages[8], *page;
+
n = find_get_pages_contig(dvnode->vfs_inode.i_mapping, i,
- req->nr_pages - i,
- req->pages + i);
- _debug("find %u at %u/%u", n, i, req->nr_pages);
+ min_t(unsigned int, nr_pages - i,
+ ARRAY_SIZE(pages)),
+ pages);
+ _debug("find %u at %u/%u", n, i, nr_pages);
+
if (n == 0) {
gfp_t gfp = dvnode->vfs_inode.i_mapping->gfp_mask;
@@ -275,23 +341,25 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
afs_stat_v(dvnode, n_inval);
ret = -ENOMEM;
- req->pages[i] = __page_cache_alloc(gfp);
- if (!req->pages[i])
+ page = __page_cache_alloc(gfp);
+ if (!page)
goto error;
- ret = add_to_page_cache_lru(req->pages[i],
+ ret = add_to_page_cache_lru(page,
dvnode->vfs_inode.i_mapping,
i, gfp);
if (ret < 0)
goto error;
- set_page_private(req->pages[i], 1);
- SetPagePrivate(req->pages[i]);
- unlock_page(req->pages[i]);
+ set_page_private(page, 1);
+ SetPagePrivate(page);
+ unlock_page(page);
+ req->nr_pages++;
i++;
} else {
+ req->nr_pages += n;
i += n;
}
- } while (i < req->nr_pages);
+ }
/* If we're going to reload, we need to lock all the pages to prevent
* races.
@@ -315,12 +383,17 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
task_io_account_read(PAGE_SIZE * req->nr_pages);
- if (req->len < req->file_size)
- goto content_has_grown;
+ if (req->len < req->file_size) {
+ /* The content has grown, so we need to expand the
+ * buffer.
+ */
+ up_write(&dvnode->validate_lock);
+ goto expand;
+ }
/* Validate the data we just read. */
- ret = -EIO;
- if (!afs_dir_check_pages(dvnode, req))
+ ret = afs_dir_check(dvnode, req);
+ if (ret < 0)
goto error_unlock;
// TODO: Trim excess pages
@@ -338,11 +411,6 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
afs_put_read(req);
_leave(" = %d", ret);
return ERR_PTR(ret);
-
-content_has_grown:
- up_write(&dvnode->validate_lock);
- afs_put_read(req);
- goto retry;
}
/*
@@ -449,6 +517,7 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
struct afs_read *req;
struct page *page;
unsigned blkoff, limit;
+ void __rcu **slot;
int ret;
_enter("{%lu},%u,,", dir->i_ino, (unsigned)ctx->pos);
@@ -473,9 +542,15 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
blkoff = ctx->pos & ~(sizeof(union afs_xdr_dir_block) - 1);
/* Fetch the appropriate page from the directory and re-add it
- * to the LRU.
+ * to the LRU. We have all the pages pinned with an extra ref.
*/
- page = req->pages[blkoff / PAGE_SIZE];
+ rcu_read_lock();
+ page = NULL;
+ slot = radix_tree_lookup_slot(&dvnode->vfs_inode.i_mapping->i_pages,
+ blkoff / PAGE_SIZE);
+ if (slot)
+ page = radix_tree_deref_slot(slot);
+ rcu_read_unlock();
if (!page) {
ret = afs_bad(dvnode, afs_file_error_dir_missing_page);
break;
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 834f47c4dc94..c8ad638590e7 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -196,21 +196,70 @@ int afs_release(struct inode *inode, struct file *file)
return ret;
}
+/*
+ * Handle completion of a read operation.
+ */
+static void afs_file_read_done(struct afs_read *req)
+{
+ struct afs_vnode *vnode = req->vnode;
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
+
+ if (iov_iter_count(req->iter) > 0) {
+ /* The read was short - clear the excess buffer. */
+ _debug("afterclear %zx %zx %llx/%llx",
+ req->iter->iov_offset,
+ iov_iter_count(req->iter),
+ req->actual_len, req->len);
+ iov_iter_zero(iov_iter_count(req->iter), req->iter);
+ }
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ page_endio(page, false, 0);
+ put_page(page);
+ }
+ rcu_read_unlock();
+
+ task_io_account_read(req->len);
+ req->cleanup = NULL;
+}
+
+/*
+ * Dispose of our locks and refs on the pages if the read failed.
+ */
+static void afs_file_read_cleanup(struct afs_read *req)
+{
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->iter->mapping->i_pages, index);
+
+ _enter("%lu,%u,%zu", index, req->nr_pages, iov_iter_count(req->iter));
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ page_endio(page, false, req->error);
+ put_page(page);
+ }
+ rcu_read_unlock();
+}
+
/*
* Dispose of a ref to a read record.
*/
void afs_put_read(struct afs_read *req)
{
- int i;
-
if (refcount_dec_and_test(&req->usage)) {
- if (req->pages) {
- for (i = 0; i < req->nr_pages; i++)
- if (req->pages[i])
- put_page(req->pages[i]);
- if (req->pages != req->array)
- kfree(req->pages);
- }
+ if (req->cleanup)
+ req->cleanup(req);
key_put(req->key);
kfree(req);
}
@@ -251,6 +300,7 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
ret = afs_end_vnode_operation(&fc);
}
+ req->error = ret;
if (ret == 0) {
afs_stat_v(vnode, n_fetches);
atomic_long_add(req->actual_len,
@@ -265,12 +315,11 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
/*
* read page from file, directory or symlink, given a key to use
*/
-int afs_page_filler(void *data, struct page *page)
+static int afs_page_filler(struct key *key, struct page *page)
{
struct inode *inode = page->mapping->host;
struct afs_vnode *vnode = AFS_FS_I(inode);
struct afs_read *req;
- struct key *key = data;
int ret;
_enter("{%x},{%lu},{%lu}", key_serial(key), inode->i_ino, page->index);
@@ -281,53 +330,52 @@ int afs_page_filler(void *data, struct page *page)
if (test_bit(AFS_VNODE_DELETED, &vnode->flags))
goto error;
- req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
+ req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
if (!req)
goto enomem;
- /* We request a full page. If the page is a partial one at the
- * end of the file, the server will return a short read and the
- * unmarshalling code will clear the unfilled space.
- */
refcount_set(&req->usage, 1);
- req->key = key_get(key);
- req->pos = (loff_t)page->index << PAGE_SHIFT;
- req->len = PAGE_SIZE;
- req->nr_pages = 1;
- req->pages = req->array;
- req->pages[0] = page;
+ req->vnode = vnode;
+ req->key = key_get(key);
+ req->pos = (loff_t)page->index << PAGE_SHIFT;
+ req->len = PAGE_SIZE;
+ req->nr_pages = 1;
+ req->done = afs_file_read_done;
+ req->cleanup = afs_file_read_cleanup;
+
get_page(page);
+ iov_iter_mapping(&req->def_iter, READ, page->mapping,
+ req->pos, req->len);
+ req->iter = &req->def_iter;
- /* read the contents of the file from the server into the
- * page */
ret = afs_fetch_data(vnode, req);
- afs_put_read(req);
-
- if (ret < 0) {
- if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
- }
-
- if (ret == -EINTR ||
- ret == -ENOMEM ||
- ret == -ERESTARTSYS ||
- ret == -EAGAIN)
- goto error;
- goto io_error;
- }
-
- SetPageUptodate(page);
- unlock_page(page);
+ if (ret < 0)
+ goto fetch_error;
+ afs_put_read(req);
_leave(" = 0");
return 0;
-io_error:
- SetPageError(page);
- goto error;
+fetch_error:
+ switch (ret) {
+ case -EINTR:
+ case -ENOMEM:
+ case -ERESTARTSYS:
+ case -EAGAIN:
+ afs_put_read(req);
+ goto error;
+ case -ENOENT:
+ _debug("got NOENT from server - marking file deleted and stale");
+ set_bit(AFS_VNODE_DELETED, &vnode->flags);
+ ret = -ESTALE;
+ /* Fall through */
+ default:
+ page_endio(page, false, ret);
+ afs_put_read(req);
+ _leave(" = %d", ret);
+ return ret;
+ }
+
enomem:
ret = -ENOMEM;
error:
@@ -362,19 +410,6 @@ static int afs_readpage(struct file *file, struct page *page)
return ret;
}
-/*
- * Make pages available as they're filled.
- */
-static void afs_readpages_page_done(struct afs_read *req)
-{
- struct page *page = req->pages[req->index];
-
- req->pages[req->index] = NULL;
- SetPageUptodate(page);
- unlock_page(page);
- put_page(page);
-}
-
/*
* Read a contiguous set of pages.
*/
@@ -386,7 +421,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
struct list_head *p;
struct page *first, *page;
pgoff_t index;
- int ret, n, i;
+ int ret, n;
/* Count the number of contiguous pages at the front of the list. Note
* that the list goes prev-wards rather than next-wards.
@@ -402,21 +437,20 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
n++;
}
- req = kzalloc(struct_size(req, array, n), GFP_NOFS);
+ req = kzalloc(sizeof(struct afs_read), GFP_NOFS);
if (!req)
return -ENOMEM;
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(afs_file_key(file));
- req->page_done = afs_readpages_page_done;
+ req->done = afs_file_read_done;
+ req->cleanup = afs_file_read_cleanup;
req->pos = first->index;
req->pos <<= PAGE_SHIFT;
- req->pages = req->array;
- /* Transfer the pages to the request. We add them in until one fails
- * to add to the LRU and then we stop (as that'll make a hole in the
- * contiguous run.
+ /* Add pages to the LRU until it fails. We keep the pages ref'd and
+ * locked until the read is complete.
*
* Note that it's possible for the file size to change whilst we're
* doing this, but we rely on the server returning less than we asked
@@ -433,8 +467,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
break;
}
- req->pages[req->nr_pages++] = page;
- req->len += PAGE_SIZE;
+ req->nr_pages++;
} while (req->nr_pages < n);
if (req->nr_pages == 0) {
@@ -442,30 +475,25 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
return 0;
}
+ req->len = req->nr_pages * PAGE_SIZE;
+ iov_iter_mapping(&req->def_iter, READ, file->f_mapping,
+ req->pos, req->len);
+ req->iter = &req->def_iter;
+
ret = afs_fetch_data(vnode, req);
if (ret < 0)
goto error;
- task_io_account_read(PAGE_SIZE * req->nr_pages);
afs_put_read(req);
return 0;
error:
if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
+ _debug("got NOENT from server - marking file deleted and stale");
set_bit(AFS_VNODE_DELETED, &vnode->flags);
ret = -ESTALE;
}
- for (i = 0; i < req->nr_pages; i++) {
- page = req->pages[i];
- if (page) {
- SetPageError(page);
- unlock_page(page);
- }
- }
-
afs_put_read(req);
return ret;
}
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 492fbd964576..8222ccf01280 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -324,7 +324,6 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
{
struct afs_read *req = call->read_request;
const __be32 *bp;
- unsigned int size;
int ret;
_enter("{%u,%zu,%zu/%llu}",
@@ -334,8 +333,6 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
switch (call->unmarshall) {
case 0:
req->actual_len = 0;
- req->index = 0;
- req->offset = req->pos & (PAGE_SIZE - 1);
call->unmarshall++;
if (call->operation_ID == FSFETCHDATA64) {
afs_extract_to_tmp64(call);
@@ -343,9 +340,13 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
call->tmp_u = htonl(0);
afs_extract_to_tmp(call);
}
+
/* Fall through */
- /* extract the returned data length */
+ /* Extract the returned data length into
+ * ->actual_len. This may indicate more or less data than was
+ * requested will be returned.
+ */
case 1:
_debug("extract data length");
ret = afs_extract_data(call, true);
@@ -354,47 +355,25 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
req->actual_len = be64_to_cpu(call->tmp64);
_debug("DATA length: %llu", req->actual_len);
- req->remain = min(req->len, req->actual_len);
- if (req->remain == 0)
+
+ if (req->actual_len == 0)
goto no_more_data;
call->unmarshall++;
-
- begin_page:
- ASSERTCMP(req->index, <, req->nr_pages);
- if (req->remain > PAGE_SIZE - req->offset)
- size = PAGE_SIZE - req->offset;
- else
- size = req->remain;
- call->iov_len = size;
- call->bvec[0].bv_len = size;
- call->bvec[0].bv_offset = req->offset;
- call->bvec[0].bv_page = req->pages[req->index];
- iov_iter_bvec(&call->def_iter, READ, call->bvec, 1, size);
- ASSERTCMP(size, <=, PAGE_SIZE);
+ call->iter = req->iter;
+ call->iov_len = min(req->actual_len, req->len);
/* Fall through */
/* extract the returned data */
case 2:
_debug("extract data %zu/%llu",
- iov_iter_count(call->iter), req->remain);
+ iov_iter_count(call->iter), req->actual_len);
ret = afs_extract_data(call, true);
if (ret < 0)
return ret;
- req->remain -= call->bvec[0].bv_len;
- req->offset += call->bvec[0].bv_len;
- ASSERTCMP(req->offset, <=, PAGE_SIZE);
- if (req->offset == PAGE_SIZE) {
- req->offset = 0;
- if (req->page_done)
- req->page_done(req);
- req->index++;
- if (req->remain > 0)
- goto begin_page;
- }
- ASSERTCMP(req->remain, ==, 0);
+ call->iter = &call->def_iter;
if (req->actual_len <= req->len)
goto no_more_data;
@@ -438,14 +417,8 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
break;
}
- for (; req->index < req->nr_pages; req->index++) {
- if (req->offset < PAGE_SIZE)
- zero_user_segment(req->pages[req->index],
- req->offset, PAGE_SIZE);
- if (req->page_done)
- req->page_done(req);
- req->offset = 0;
- }
+ if (req->done)
+ req->done(req);
_leave(" = 0 [done]");
return 0;
@@ -547,6 +520,7 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
call->out_scb = scb;
call->out_volsync = NULL;
call->read_request = afs_get_read(req);
+ req->call_debug_id = call->debug_id;
/* marshall the parameters */
bp = call->request;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 521a03023112..e676ad145272 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -31,6 +31,7 @@
struct pagevec;
struct afs_call;
+struct afs_vnode;
/*
* Partial file-locking emulation mode. (The problem being that AFS3 only
@@ -226,18 +227,18 @@ struct afs_read {
loff_t pos; /* Where to start reading */
loff_t len; /* How much we're asking for */
loff_t actual_len; /* How much we're actually getting */
- loff_t remain; /* Amount remaining */
loff_t file_size; /* File size returned by server */
struct key *key; /* The key to use to reissue the read */
+ struct afs_vnode *vnode; /* The file being read into. */
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
- unsigned int index; /* Which page we're reading into */
+ unsigned int call_debug_id;
unsigned int nr_pages;
- unsigned int offset; /* offset into current page */
- struct afs_vnode *vnode;
- void (*page_done)(struct afs_read *);
- struct page **pages;
- struct page *array[];
+ int error;
+ void (*done)(struct afs_read *);
+ void (*cleanup)(struct afs_read *);
+ struct iov_iter *iter; /* Iterator representing the buffer */
+ struct iov_iter def_iter; /* Default iterator */
};
/*
@@ -924,7 +925,6 @@ extern void afs_put_wb_key(struct afs_wb_key *);
extern int afs_open(struct inode *, struct file *);
extern int afs_release(struct inode *, struct file *);
extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
-extern int afs_page_filler(void *, struct page *);
extern void afs_put_read(struct afs_read *);
static inline struct afs_read *afs_get_read(struct afs_read *req)
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 8473f9bc3548..174e355aee6d 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -45,7 +45,7 @@ static int afs_fill_page(struct file *file,
return 0;
}
- req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
+ req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
if (!req)
return -ENOMEM;
@@ -54,9 +54,9 @@ static int afs_fill_page(struct file *file,
req->pos = pos;
req->len = len;
req->nr_pages = 1;
- req->pages = req->array;
- req->pages[0] = page;
- get_page(page);
+ iov_iter_mapping(&req->def_iter, READ, vnode->vfs_inode.i_mapping,
+ pos, len);
+ req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
afs_put_read(req);
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 1164f48e308d..518b9489ff9e 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -439,7 +439,6 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
{
struct afs_read *req = call->read_request;
const __be32 *bp;
- unsigned int size;
int ret;
_enter("{%u,%zu, %zu/%llu}",
@@ -449,13 +448,15 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
switch (call->unmarshall) {
case 0:
req->actual_len = 0;
- req->index = 0;
- req->offset = req->pos & (PAGE_SIZE - 1);
afs_extract_to_tmp64(call);
call->unmarshall++;
+
/* Fall through */
- /* extract the returned data length */
+ /* Extract the returned data length into ->actual_len. This
+ * may indicate more or less data than was requested will be
+ * returned.
+ */
case 1:
_debug("extract data length");
ret = afs_extract_data(call, true);
@@ -464,47 +465,25 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
req->actual_len = be64_to_cpu(call->tmp64);
_debug("DATA length: %llu", req->actual_len);
- req->remain = min(req->len, req->actual_len);
- if (req->remain == 0)
+
+ if (req->actual_len == 0)
goto no_more_data;
call->unmarshall++;
-
- begin_page:
- ASSERTCMP(req->index, <, req->nr_pages);
- if (req->remain > PAGE_SIZE - req->offset)
- size = PAGE_SIZE - req->offset;
- else
- size = req->remain;
- call->iov_len = size;
- call->bvec[0].bv_len = size;
- call->bvec[0].bv_offset = req->offset;
- call->bvec[0].bv_page = req->pages[req->index];
- iov_iter_bvec(&call->def_iter, READ, call->bvec, 1, size);
- ASSERTCMP(size, <=, PAGE_SIZE);
+ call->iter = req->iter;
+ call->iov_len = min(req->actual_len, req->len);
/* Fall through */
/* extract the returned data */
case 2:
_debug("extract data %zu/%llu",
- iov_iter_count(call->iter), req->remain);
+ iov_iter_count(call->iter), req->actual_len);
ret = afs_extract_data(call, true);
if (ret < 0)
return ret;
- req->remain -= call->bvec[0].bv_len;
- req->offset += call->bvec[0].bv_len;
- ASSERTCMP(req->offset, <=, PAGE_SIZE);
- if (req->offset == PAGE_SIZE) {
- req->offset = 0;
- if (req->page_done)
- req->page_done(req);
- req->index++;
- if (req->remain > 0)
- goto begin_page;
- }
- ASSERTCMP(req->remain, ==, 0);
+ call->iter = &call->def_iter;
if (req->actual_len <= req->len)
goto no_more_data;
@@ -552,14 +531,8 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
break;
}
- for (; req->index < req->nr_pages; req->index++) {
- if (req->offset < PAGE_SIZE)
- zero_user_segment(req->pages[req->index],
- req->offset, PAGE_SIZE);
- if (req->page_done)
- req->page_done(req);
- req->offset = 0;
- }
+ if (req->done)
+ req->done(req);
_leave(" = 0 [done]");
return 0;
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 50/61] afs: Set up the iov_iter before calling afs_extract_data()
@ 2020-05-04 17:15 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
afs_extract_data sets up a temporary iov_iter and passes it to AF_RXRPC
each time it is called to describe the remaining buffer to be filled.
Instead:
(1) Put an iterator in the afs_call struct.
(2) Set the iterator for each marshalling stage to load data into the
appropriate places. A number of convenience functions are provided to
this end (eg. afs_extract_to_buf()).
This iterator is then passed to afs_extract_data().
(3) Use the new ITER_MAPPING iterator when reading data to load directly
into the inode's pages without needing to create a list of them. This
comes with a page-done callback that can be used to unlock pages as
they are filled.
This will allow O_DIRECT calls to be supported in future patches.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/afs/dir.c | 223 +++++++++++++++++++++++++++++++++++-----------------
fs/afs/file.c | 188 +++++++++++++++++++++++++-------------------
fs/afs/fsclient.c | 54 +++----------
fs/afs/internal.h | 16 ++--
fs/afs/write.c | 8 +-
fs/afs/yfsclient.c | 53 +++---------
6 files changed, 296 insertions(+), 246 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 9d8504885f6a..a10bcf632e0c 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -104,6 +104,35 @@ struct afs_lookup_cookie {
struct afs_fid fids[50];
};
+/*
+ * Drop the refs that we're holding on the pages we were reading into. We've
+ * got refs on the first nr_pages pages.
+ */
+static void afs_dir_read_cleanup(struct afs_read *req)
+{
+ struct address_space *mapping = req->iter->mapping;
+ struct page *page;
+ pgoff_t last = req->nr_pages - 1;
+
+ XA_STATE(xas, &mapping->i_pages, 0);
+
+ if (unlikely(!req->nr_pages))
+ return;
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+ ASSERTCMP(page->mapping, ==, mapping);
+
+ put_page(page);
+ }
+
+ rcu_read_unlock();
+}
+
/*
* check that a directory page is valid
*/
@@ -129,7 +158,7 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
qty /= sizeof(union afs_xdr_dir_block);
/* check them */
- dbuf = kmap(page);
+ dbuf = kmap_atomic(page);
for (tmp = 0; tmp < qty; tmp++) {
if (dbuf->blocks[tmp].hdr.magic != AFS_DIR_MAGIC) {
printk("kAFS: %s(%lx): bad magic %d/%d is %04hx\n",
@@ -148,7 +177,7 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
((u8 *)&dbuf->blocks[tmp])[AFS_DIR_BLOCK_SIZE - 1] = 0;
}
- kunmap(page);
+ kunmap_atomic(dbuf);
checked:
afs_stat_v(dvnode, n_read_dir);
@@ -159,35 +188,74 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
}
/*
- * Check the contents of a directory that we've just read.
+ * Dump the contents of a directory.
*/
-static bool afs_dir_check_pages(struct afs_vnode *dvnode, struct afs_read *req)
+static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
{
struct afs_xdr_dir_page *dbuf;
- unsigned int i, j, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
+ struct address_space *mapping = dvnode->vfs_inode.i_mapping;
+ struct page *page;
+ unsigned int i, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
+ pgoff_t last = req->nr_pages - 1;
- for (i = 0; i < req->nr_pages; i++)
- if (!afs_dir_check_page(dvnode, req->pages[i], req->actual_len))
- goto bad;
- return true;
+ XA_STATE(xas, &mapping->i_pages, 0);
-bad:
- pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx r=%llx\n",
+ pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode,
- req->file_size, req->len, req->actual_len, req->remain);
- pr_warn("DIR %llx %x %x %x\n",
- req->pos, req->index, req->nr_pages, req->offset);
+ req->file_size, req->len, req->actual_len);
+ pr_warn("DIR %llx %x %zx %zx\n",
+ req->pos, req->nr_pages,
+ req->iter->iov_offset, iov_iter_count(req->iter));
+
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
- for (i = 0; i < req->nr_pages; i++) {
- dbuf = kmap(req->pages[i]);
- for (j = 0; j < qty; j++) {
- union afs_xdr_dir_block *block = &dbuf->blocks[j];
+ BUG_ON(PageCompound(page));
+ BUG_ON(page->mapping != mapping);
- pr_warn("[%02x] %32phN\n", i * qty + j, block);
+ dbuf = kmap_atomic(page);
+ for (i = 0; i < qty; i++) {
+ union afs_xdr_dir_block *block = &dbuf->blocks[i];
+
+ pr_warn("[%02lx] %32phN\n", page->index * qty + i, block);
}
- kunmap(req->pages[i]);
+ kunmap_atomic(dbuf);
}
- return false;
+}
+
+/*
+ * Check all the pages in a directory. All the pages are held pinned.
+ */
+static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
+{
+ struct address_space *mapping = dvnode->vfs_inode.i_mapping;
+ struct page *page;
+ pgoff_t last = req->nr_pages - 1;
+ int ret = 0;
+
+ XA_STATE(xas, &mapping->i_pages, 0);
+
+ if (unlikely(!req->nr_pages))
+ return 0;
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ if (xas_retry(&xas, page))
+ continue;
+
+ BUG_ON(PageCompound(page));
+ BUG_ON(page->mapping != mapping);
+
+ ret = afs_dir_check_page(dvnode, page, req->file_size);
+ if (ret < 0) {
+ afs_dir_dump(dvnode, req);
+ break;
+ }
+ }
+
+ rcu_read_unlock();
+ return ret;
}
/*
@@ -216,58 +284,56 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
{
struct afs_read *req;
loff_t i_size;
- int nr_pages, nr_inline, i, n;
- int ret = -ENOMEM;
+ int nr_pages, i, n;
+ int ret;
-retry:
+ _enter("");
+
+ req = kzalloc(sizeof(*req), GFP_KERNEL);
+ if (!req)
+ return ERR_PTR(-ENOMEM);
+
+ refcount_set(&req->usage, 1);
+ req->key = key_get(key);
+ req->cleanup = afs_dir_read_cleanup;
+
+expand:
i_size = i_size_read(&dvnode->vfs_inode);
- if (i_size < 2048)
- return ERR_PTR(afs_bad(dvnode, afs_file_error_dir_small));
+ if (i_size < 2048) {
+ ret = afs_bad(dvnode, afs_file_error_dir_small);
+ goto error;
+ }
if (i_size > 2048 * 1024) {
trace_afs_file_error(dvnode, -EFBIG, afs_file_error_dir_big);
- return ERR_PTR(-EFBIG);
+ ret = -EFBIG;
+ goto error;
}
_enter("%llu", i_size);
- /* Get a request record to hold the page list. We want to hold it
- * inline if we can, but we don't want to make an order 1 allocation.
- */
nr_pages = (i_size + PAGE_SIZE - 1) / PAGE_SIZE;
- nr_inline = nr_pages;
- if (nr_inline > (PAGE_SIZE - sizeof(*req)) / sizeof(struct page *))
- nr_inline = 0;
- req = kzalloc(struct_size(req, array, nr_inline), GFP_KERNEL);
- if (!req)
- return ERR_PTR(-ENOMEM);
-
- refcount_set(&req->usage, 1);
- req->key = key_get(key);
- req->nr_pages = nr_pages;
req->actual_len = i_size; /* May change */
req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
req->data_version = dvnode->status.data_version; /* May change */
- if (nr_inline > 0) {
- req->pages = req->array;
- } else {
- req->pages = kcalloc(nr_pages, sizeof(struct page *),
- GFP_KERNEL);
- if (!req->pages)
- goto error;
- }
+ iov_iter_mapping(&req->def_iter, READ, dvnode->vfs_inode.i_mapping,
+ 0, i_size);
+ req->iter = &req->def_iter;
- /* Get a list of all the pages that hold or will hold the directory
- * content. We need to fill in any gaps that we might find where the
- * memory reclaimer has been at work. If there are any gaps, we will
+ /* Fill in any gaps that we might find where the memory reclaimer has
+ * been at work and pin all the pages. If there are any gaps, we will
* need to reread the entire directory contents.
*/
- i = 0;
- do {
+ i = req->nr_pages;
+ while (i < nr_pages) {
+ struct page *pages[8], *page;
+
n = find_get_pages_contig(dvnode->vfs_inode.i_mapping, i,
- req->nr_pages - i,
- req->pages + i);
- _debug("find %u at %u/%u", n, i, req->nr_pages);
+ min_t(unsigned int, nr_pages - i,
+ ARRAY_SIZE(pages)),
+ pages);
+ _debug("find %u at %u/%u", n, i, nr_pages);
+
if (n == 0) {
gfp_t gfp = dvnode->vfs_inode.i_mapping->gfp_mask;
@@ -275,23 +341,25 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
afs_stat_v(dvnode, n_inval);
ret = -ENOMEM;
- req->pages[i] = __page_cache_alloc(gfp);
- if (!req->pages[i])
+ page = __page_cache_alloc(gfp);
+ if (!page)
goto error;
- ret = add_to_page_cache_lru(req->pages[i],
+ ret = add_to_page_cache_lru(page,
dvnode->vfs_inode.i_mapping,
i, gfp);
if (ret < 0)
goto error;
- set_page_private(req->pages[i], 1);
- SetPagePrivate(req->pages[i]);
- unlock_page(req->pages[i]);
+ set_page_private(page, 1);
+ SetPagePrivate(page);
+ unlock_page(page);
+ req->nr_pages++;
i++;
} else {
+ req->nr_pages += n;
i += n;
}
- } while (i < req->nr_pages);
+ }
/* If we're going to reload, we need to lock all the pages to prevent
* races.
@@ -315,12 +383,17 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
task_io_account_read(PAGE_SIZE * req->nr_pages);
- if (req->len < req->file_size)
- goto content_has_grown;
+ if (req->len < req->file_size) {
+ /* The content has grown, so we need to expand the
+ * buffer.
+ */
+ up_write(&dvnode->validate_lock);
+ goto expand;
+ }
/* Validate the data we just read. */
- ret = -EIO;
- if (!afs_dir_check_pages(dvnode, req))
+ ret = afs_dir_check(dvnode, req);
+ if (ret < 0)
goto error_unlock;
// TODO: Trim excess pages
@@ -338,11 +411,6 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
afs_put_read(req);
_leave(" = %d", ret);
return ERR_PTR(ret);
-
-content_has_grown:
- up_write(&dvnode->validate_lock);
- afs_put_read(req);
- goto retry;
}
/*
@@ -449,6 +517,7 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
struct afs_read *req;
struct page *page;
unsigned blkoff, limit;
+ void __rcu **slot;
int ret;
_enter("{%lu},%u,,", dir->i_ino, (unsigned)ctx->pos);
@@ -473,9 +542,15 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
blkoff = ctx->pos & ~(sizeof(union afs_xdr_dir_block) - 1);
/* Fetch the appropriate page from the directory and re-add it
- * to the LRU.
+ * to the LRU. We have all the pages pinned with an extra ref.
*/
- page = req->pages[blkoff / PAGE_SIZE];
+ rcu_read_lock();
+ page = NULL;
+ slot = radix_tree_lookup_slot(&dvnode->vfs_inode.i_mapping->i_pages,
+ blkoff / PAGE_SIZE);
+ if (slot)
+ page = radix_tree_deref_slot(slot);
+ rcu_read_unlock();
if (!page) {
ret = afs_bad(dvnode, afs_file_error_dir_missing_page);
break;
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 834f47c4dc94..c8ad638590e7 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -196,21 +196,70 @@ int afs_release(struct inode *inode, struct file *file)
return ret;
}
+/*
+ * Handle completion of a read operation.
+ */
+static void afs_file_read_done(struct afs_read *req)
+{
+ struct afs_vnode *vnode = req->vnode;
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
+
+ if (iov_iter_count(req->iter) > 0) {
+ /* The read was short - clear the excess buffer. */
+ _debug("afterclear %zx %zx %llx/%llx",
+ req->iter->iov_offset,
+ iov_iter_count(req->iter),
+ req->actual_len, req->len);
+ iov_iter_zero(iov_iter_count(req->iter), req->iter);
+ }
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ page_endio(page, false, 0);
+ put_page(page);
+ }
+ rcu_read_unlock();
+
+ task_io_account_read(req->len);
+ req->cleanup = NULL;
+}
+
+/*
+ * Dispose of our locks and refs on the pages if the read failed.
+ */
+static void afs_file_read_cleanup(struct afs_read *req)
+{
+ struct page *page;
+ pgoff_t index = req->pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->nr_pages - 1;
+
+ XA_STATE(xas, &req->iter->mapping->i_pages, index);
+
+ _enter("%lu,%u,%zu", index, req->nr_pages, iov_iter_count(req->iter));
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
+
+ page_endio(page, false, req->error);
+ put_page(page);
+ }
+ rcu_read_unlock();
+}
+
/*
* Dispose of a ref to a read record.
*/
void afs_put_read(struct afs_read *req)
{
- int i;
-
if (refcount_dec_and_test(&req->usage)) {
- if (req->pages) {
- for (i = 0; i < req->nr_pages; i++)
- if (req->pages[i])
- put_page(req->pages[i]);
- if (req->pages != req->array)
- kfree(req->pages);
- }
+ if (req->cleanup)
+ req->cleanup(req);
key_put(req->key);
kfree(req);
}
@@ -251,6 +300,7 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
ret = afs_end_vnode_operation(&fc);
}
+ req->error = ret;
if (ret == 0) {
afs_stat_v(vnode, n_fetches);
atomic_long_add(req->actual_len,
@@ -265,12 +315,11 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
/*
* read page from file, directory or symlink, given a key to use
*/
-int afs_page_filler(void *data, struct page *page)
+static int afs_page_filler(struct key *key, struct page *page)
{
struct inode *inode = page->mapping->host;
struct afs_vnode *vnode = AFS_FS_I(inode);
struct afs_read *req;
- struct key *key = data;
int ret;
_enter("{%x},{%lu},{%lu}", key_serial(key), inode->i_ino, page->index);
@@ -281,53 +330,52 @@ int afs_page_filler(void *data, struct page *page)
if (test_bit(AFS_VNODE_DELETED, &vnode->flags))
goto error;
- req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
+ req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
if (!req)
goto enomem;
- /* We request a full page. If the page is a partial one at the
- * end of the file, the server will return a short read and the
- * unmarshalling code will clear the unfilled space.
- */
refcount_set(&req->usage, 1);
- req->key = key_get(key);
- req->pos = (loff_t)page->index << PAGE_SHIFT;
- req->len = PAGE_SIZE;
- req->nr_pages = 1;
- req->pages = req->array;
- req->pages[0] = page;
+ req->vnode = vnode;
+ req->key = key_get(key);
+ req->pos = (loff_t)page->index << PAGE_SHIFT;
+ req->len = PAGE_SIZE;
+ req->nr_pages = 1;
+ req->done = afs_file_read_done;
+ req->cleanup = afs_file_read_cleanup;
+
get_page(page);
+ iov_iter_mapping(&req->def_iter, READ, page->mapping,
+ req->pos, req->len);
+ req->iter = &req->def_iter;
- /* read the contents of the file from the server into the
- * page */
ret = afs_fetch_data(vnode, req);
- afs_put_read(req);
-
- if (ret < 0) {
- if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
- }
-
- if (ret == -EINTR ||
- ret == -ENOMEM ||
- ret == -ERESTARTSYS ||
- ret == -EAGAIN)
- goto error;
- goto io_error;
- }
-
- SetPageUptodate(page);
- unlock_page(page);
+ if (ret < 0)
+ goto fetch_error;
+ afs_put_read(req);
_leave(" = 0");
return 0;
-io_error:
- SetPageError(page);
- goto error;
+fetch_error:
+ switch (ret) {
+ case -EINTR:
+ case -ENOMEM:
+ case -ERESTARTSYS:
+ case -EAGAIN:
+ afs_put_read(req);
+ goto error;
+ case -ENOENT:
+ _debug("got NOENT from server - marking file deleted and stale");
+ set_bit(AFS_VNODE_DELETED, &vnode->flags);
+ ret = -ESTALE;
+ /* Fall through */
+ default:
+ page_endio(page, false, ret);
+ afs_put_read(req);
+ _leave(" = %d", ret);
+ return ret;
+ }
+
enomem:
ret = -ENOMEM;
error:
@@ -362,19 +410,6 @@ static int afs_readpage(struct file *file, struct page *page)
return ret;
}
-/*
- * Make pages available as they're filled.
- */
-static void afs_readpages_page_done(struct afs_read *req)
-{
- struct page *page = req->pages[req->index];
-
- req->pages[req->index] = NULL;
- SetPageUptodate(page);
- unlock_page(page);
- put_page(page);
-}
-
/*
* Read a contiguous set of pages.
*/
@@ -386,7 +421,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
struct list_head *p;
struct page *first, *page;
pgoff_t index;
- int ret, n, i;
+ int ret, n;
/* Count the number of contiguous pages at the front of the list. Note
* that the list goes prev-wards rather than next-wards.
@@ -402,21 +437,20 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
n++;
}
- req = kzalloc(struct_size(req, array, n), GFP_NOFS);
+ req = kzalloc(sizeof(struct afs_read), GFP_NOFS);
if (!req)
return -ENOMEM;
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(afs_file_key(file));
- req->page_done = afs_readpages_page_done;
+ req->done = afs_file_read_done;
+ req->cleanup = afs_file_read_cleanup;
req->pos = first->index;
req->pos <<= PAGE_SHIFT;
- req->pages = req->array;
- /* Transfer the pages to the request. We add them in until one fails
- * to add to the LRU and then we stop (as that'll make a hole in the
- * contiguous run.
+ /* Add pages to the LRU until it fails. We keep the pages ref'd and
+ * locked until the read is complete.
*
* Note that it's possible for the file size to change whilst we're
* doing this, but we rely on the server returning less than we asked
@@ -433,8 +467,7 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
break;
}
- req->pages[req->nr_pages++] = page;
- req->len += PAGE_SIZE;
+ req->nr_pages++;
} while (req->nr_pages < n);
if (req->nr_pages == 0) {
@@ -442,30 +475,25 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
return 0;
}
+ req->len = req->nr_pages * PAGE_SIZE;
+ iov_iter_mapping(&req->def_iter, READ, file->f_mapping,
+ req->pos, req->len);
+ req->iter = &req->def_iter;
+
ret = afs_fetch_data(vnode, req);
if (ret < 0)
goto error;
- task_io_account_read(PAGE_SIZE * req->nr_pages);
afs_put_read(req);
return 0;
error:
if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
+ _debug("got NOENT from server - marking file deleted and stale");
set_bit(AFS_VNODE_DELETED, &vnode->flags);
ret = -ESTALE;
}
- for (i = 0; i < req->nr_pages; i++) {
- page = req->pages[i];
- if (page) {
- SetPageError(page);
- unlock_page(page);
- }
- }
-
afs_put_read(req);
return ret;
}
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 492fbd964576..8222ccf01280 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -324,7 +324,6 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
{
struct afs_read *req = call->read_request;
const __be32 *bp;
- unsigned int size;
int ret;
_enter("{%u,%zu,%zu/%llu}",
@@ -334,8 +333,6 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
switch (call->unmarshall) {
case 0:
req->actual_len = 0;
- req->index = 0;
- req->offset = req->pos & (PAGE_SIZE - 1);
call->unmarshall++;
if (call->operation_ID == FSFETCHDATA64) {
afs_extract_to_tmp64(call);
@@ -343,9 +340,13 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
call->tmp_u = htonl(0);
afs_extract_to_tmp(call);
}
+
/* Fall through */
- /* extract the returned data length */
+ /* Extract the returned data length into
+ * ->actual_len. This may indicate more or less data than was
+ * requested will be returned.
+ */
case 1:
_debug("extract data length");
ret = afs_extract_data(call, true);
@@ -354,47 +355,25 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
req->actual_len = be64_to_cpu(call->tmp64);
_debug("DATA length: %llu", req->actual_len);
- req->remain = min(req->len, req->actual_len);
- if (req->remain == 0)
+
+ if (req->actual_len == 0)
goto no_more_data;
call->unmarshall++;
-
- begin_page:
- ASSERTCMP(req->index, <, req->nr_pages);
- if (req->remain > PAGE_SIZE - req->offset)
- size = PAGE_SIZE - req->offset;
- else
- size = req->remain;
- call->iov_len = size;
- call->bvec[0].bv_len = size;
- call->bvec[0].bv_offset = req->offset;
- call->bvec[0].bv_page = req->pages[req->index];
- iov_iter_bvec(&call->def_iter, READ, call->bvec, 1, size);
- ASSERTCMP(size, <=, PAGE_SIZE);
+ call->iter = req->iter;
+ call->iov_len = min(req->actual_len, req->len);
/* Fall through */
/* extract the returned data */
case 2:
_debug("extract data %zu/%llu",
- iov_iter_count(call->iter), req->remain);
+ iov_iter_count(call->iter), req->actual_len);
ret = afs_extract_data(call, true);
if (ret < 0)
return ret;
- req->remain -= call->bvec[0].bv_len;
- req->offset += call->bvec[0].bv_len;
- ASSERTCMP(req->offset, <=, PAGE_SIZE);
- if (req->offset == PAGE_SIZE) {
- req->offset = 0;
- if (req->page_done)
- req->page_done(req);
- req->index++;
- if (req->remain > 0)
- goto begin_page;
- }
- ASSERTCMP(req->remain, ==, 0);
+ call->iter = &call->def_iter;
if (req->actual_len <= req->len)
goto no_more_data;
@@ -438,14 +417,8 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
break;
}
- for (; req->index < req->nr_pages; req->index++) {
- if (req->offset < PAGE_SIZE)
- zero_user_segment(req->pages[req->index],
- req->offset, PAGE_SIZE);
- if (req->page_done)
- req->page_done(req);
- req->offset = 0;
- }
+ if (req->done)
+ req->done(req);
_leave(" = 0 [done]");
return 0;
@@ -547,6 +520,7 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
call->out_scb = scb;
call->out_volsync = NULL;
call->read_request = afs_get_read(req);
+ req->call_debug_id = call->debug_id;
/* marshall the parameters */
bp = call->request;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 521a03023112..e676ad145272 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -31,6 +31,7 @@
struct pagevec;
struct afs_call;
+struct afs_vnode;
/*
* Partial file-locking emulation mode. (The problem being that AFS3 only
@@ -226,18 +227,18 @@ struct afs_read {
loff_t pos; /* Where to start reading */
loff_t len; /* How much we're asking for */
loff_t actual_len; /* How much we're actually getting */
- loff_t remain; /* Amount remaining */
loff_t file_size; /* File size returned by server */
struct key *key; /* The key to use to reissue the read */
+ struct afs_vnode *vnode; /* The file being read into. */
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
- unsigned int index; /* Which page we're reading into */
+ unsigned int call_debug_id;
unsigned int nr_pages;
- unsigned int offset; /* offset into current page */
- struct afs_vnode *vnode;
- void (*page_done)(struct afs_read *);
- struct page **pages;
- struct page *array[];
+ int error;
+ void (*done)(struct afs_read *);
+ void (*cleanup)(struct afs_read *);
+ struct iov_iter *iter; /* Iterator representing the buffer */
+ struct iov_iter def_iter; /* Default iterator */
};
/*
@@ -924,7 +925,6 @@ extern void afs_put_wb_key(struct afs_wb_key *);
extern int afs_open(struct inode *, struct file *);
extern int afs_release(struct inode *, struct file *);
extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
-extern int afs_page_filler(void *, struct page *);
extern void afs_put_read(struct afs_read *);
static inline struct afs_read *afs_get_read(struct afs_read *req)
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 8473f9bc3548..174e355aee6d 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -45,7 +45,7 @@ static int afs_fill_page(struct file *file,
return 0;
}
- req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
+ req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
if (!req)
return -ENOMEM;
@@ -54,9 +54,9 @@ static int afs_fill_page(struct file *file,
req->pos = pos;
req->len = len;
req->nr_pages = 1;
- req->pages = req->array;
- req->pages[0] = page;
- get_page(page);
+ iov_iter_mapping(&req->def_iter, READ, vnode->vfs_inode.i_mapping,
+ pos, len);
+ req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
afs_put_read(req);
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 1164f48e308d..518b9489ff9e 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -439,7 +439,6 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
{
struct afs_read *req = call->read_request;
const __be32 *bp;
- unsigned int size;
int ret;
_enter("{%u,%zu, %zu/%llu}",
@@ -449,13 +448,15 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
switch (call->unmarshall) {
case 0:
req->actual_len = 0;
- req->index = 0;
- req->offset = req->pos & (PAGE_SIZE - 1);
afs_extract_to_tmp64(call);
call->unmarshall++;
+
/* Fall through */
- /* extract the returned data length */
+ /* Extract the returned data length into ->actual_len. This
+ * may indicate more or less data than was requested will be
+ * returned.
+ */
case 1:
_debug("extract data length");
ret = afs_extract_data(call, true);
@@ -464,47 +465,25 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
req->actual_len = be64_to_cpu(call->tmp64);
_debug("DATA length: %llu", req->actual_len);
- req->remain = min(req->len, req->actual_len);
- if (req->remain == 0)
+
+ if (req->actual_len == 0)
goto no_more_data;
call->unmarshall++;
-
- begin_page:
- ASSERTCMP(req->index, <, req->nr_pages);
- if (req->remain > PAGE_SIZE - req->offset)
- size = PAGE_SIZE - req->offset;
- else
- size = req->remain;
- call->iov_len = size;
- call->bvec[0].bv_len = size;
- call->bvec[0].bv_offset = req->offset;
- call->bvec[0].bv_page = req->pages[req->index];
- iov_iter_bvec(&call->def_iter, READ, call->bvec, 1, size);
- ASSERTCMP(size, <=, PAGE_SIZE);
+ call->iter = req->iter;
+ call->iov_len = min(req->actual_len, req->len);
/* Fall through */
/* extract the returned data */
case 2:
_debug("extract data %zu/%llu",
- iov_iter_count(call->iter), req->remain);
+ iov_iter_count(call->iter), req->actual_len);
ret = afs_extract_data(call, true);
if (ret < 0)
return ret;
- req->remain -= call->bvec[0].bv_len;
- req->offset += call->bvec[0].bv_len;
- ASSERTCMP(req->offset, <=, PAGE_SIZE);
- if (req->offset == PAGE_SIZE) {
- req->offset = 0;
- if (req->page_done)
- req->page_done(req);
- req->index++;
- if (req->remain > 0)
- goto begin_page;
- }
- ASSERTCMP(req->remain, ==, 0);
+ call->iter = &call->def_iter;
if (req->actual_len <= req->len)
goto no_more_data;
@@ -552,14 +531,8 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
break;
}
- for (; req->index < req->nr_pages; req->index++) {
- if (req->offset < PAGE_SIZE)
- zero_user_segment(req->pages[req->index],
- req->offset, PAGE_SIZE);
- if (req->page_done)
- req->page_done(req);
- req->offset = 0;
- }
+ if (req->done)
+ req->done(req);
_leave(" = 0 [done]");
return 0;
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 51/61] afs: Use ITER_MAPPING for writing
2020-05-04 17:07 ` David Howells
` (50 preceding siblings ...)
(?)
@ 2020-05-04 17:15 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Use a single ITER_MAPPING iterator to describe the portion of a file to be
transmitted to the server rather than generating a series of small
ITER_BVEC iterators on the fly. This will make it easier to implement AIO
in afs.
In theory we could maybe use one giant ITER_BVEC, but that means
potentially allocating a huge array of bio_vec structs (max 256 per page)
when in fact the pagecache already has a structure listing all the relevant
pages (radix_tree/xarray) that can be walked over.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/fsclient.c | 38 ++++-------------
fs/afs/internal.h | 18 +++-----
fs/afs/rxrpc.c | 99 +++++++-------------------------------------
fs/afs/write.c | 80 +++++++++++++++++++-----------------
fs/afs/yfsclient.c | 19 ++------
include/trace/events/afs.h | 51 ++++++++---------------
6 files changed, 96 insertions(+), 209 deletions(-)
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 8222ccf01280..db80c2618778 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -1108,10 +1108,7 @@ static const struct afs_call_type afs_RXFSStoreData64 = {
/*
* store a set of pages to a very large file
*/
-static int afs_fs_store_data64(struct afs_fs_cursor *fc,
- struct address_space *mapping,
- pgoff_t first, pgoff_t last,
- unsigned offset, unsigned to,
+static int afs_fs_store_data64(struct afs_fs_cursor *fc, struct iov_iter *iter,
loff_t size, loff_t pos, loff_t i_size,
struct afs_status_cb *scb)
{
@@ -1130,12 +1127,7 @@ static int afs_fs_store_data64(struct afs_fs_cursor *fc,
return -ENOMEM;
call->key = fc->key;
- call->mapping = mapping;
- call->first = first;
- call->last = last;
- call->first_offset = offset;
- call->last_to = to;
- call->send_pages = true;
+ call->write_iter = iter;
call->out_scb = scb;
/* marshall the parameters */
@@ -1166,30 +1158,24 @@ static int afs_fs_store_data64(struct afs_fs_cursor *fc,
}
/*
- * store a set of pages
+ * Write data to a file on the server.
*/
-int afs_fs_store_data(struct afs_fs_cursor *fc, struct address_space *mapping,
- pgoff_t first, pgoff_t last,
- unsigned offset, unsigned to,
+int afs_fs_store_data(struct afs_fs_cursor *fc, struct iov_iter *iter, loff_t pos,
struct afs_status_cb *scb)
{
struct afs_vnode *vnode = fc->vnode;
struct afs_call *call;
struct afs_net *net = afs_v2net(vnode);
- loff_t size, pos, i_size;
+ loff_t size, i_size;
__be32 *bp;
if (test_bit(AFS_SERVER_FL_IS_YFS, &fc->cbi->server->flags))
- return yfs_fs_store_data(fc, mapping, first, last, offset, to, scb);
+ return yfs_fs_store_data(fc, iter, pos, scb);
_enter(",%x,{%llx:%llu},,",
key_serial(fc->key), vnode->fid.vid, vnode->fid.vnode);
- size = (loff_t)to - (loff_t)offset;
- if (first != last)
- size += (loff_t)(last - first) << PAGE_SHIFT;
- pos = (loff_t)first << PAGE_SHIFT;
- pos += offset;
+ size = iov_iter_count(iter);
i_size = i_size_read(&vnode->vfs_inode);
if (pos + size > i_size)
@@ -1200,8 +1186,7 @@ int afs_fs_store_data(struct afs_fs_cursor *fc, struct address_space *mapping,
(unsigned long long) i_size);
if (pos >> 32 || i_size >> 32 || size >> 32 || (pos + size) >> 32)
- return afs_fs_store_data64(fc, mapping, first, last, offset, to,
- size, pos, i_size, scb);
+ return afs_fs_store_data64(fc, iter, size, pos, i_size, scb);
call = afs_alloc_flat_call(net, &afs_RXFSStoreData,
(4 + 6 + 3) * 4,
@@ -1210,12 +1195,7 @@ int afs_fs_store_data(struct afs_fs_cursor *fc, struct address_space *mapping,
return -ENOMEM;
call->key = fc->key;
- call->mapping = mapping;
- call->first = first;
- call->last = last;
- call->first_offset = offset;
- call->last_to = to;
- call->send_pages = true;
+ call->write_iter = iter;
call->out_scb = scb;
/* marshall the parameters */
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index e676ad145272..0cd9e998d52c 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -118,6 +118,7 @@ struct afs_call {
struct address_space *mapping; /* Pages being written from */
size_t iov_len; /* Size of *iter to be used */
struct iov_iter def_iter; /* Default buffer/data iterator */
+ struct iov_iter *write_iter; /* Iterator defining write to be made */
struct iov_iter *iter; /* Iterator currently in use */
union { /* Convenience for ->def_iter */
struct kvec kvec[1];
@@ -138,8 +139,6 @@ struct afs_call {
struct afs_volume_status *out_volstatus;
struct afs_read *read_request;
unsigned int server_index;
- pgoff_t first; /* first page in mapping to deal with */
- pgoff_t last; /* last page in mapping to deal with */
atomic_t usage;
enum afs_call_state state;
spinlock_t state_lock;
@@ -149,15 +148,10 @@ struct afs_call {
unsigned int max_lifespan; /* Maximum lifespan to set if not 0 */
unsigned request_size; /* size of request data */
unsigned reply_max; /* maximum size of reply */
- unsigned first_offset; /* offset into mapping[first] */
- union {
- unsigned last_to; /* amount of mapping[last] */
- unsigned count2; /* count used in unmarshalling */
- };
+ unsigned count2; /* count used in unmarshalling */
unsigned char unmarshall; /* unmarshalling phase */
unsigned char addr_ix; /* Address in ->alist */
bool drop_ref; /* T if need to drop ref for incoming call */
- bool send_pages; /* T if data from mapping should be sent */
bool need_attention; /* T if RxRPC poked us */
bool async; /* T if asynchronous */
bool upgrade; /* T to request service upgrade */
@@ -962,8 +956,8 @@ extern int afs_fs_symlink(struct afs_fs_cursor *, const char *, const char *,
extern int afs_fs_rename(struct afs_fs_cursor *, const char *,
struct afs_vnode *, const char *,
struct afs_status_cb *, struct afs_status_cb *);
-extern int afs_fs_store_data(struct afs_fs_cursor *, struct address_space *,
- pgoff_t, pgoff_t, unsigned, unsigned, struct afs_status_cb *);
+extern int afs_fs_store_data(struct afs_fs_cursor *, struct iov_iter *, loff_t,
+ struct afs_status_cb *);
extern int afs_fs_setattr(struct afs_fs_cursor *, struct iattr *, struct afs_status_cb *);
extern int afs_fs_get_volume_status(struct afs_fs_cursor *, struct afs_volume_status *);
extern int afs_fs_set_lock(struct afs_fs_cursor *, afs_lock_type_t, struct afs_status_cb *);
@@ -1378,8 +1372,8 @@ extern int yfs_fs_symlink(struct afs_fs_cursor *, const char *, const char *,
struct afs_status_cb *, struct afs_fid *, struct afs_status_cb *);
extern int yfs_fs_rename(struct afs_fs_cursor *, const char *, struct afs_vnode *, const char *,
struct afs_status_cb *, struct afs_status_cb *);
-extern int yfs_fs_store_data(struct afs_fs_cursor *, struct address_space *,
- pgoff_t, pgoff_t, unsigned, unsigned, struct afs_status_cb *);
+extern int yfs_fs_store_data(struct afs_fs_cursor *, struct iov_iter *, loff_t,
+ struct afs_status_cb *);
extern int yfs_fs_setattr(struct afs_fs_cursor *, struct iattr *, struct afs_status_cb *);
extern int yfs_fs_get_volume_status(struct afs_fs_cursor *, struct afs_volume_status *);
extern int yfs_fs_set_lock(struct afs_fs_cursor *, afs_lock_type_t, struct afs_status_cb *);
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index b2296feaaff3..98da499232a3 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -274,39 +274,6 @@ void afs_flat_call_destructor(struct afs_call *call)
call->buffer = NULL;
}
-#define AFS_BVEC_MAX 8
-
-/*
- * Load the given bvec with the next few pages.
- */
-static void afs_load_bvec(struct afs_call *call, struct msghdr *msg,
- struct bio_vec *bv, pgoff_t first, pgoff_t last,
- unsigned offset)
-{
- struct page *pages[AFS_BVEC_MAX];
- unsigned int nr, n, i, to, bytes = 0;
-
- nr = min_t(pgoff_t, last - first + 1, AFS_BVEC_MAX);
- n = find_get_pages_contig(call->mapping, first, nr, pages);
- ASSERTCMP(n, ==, nr);
-
- msg->msg_flags |= MSG_MORE;
- for (i = 0; i < nr; i++) {
- to = PAGE_SIZE;
- if (first + i >= last) {
- to = call->last_to;
- msg->msg_flags &= ~MSG_MORE;
- }
- bv[i].bv_page = pages[i];
- bv[i].bv_len = to - offset;
- bv[i].bv_offset = offset;
- bytes += to - offset;
- offset = 0;
- }
-
- iov_iter_bvec(&msg->msg_iter, WRITE, bv, nr, bytes);
-}
-
/*
* Advance the AFS call state when the RxRPC call ends the transmit phase.
*/
@@ -319,41 +286,6 @@ static void afs_notify_end_request_tx(struct sock *sock,
afs_set_call_state(call, AFS_CALL_CL_REQUESTING, AFS_CALL_CL_AWAIT_REPLY);
}
-/*
- * attach the data from a bunch of pages on an inode to a call
- */
-static int afs_send_pages(struct afs_call *call, struct msghdr *msg)
-{
- struct bio_vec bv[AFS_BVEC_MAX];
- unsigned int bytes, nr, loop, offset;
- pgoff_t first = call->first, last = call->last;
- int ret;
-
- offset = call->first_offset;
- call->first_offset = 0;
-
- do {
- afs_load_bvec(call, msg, bv, first, last, offset);
- trace_afs_send_pages(call, msg, first, last, offset);
-
- offset = 0;
- bytes = msg->msg_iter.count;
- nr = msg->msg_iter.nr_segs;
-
- ret = rxrpc_kernel_send_data(call->net->socket, call->rxcall, msg,
- bytes, afs_notify_end_request_tx);
- for (loop = 0; loop < nr; loop++)
- put_page(bv[loop].bv_page);
- if (ret < 0)
- break;
-
- first += nr;
- } while (first <= last);
-
- trace_afs_sent_pages(call, call->first, last, first, ret);
- return ret;
-}
-
/*
* Initiate a call and synchronously queue up the parameters for dispatch. Any
* error is stored into the call struct, which the caller must check for.
@@ -385,19 +317,8 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
* after the initial fixed part.
*/
tx_total_len = call->request_size;
- if (call->send_pages) {
- if (call->last == call->first) {
- tx_total_len += call->last_to - call->first_offset;
- } else {
- /* It looks mathematically like you should be able to
- * combine the following lines with the ones above, but
- * unsigned arithmetic is fun when it wraps...
- */
- tx_total_len += PAGE_SIZE - call->first_offset;
- tx_total_len += call->last_to;
- tx_total_len += (call->last - call->first - 1) * PAGE_SIZE;
- }
- }
+ if (call->write_iter)
+ tx_total_len += iov_iter_count(call->write_iter);
/* If the call is going to be asynchronous, we need an extra ref for
* the call to hold itself so the caller need not hang on to its ref.
@@ -439,7 +360,7 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
iov_iter_kvec(&msg.msg_iter, WRITE, iov, 1, call->request_size);
msg.msg_control = NULL;
msg.msg_controllen = 0;
- msg.msg_flags = MSG_WAITALL | (call->send_pages ? MSG_MORE : 0);
+ msg.msg_flags = MSG_WAITALL | (call->write_iter ? MSG_MORE : 0);
ret = rxrpc_kernel_send_data(call->net->socket, rxcall,
&msg, call->request_size,
@@ -447,8 +368,18 @@ void afs_make_call(struct afs_addr_cursor *ac, struct afs_call *call, gfp_t gfp)
if (ret < 0)
goto error_do_abort;
- if (call->send_pages) {
- ret = afs_send_pages(call, &msg);
+ if (call->write_iter) {
+ msg.msg_iter = *call->write_iter;
+ msg.msg_flags &= ~MSG_MORE;
+ trace_afs_send_data(call, &msg);
+
+ ret = rxrpc_kernel_send_data(call->net->socket,
+ call->rxcall, &msg,
+ iov_iter_count(&msg.msg_iter),
+ afs_notify_end_request_tx);
+ *call->write_iter = msg.msg_iter;
+
+ trace_afs_sent_data(call, &msg, ret);
if (ret < 0)
goto error_do_abort;
}
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 174e355aee6d..44dd4d0bad70 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -311,38 +311,30 @@ static void afs_redirty_pages(struct writeback_control *wbc,
/*
* completion of write to server
*/
-static void afs_pages_written_back(struct afs_vnode *vnode,
- pgoff_t first, pgoff_t last)
+static void afs_pages_written_back(struct afs_vnode *vnode, pgoff_t start, pgoff_t last)
{
- struct pagevec pv;
+ struct address_space *mapping = vnode->vfs_inode.i_mapping;
+ struct page *page;
unsigned long priv;
- unsigned count, loop;
+
+ XA_STATE(xas, &mapping->i_pages, start);
_enter("{%llx:%llu},{%lx-%lx}",
- vnode->fid.vid, vnode->fid.vnode, first, last);
+ vnode->fid.vid, vnode->fid.vnode, start, last);
- pagevec_init(&pv);
+ rcu_read_lock();
- do {
- _debug("done %lx-%lx", first, last);
+ xas_for_each(&xas, page, last) {
+ ASSERT(PageWriteback(page));
- count = last - first + 1;
- if (count > PAGEVEC_SIZE)
- count = PAGEVEC_SIZE;
- pv.nr = find_get_pages_contig(vnode->vfs_inode.i_mapping,
- first, count, pv.pages);
- ASSERTCMP(pv.nr, ==, count);
+ priv = page_private(page);
+ trace_afs_page_dirty(vnode, tracepoint_string("clear"),
+ page->index, priv);
+ set_page_private(page, 0);
+ page_endio(page, true, 0);
+ }
- for (loop = 0; loop < count; loop++) {
- priv = page_private(pv.pages[loop]);
- trace_afs_page_dirty(vnode, tracepoint_string("clear"),
- pv.pages[loop]->index, priv);
- set_page_private(pv.pages[loop], 0);
- end_page_writeback(pv.pages[loop]);
- }
- first += count;
- __pagevec_release(&pv);
- } while (first <= last);
+ rcu_read_unlock();
afs_prune_wb_keys(vnode);
_leave("");
@@ -351,23 +343,22 @@ static void afs_pages_written_back(struct afs_vnode *vnode,
/*
* write to a file
*/
-static int afs_store_data(struct address_space *mapping,
- pgoff_t first, pgoff_t last,
- unsigned offset, unsigned to)
+static int afs_store_data(struct afs_vnode *vnode, struct iov_iter *iter,
+ loff_t pos, pgoff_t first, pgoff_t last)
{
- struct afs_vnode *vnode = AFS_FS_I(mapping->host);
struct afs_fs_cursor fc;
struct afs_status_cb *scb;
struct afs_wb_key *wbk = NULL;
struct list_head *p;
+ loff_t count = iov_iter_count(iter);
int ret = -ENOKEY, ret2;
- _enter("%s{%llx:%llu.%u},%lx,%lx,%x,%x",
+ _enter("%s{%llx:%llu.%u},%llx,%llx",
vnode->volume->name,
vnode->fid.vid,
vnode->fid.vnode,
vnode->fid.unique,
- first, last, offset, to);
+ count, pos);
scb = kzalloc(sizeof(struct afs_status_cb), GFP_NOFS);
if (!scb)
@@ -407,7 +398,7 @@ static int afs_store_data(struct address_space *mapping,
while (afs_select_fileserver(&fc)) {
fc.cb_break = afs_calc_vnode_cb_break(vnode);
- afs_fs_store_data(&fc, mapping, first, last, offset, to, scb);
+ afs_fs_store_data(&fc, iter, pos, scb);
}
afs_check_for_remote_deletion(&fc, vnode);
@@ -421,9 +412,7 @@ static int afs_store_data(struct address_space *mapping,
switch (ret) {
case 0:
afs_stat_v(vnode, n_stores);
- atomic_long_add((last * PAGE_SIZE + to) -
- (first * PAGE_SIZE + offset),
- &afs_v2net(vnode)->n_store_bytes);
+ atomic_long_add(count, &afs_v2net(vnode)->n_store_bytes);
break;
case -EACCES:
case -EPERM:
@@ -454,10 +443,12 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
pgoff_t final_page)
{
struct afs_vnode *vnode = AFS_FS_I(mapping->host);
+ struct iov_iter iter;
struct page *pages[8], *page;
unsigned long count, priv;
unsigned n, offset, to, f, t;
pgoff_t start, first, last;
+ loff_t a, b;
int loop, ret;
_enter(",%lx", primary_page->index);
@@ -557,10 +548,17 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
first = primary_page->index;
last = first + count - 1;
-
_debug("write back %lx[%u..] to %lx[..%u]", first, offset, last, to);
- ret = afs_store_data(mapping, first, last, offset, to);
+ a = first;
+ a <<= PAGE_SHIFT;
+ a += offset;
+ b = last;
+ b <<= PAGE_SHIFT;
+ b += to;
+ iov_iter_mapping(&iter, WRITE, mapping, a, b - a);
+
+ ret = afs_store_data(vnode, &iter, a, first, last);
switch (ret) {
case 0:
ret = count;
@@ -848,6 +846,8 @@ int afs_launder_page(struct page *page)
{
struct address_space *mapping = page->mapping;
struct afs_vnode *vnode = AFS_FS_I(mapping->host);
+ struct iov_iter iter;
+ struct bio_vec bv[1];
unsigned long priv;
unsigned int f, t;
int ret = 0;
@@ -863,9 +863,15 @@ int afs_launder_page(struct page *page)
t = priv >> AFS_PRIV_SHIFT;
}
+ bv[0].bv_page = page;
+ bv[0].bv_offset = f;
+ bv[0].bv_len = t - f;
+ iov_iter_bvec(&iter, WRITE, bv, 1, bv[0].bv_len);
+
trace_afs_page_dirty(vnode, tracepoint_string("launder"),
page->index, priv);
- ret = afs_store_data(mapping, page->index, page->index, t, f);
+ ret = afs_store_data(vnode, &iter, (loff_t)page->index << PAGE_SHIFT,
+ page->index, page->index);
}
trace_afs_page_dirty(vnode, tracepoint_string("laundered"),
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 518b9489ff9e..0b744a117dde 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -1216,25 +1216,19 @@ static const struct afs_call_type yfs_RXYFSStoreData64 = {
/*
* Store a set of pages to a large file.
*/
-int yfs_fs_store_data(struct afs_fs_cursor *fc, struct address_space *mapping,
- pgoff_t first, pgoff_t last,
- unsigned offset, unsigned to,
+int yfs_fs_store_data(struct afs_fs_cursor *fc, struct iov_iter *iter, loff_t pos,
struct afs_status_cb *scb)
{
struct afs_vnode *vnode = fc->vnode;
struct afs_call *call;
struct afs_net *net = afs_v2net(vnode);
- loff_t size, pos, i_size;
+ loff_t size, i_size;
__be32 *bp;
_enter(",%x,{%llx:%llu},,",
key_serial(fc->key), vnode->fid.vid, vnode->fid.vnode);
- size = (loff_t)to - (loff_t)offset;
- if (first != last)
- size += (loff_t)(last - first) << PAGE_SHIFT;
- pos = (loff_t)first << PAGE_SHIFT;
- pos += offset;
+ size = iov_iter_count(iter);
i_size = i_size_read(&vnode->vfs_inode);
if (pos + size > i_size)
@@ -1256,12 +1250,7 @@ int yfs_fs_store_data(struct afs_fs_cursor *fc, struct address_space *mapping,
return -ENOMEM;
call->key = fc->key;
- call->mapping = mapping;
- call->first = first;
- call->last = last;
- call->first_offset = offset;
- call->last_to = to;
- call->send_pages = true;
+ call->write_iter = iter;
call->out_scb = scb;
/* marshall the parameters */
diff --git a/include/trace/events/afs.h b/include/trace/events/afs.h
index c612cabbc378..f663cd482abb 100644
--- a/include/trace/events/afs.h
+++ b/include/trace/events/afs.h
@@ -762,65 +762,52 @@ TRACE_EVENT(afs_call_done,
__entry->rx_call)
);
-TRACE_EVENT(afs_send_pages,
- TP_PROTO(struct afs_call *call, struct msghdr *msg,
- pgoff_t first, pgoff_t last, unsigned int offset),
+TRACE_EVENT(afs_send_data,
+ TP_PROTO(struct afs_call *call, struct msghdr *msg),
- TP_ARGS(call, msg, first, last, offset),
+ TP_ARGS(call, msg),
TP_STRUCT__entry(
__field(unsigned int, call )
- __field(pgoff_t, first )
- __field(pgoff_t, last )
- __field(unsigned int, nr )
- __field(unsigned int, bytes )
- __field(unsigned int, offset )
__field(unsigned int, flags )
+ __field(loff_t, offset )
+ __field(loff_t, count )
),
TP_fast_assign(
__entry->call = call->debug_id;
- __entry->first = first;
- __entry->last = last;
- __entry->nr = msg->msg_iter.nr_segs;
- __entry->bytes = msg->msg_iter.count;
- __entry->offset = offset;
__entry->flags = msg->msg_flags;
+ __entry->offset = msg->msg_iter.iov_offset;
+ __entry->count = iov_iter_count(&msg->msg_iter);
),
- TP_printk(" c=%08x %lx-%lx-%lx b=%x o=%x f=%x",
- __entry->call,
- __entry->first, __entry->first + __entry->nr - 1, __entry->last,
- __entry->bytes, __entry->offset,
+ TP_printk(" c=%08x o=%llx c=%llx f=%x",
+ __entry->call, __entry->offset, __entry->count,
__entry->flags)
);
-TRACE_EVENT(afs_sent_pages,
- TP_PROTO(struct afs_call *call, pgoff_t first, pgoff_t last,
- pgoff_t cursor, int ret),
+TRACE_EVENT(afs_sent_data,
+ TP_PROTO(struct afs_call *call, struct msghdr *msg, int ret),
- TP_ARGS(call, first, last, cursor, ret),
+ TP_ARGS(call, msg, ret),
TP_STRUCT__entry(
__field(unsigned int, call )
- __field(pgoff_t, first )
- __field(pgoff_t, last )
- __field(pgoff_t, cursor )
__field(int, ret )
+ __field(loff_t, offset )
+ __field(loff_t, count )
),
TP_fast_assign(
__entry->call = call->debug_id;
- __entry->first = first;
- __entry->last = last;
- __entry->cursor = cursor;
__entry->ret = ret;
+ __entry->offset = msg->msg_iter.iov_offset;
+ __entry->count = iov_iter_count(&msg->msg_iter);
),
- TP_printk(" c=%08x %lx-%lx c=%lx r=%d",
- __entry->call,
- __entry->first, __entry->last,
- __entry->cursor, __entry->ret)
+ TP_printk(" c=%08x o=%llx c=%llx r=%d",
+ __entry->call, __entry->offset, __entry->count,
+ __entry->ret)
);
TRACE_EVENT(afs_dir_check_failed,
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 52/61] afs: Interpose struct fscache_io_request into struct afs_read
@ 2020-05-04 17:15 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Embed an fscache_io_request struct into struct afs_read and remove some of
the redundant members from the latter.
Change all references to those removed members to use the fscache ones
instead.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/dir.c | 38 ++++++++++++++++++++++++--------------
fs/afs/file.c | 47 ++++++++++++++++++++++++-----------------------
fs/afs/fsclient.c | 28 ++++++++++++++--------------
fs/afs/internal.h | 12 ++++--------
fs/afs/write.c | 6 +++---
fs/afs/yfsclient.c | 18 +++++++++---------
6 files changed, 78 insertions(+), 71 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index a10bcf632e0c..0b3f33269fdd 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -110,13 +110,14 @@ struct afs_lookup_cookie {
*/
static void afs_dir_read_cleanup(struct afs_read *req)
{
- struct address_space *mapping = req->iter->mapping;
+ struct afs_vnode *vnode = req->vnode;
+ struct address_space *mapping = vnode->vfs_inode.i_mapping;
struct page *page;
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
XA_STATE(xas, &mapping->i_pages, 0);
- if (unlikely(!req->nr_pages))
+ if (unlikely(!req->cache.nr_pages))
return;
rcu_read_lock();
@@ -133,6 +134,13 @@ static void afs_dir_read_cleanup(struct afs_read *req)
rcu_read_unlock();
}
+/*
+ * Do nothing upon completion of the request.
+ */
+static void afs_dir_read_done(struct fscache_io_request *fsreq)
+{
+}
+
/*
* check that a directory page is valid
*/
@@ -196,15 +204,15 @@ static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
struct address_space *mapping = dvnode->vfs_inode.i_mapping;
struct page *page;
unsigned int i, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
XA_STATE(xas, &mapping->i_pages, 0);
pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode,
- req->file_size, req->len, req->actual_len);
+ req->file_size, req->cache.len, req->actual_len);
pr_warn("DIR %llx %x %zx %zx\n",
- req->pos, req->nr_pages,
+ req->cache.pos, req->cache.nr_pages,
req->iter->iov_offset, iov_iter_count(req->iter));
xas_for_each(&xas, page, last) {
@@ -231,12 +239,12 @@ static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
{
struct address_space *mapping = dvnode->vfs_inode.i_mapping;
struct page *page;
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
int ret = 0;
XA_STATE(xas, &mapping->i_pages, 0);
- if (unlikely(!req->nr_pages))
+ if (unlikely(!req->cache.nr_pages))
return 0;
rcu_read_lock();
@@ -295,7 +303,9 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
refcount_set(&req->usage, 1);
req->key = key_get(key);
+ req->vnode = dvnode;
req->cleanup = afs_dir_read_cleanup;
+ req->cache.io_done = afs_dir_read_done;
expand:
i_size = i_size_read(&dvnode->vfs_inode);
@@ -314,7 +324,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
nr_pages = (i_size + PAGE_SIZE - 1) / PAGE_SIZE;
req->actual_len = i_size; /* May change */
- req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
+ req->cache.len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
req->data_version = dvnode->status.data_version; /* May change */
iov_iter_mapping(&req->def_iter, READ, dvnode->vfs_inode.i_mapping,
0, i_size);
@@ -324,7 +334,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
* been at work and pin all the pages. If there are any gaps, we will
* need to reread the entire directory contents.
*/
- i = req->nr_pages;
+ i = req->cache.nr_pages;
while (i < nr_pages) {
struct page *pages[8], *page;
@@ -353,10 +363,10 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
set_page_private(page, 1);
SetPagePrivate(page);
unlock_page(page);
- req->nr_pages++;
+ req->cache.nr_pages++;
i++;
} else {
- req->nr_pages += n;
+ req->cache.nr_pages += n;
i += n;
}
}
@@ -381,9 +391,9 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
if (ret < 0)
goto error_unlock;
- task_io_account_read(PAGE_SIZE * req->nr_pages);
+ task_io_account_read(PAGE_SIZE * req->cache.nr_pages);
- if (req->len < req->file_size) {
+ if (req->cache.len < req->file_size) {
/* The content has grown, so we need to expand the
* buffer.
*/
diff --git a/fs/afs/file.c b/fs/afs/file.c
index c8ad638590e7..ea9f6d45d9ff 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -199,12 +199,13 @@ int afs_release(struct inode *inode, struct file *file)
/*
* Handle completion of a read operation.
*/
-static void afs_file_read_done(struct afs_read *req)
+static void afs_file_read_done(struct fscache_io_request *fsreq)
{
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
struct afs_vnode *vnode = req->vnode;
struct page *page;
- pgoff_t index = req->pos >> PAGE_SHIFT;
- pgoff_t last = index + req->nr_pages - 1;
+ pgoff_t index = req->cache.pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->cache.nr_pages - 1;
XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
@@ -213,7 +214,7 @@ static void afs_file_read_done(struct afs_read *req)
_debug("afterclear %zx %zx %llx/%llx",
req->iter->iov_offset,
iov_iter_count(req->iter),
- req->actual_len, req->len);
+ req->actual_len, req->cache.len);
iov_iter_zero(iov_iter_count(req->iter), req->iter);
}
@@ -224,7 +225,7 @@ static void afs_file_read_done(struct afs_read *req)
}
rcu_read_unlock();
- task_io_account_read(req->len);
+ task_io_account_read(req->cache.len);
req->cleanup = NULL;
}
@@ -234,19 +235,19 @@ static void afs_file_read_done(struct afs_read *req)
static void afs_file_read_cleanup(struct afs_read *req)
{
struct page *page;
- pgoff_t index = req->pos >> PAGE_SHIFT;
- pgoff_t last = index + req->nr_pages - 1;
+ pgoff_t index = req->cache.pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->cache.nr_pages - 1;
XA_STATE(xas, &req->iter->mapping->i_pages, index);
- _enter("%lu,%u,%zu", index, req->nr_pages, iov_iter_count(req->iter));
+ _enter("%lu,%u,%zu", index, req->cache.nr_pages, iov_iter_count(req->iter));
rcu_read_lock();
xas_for_each(&xas, page, last) {
BUG_ON(xa_is_value(page));
BUG_ON(PageCompound(page));
- page_endio(page, false, req->error);
+ page_endio(page, false, req->cache.error);
put_page(page);
}
rcu_read_unlock();
@@ -300,7 +301,7 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
ret = afs_end_vnode_operation(&fc);
}
- req->error = ret;
+ req->cache.error = ret;
if (ret == 0) {
afs_stat_v(vnode, n_fetches);
atomic_long_add(req->actual_len,
@@ -337,15 +338,15 @@ static int afs_page_filler(struct key *key, struct page *page)
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(key);
- req->pos = (loff_t)page->index << PAGE_SHIFT;
- req->len = PAGE_SIZE;
- req->nr_pages = 1;
- req->done = afs_file_read_done;
+ req->cache.nr_pages = 1;
+ req->cache.pos = (loff_t)page->index << PAGE_SHIFT;
+ req->cache.len = PAGE_SIZE;
+ req->cache.io_done = afs_file_read_done;
req->cleanup = afs_file_read_cleanup;
get_page(page);
iov_iter_mapping(&req->def_iter, READ, page->mapping,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
@@ -444,10 +445,10 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(afs_file_key(file));
- req->done = afs_file_read_done;
req->cleanup = afs_file_read_cleanup;
- req->pos = first->index;
- req->pos <<= PAGE_SHIFT;
+ req->cache.io_done = afs_file_read_done;
+ req->cache.pos = first->index;
+ req->cache.pos <<= PAGE_SHIFT;
/* Add pages to the LRU until it fails. We keep the pages ref'd and
* locked until the read is complete.
@@ -467,17 +468,17 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
break;
}
- req->nr_pages++;
- } while (req->nr_pages < n);
+ req->cache.nr_pages++;
+ } while (req->cache.nr_pages < n);
- if (req->nr_pages == 0) {
+ if (req->cache.nr_pages == 0) {
afs_put_read(req);
return 0;
}
- req->len = req->nr_pages * PAGE_SIZE;
+ req->cache.len = req->cache.nr_pages * PAGE_SIZE;
iov_iter_mapping(&req->def_iter, READ, file->f_mapping,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index db80c2618778..c9789294fc68 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -361,7 +361,7 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
call->unmarshall++;
call->iter = req->iter;
- call->iov_len = min(req->actual_len, req->len);
+ call->iov_len = min(req->actual_len, req->cache.len);
/* Fall through */
/* extract the returned data */
@@ -374,17 +374,17 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
return ret;
call->iter = &call->def_iter;
- if (req->actual_len <= req->len)
+ if (req->actual_len <= req->cache.len)
goto no_more_data;
/* Discard any excess data the server gave us */
- afs_extract_discard(call, req->actual_len - req->len);
+ afs_extract_discard(call, req->actual_len - req->cache.len);
call->unmarshall = 3;
/* Fall through */
case 3:
_debug("extract discard %zu/%llu",
- iov_iter_count(call->iter), req->actual_len - req->len);
+ iov_iter_count(call->iter), req->actual_len - req->cache.len);
ret = afs_extract_data(call, true);
if (ret < 0)
@@ -417,8 +417,8 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
break;
}
- if (req->done)
- req->done(req);
+ if (req->cache.io_done)
+ req->cache.io_done(&req->cache);
_leave(" = 0 [done]");
return 0;
@@ -478,10 +478,10 @@ static int afs_fs_fetch_data64(struct afs_fs_cursor *fc,
bp[1] = htonl(vnode->fid.vid);
bp[2] = htonl(vnode->fid.vnode);
bp[3] = htonl(vnode->fid.unique);
- bp[4] = htonl(upper_32_bits(req->pos));
- bp[5] = htonl(lower_32_bits(req->pos));
+ bp[4] = htonl(upper_32_bits(req->cache.pos));
+ bp[5] = htonl(lower_32_bits(req->cache.pos));
bp[6] = 0;
- bp[7] = htonl(lower_32_bits(req->len));
+ bp[7] = htonl(lower_32_bits(req->cache.len));
afs_use_fs_server(call, fc->cbi);
trace_afs_make_fs_call(call, &vnode->fid);
@@ -505,9 +505,9 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
if (test_bit(AFS_SERVER_FL_IS_YFS, &fc->cbi->server->flags))
return yfs_fs_fetch_data(fc, scb, req);
- if (upper_32_bits(req->pos) ||
- upper_32_bits(req->len) ||
- upper_32_bits(req->pos + req->len))
+ if (upper_32_bits(req->cache.pos) ||
+ upper_32_bits(req->cache.len) ||
+ upper_32_bits(req->cache.pos + req->cache.len))
return afs_fs_fetch_data64(fc, scb, req);
_enter("");
@@ -528,8 +528,8 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
bp[1] = htonl(vnode->fid.vid);
bp[2] = htonl(vnode->fid.vnode);
bp[3] = htonl(vnode->fid.unique);
- bp[4] = htonl(lower_32_bits(req->pos));
- bp[5] = htonl(lower_32_bits(req->len));
+ bp[4] = htonl(lower_32_bits(req->cache.pos));
+ bp[5] = htonl(lower_32_bits(req->cache.len));
afs_use_fs_server(call, fc->cbi);
trace_afs_make_fs_call(call, &vnode->fid);
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 0cd9e998d52c..9f56acdc2ed9 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -218,8 +218,9 @@ static inline struct key *afs_file_key(struct file *file)
* Record of an outstanding read operation on a vnode.
*/
struct afs_read {
- loff_t pos; /* Where to start reading */
- loff_t len; /* How much we're asking for */
+ struct fscache_io_request cache;
+ struct iov_iter def_iter; /* Default iterator */
+ struct iov_iter *iter; /* Iterator to use */
loff_t actual_len; /* How much we're actually getting */
loff_t file_size; /* File size returned by server */
struct key *key; /* The key to use to reissue the read */
@@ -227,12 +228,7 @@ struct afs_read {
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
unsigned int call_debug_id;
- unsigned int nr_pages;
- int error;
- void (*done)(struct afs_read *);
- void (*cleanup)(struct afs_read *);
- struct iov_iter *iter; /* Iterator representing the buffer */
- struct iov_iter def_iter; /* Default iterator */
+ void (*cleanup)(struct afs_read *req);
};
/*
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 44dd4d0bad70..390fee44446c 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -51,9 +51,9 @@ static int afs_fill_page(struct file *file,
refcount_set(&req->usage, 1);
req->key = afs_file_key(file);
- req->pos = pos;
- req->len = len;
- req->nr_pages = 1;
+ req->cache.pos = pos;
+ req->cache.len = len;
+ req->cache.nr_pages = 1;
iov_iter_mapping(&req->def_iter, READ, vnode->vfs_inode.i_mapping,
pos, len);
req->iter = &req->def_iter;
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 0b744a117dde..6ea97233c0d2 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -471,7 +471,7 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
call->unmarshall++;
call->iter = req->iter;
- call->iov_len = min(req->actual_len, req->len);
+ call->iov_len = min(req->actual_len, req->cache.len);
/* Fall through */
/* extract the returned data */
@@ -484,17 +484,17 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
return ret;
call->iter = &call->def_iter;
- if (req->actual_len <= req->len)
+ if (req->actual_len <= req->cache.len)
goto no_more_data;
/* Discard any excess data the server gave us */
- afs_extract_discard(call, req->actual_len - req->len);
+ afs_extract_discard(call, req->actual_len - req->cache.len);
call->unmarshall = 3;
/* Fall through */
case 3:
_debug("extract discard %zu/%llu",
- iov_iter_count(call->iter), req->actual_len - req->len);
+ iov_iter_count(call->iter), req->actual_len - req->cache.len);
ret = afs_extract_data(call, true);
if (ret < 0)
@@ -531,8 +531,8 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
break;
}
- if (req->done)
- req->done(req);
+ if (req->cache.io_done)
+ req->cache.io_done(&req->cache);
_leave(" = 0 [done]");
return 0;
@@ -567,7 +567,7 @@ int yfs_fs_fetch_data(struct afs_fs_cursor *fc, struct afs_status_cb *scb,
_enter(",%x,{%llx:%llu},%llx,%llx",
key_serial(fc->key), vnode->fid.vid, vnode->fid.vnode,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
call = afs_alloc_flat_call(net, &yfs_RXYFSFetchData64,
sizeof(__be32) * 2 +
@@ -589,8 +589,8 @@ int yfs_fs_fetch_data(struct afs_fs_cursor *fc, struct afs_status_cb *scb,
bp = xdr_encode_u32(bp, YFSFETCHDATA64);
bp = xdr_encode_u32(bp, 0); /* RPC flags */
bp = xdr_encode_YFSFid(bp, &vnode->fid);
- bp = xdr_encode_u64(bp, req->pos);
- bp = xdr_encode_u64(bp, req->len);
+ bp = xdr_encode_u64(bp, req->cache.pos);
+ bp = xdr_encode_u64(bp, req->cache.len);
yfs_check_req(call, bp);
afs_use_fs_server(call, fc->cbi);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 52/61] afs: Interpose struct fscache_io_request into struct afs_read
@ 2020-05-04 17:15 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Embed an fscache_io_request struct into struct afs_read and remove some of
the redundant members from the latter.
Change all references to those removed members to use the fscache ones
instead.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
fs/afs/dir.c | 38 ++++++++++++++++++++++++--------------
fs/afs/file.c | 47 ++++++++++++++++++++++++-----------------------
fs/afs/fsclient.c | 28 ++++++++++++++--------------
fs/afs/internal.h | 12 ++++--------
fs/afs/write.c | 6 +++---
fs/afs/yfsclient.c | 18 +++++++++---------
6 files changed, 78 insertions(+), 71 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index a10bcf632e0c..0b3f33269fdd 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -110,13 +110,14 @@ struct afs_lookup_cookie {
*/
static void afs_dir_read_cleanup(struct afs_read *req)
{
- struct address_space *mapping = req->iter->mapping;
+ struct afs_vnode *vnode = req->vnode;
+ struct address_space *mapping = vnode->vfs_inode.i_mapping;
struct page *page;
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
XA_STATE(xas, &mapping->i_pages, 0);
- if (unlikely(!req->nr_pages))
+ if (unlikely(!req->cache.nr_pages))
return;
rcu_read_lock();
@@ -133,6 +134,13 @@ static void afs_dir_read_cleanup(struct afs_read *req)
rcu_read_unlock();
}
+/*
+ * Do nothing upon completion of the request.
+ */
+static void afs_dir_read_done(struct fscache_io_request *fsreq)
+{
+}
+
/*
* check that a directory page is valid
*/
@@ -196,15 +204,15 @@ static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
struct address_space *mapping = dvnode->vfs_inode.i_mapping;
struct page *page;
unsigned int i, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
XA_STATE(xas, &mapping->i_pages, 0);
pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode,
- req->file_size, req->len, req->actual_len);
+ req->file_size, req->cache.len, req->actual_len);
pr_warn("DIR %llx %x %zx %zx\n",
- req->pos, req->nr_pages,
+ req->cache.pos, req->cache.nr_pages,
req->iter->iov_offset, iov_iter_count(req->iter));
xas_for_each(&xas, page, last) {
@@ -231,12 +239,12 @@ static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
{
struct address_space *mapping = dvnode->vfs_inode.i_mapping;
struct page *page;
- pgoff_t last = req->nr_pages - 1;
+ pgoff_t last = req->cache.nr_pages - 1;
int ret = 0;
XA_STATE(xas, &mapping->i_pages, 0);
- if (unlikely(!req->nr_pages))
+ if (unlikely(!req->cache.nr_pages))
return 0;
rcu_read_lock();
@@ -295,7 +303,9 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
refcount_set(&req->usage, 1);
req->key = key_get(key);
+ req->vnode = dvnode;
req->cleanup = afs_dir_read_cleanup;
+ req->cache.io_done = afs_dir_read_done;
expand:
i_size = i_size_read(&dvnode->vfs_inode);
@@ -314,7 +324,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
nr_pages = (i_size + PAGE_SIZE - 1) / PAGE_SIZE;
req->actual_len = i_size; /* May change */
- req->len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
+ req->cache.len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
req->data_version = dvnode->status.data_version; /* May change */
iov_iter_mapping(&req->def_iter, READ, dvnode->vfs_inode.i_mapping,
0, i_size);
@@ -324,7 +334,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
* been at work and pin all the pages. If there are any gaps, we will
* need to reread the entire directory contents.
*/
- i = req->nr_pages;
+ i = req->cache.nr_pages;
while (i < nr_pages) {
struct page *pages[8], *page;
@@ -353,10 +363,10 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
set_page_private(page, 1);
SetPagePrivate(page);
unlock_page(page);
- req->nr_pages++;
+ req->cache.nr_pages++;
i++;
} else {
- req->nr_pages += n;
+ req->cache.nr_pages += n;
i += n;
}
}
@@ -381,9 +391,9 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
if (ret < 0)
goto error_unlock;
- task_io_account_read(PAGE_SIZE * req->nr_pages);
+ task_io_account_read(PAGE_SIZE * req->cache.nr_pages);
- if (req->len < req->file_size) {
+ if (req->cache.len < req->file_size) {
/* The content has grown, so we need to expand the
* buffer.
*/
diff --git a/fs/afs/file.c b/fs/afs/file.c
index c8ad638590e7..ea9f6d45d9ff 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -199,12 +199,13 @@ int afs_release(struct inode *inode, struct file *file)
/*
* Handle completion of a read operation.
*/
-static void afs_file_read_done(struct afs_read *req)
+static void afs_file_read_done(struct fscache_io_request *fsreq)
{
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
struct afs_vnode *vnode = req->vnode;
struct page *page;
- pgoff_t index = req->pos >> PAGE_SHIFT;
- pgoff_t last = index + req->nr_pages - 1;
+ pgoff_t index = req->cache.pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->cache.nr_pages - 1;
XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
@@ -213,7 +214,7 @@ static void afs_file_read_done(struct afs_read *req)
_debug("afterclear %zx %zx %llx/%llx",
req->iter->iov_offset,
iov_iter_count(req->iter),
- req->actual_len, req->len);
+ req->actual_len, req->cache.len);
iov_iter_zero(iov_iter_count(req->iter), req->iter);
}
@@ -224,7 +225,7 @@ static void afs_file_read_done(struct afs_read *req)
}
rcu_read_unlock();
- task_io_account_read(req->len);
+ task_io_account_read(req->cache.len);
req->cleanup = NULL;
}
@@ -234,19 +235,19 @@ static void afs_file_read_done(struct afs_read *req)
static void afs_file_read_cleanup(struct afs_read *req)
{
struct page *page;
- pgoff_t index = req->pos >> PAGE_SHIFT;
- pgoff_t last = index + req->nr_pages - 1;
+ pgoff_t index = req->cache.pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->cache.nr_pages - 1;
XA_STATE(xas, &req->iter->mapping->i_pages, index);
- _enter("%lu,%u,%zu", index, req->nr_pages, iov_iter_count(req->iter));
+ _enter("%lu,%u,%zu", index, req->cache.nr_pages, iov_iter_count(req->iter));
rcu_read_lock();
xas_for_each(&xas, page, last) {
BUG_ON(xa_is_value(page));
BUG_ON(PageCompound(page));
- page_endio(page, false, req->error);
+ page_endio(page, false, req->cache.error);
put_page(page);
}
rcu_read_unlock();
@@ -300,7 +301,7 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
ret = afs_end_vnode_operation(&fc);
}
- req->error = ret;
+ req->cache.error = ret;
if (ret == 0) {
afs_stat_v(vnode, n_fetches);
atomic_long_add(req->actual_len,
@@ -337,15 +338,15 @@ static int afs_page_filler(struct key *key, struct page *page)
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(key);
- req->pos = (loff_t)page->index << PAGE_SHIFT;
- req->len = PAGE_SIZE;
- req->nr_pages = 1;
- req->done = afs_file_read_done;
+ req->cache.nr_pages = 1;
+ req->cache.pos = (loff_t)page->index << PAGE_SHIFT;
+ req->cache.len = PAGE_SIZE;
+ req->cache.io_done = afs_file_read_done;
req->cleanup = afs_file_read_cleanup;
get_page(page);
iov_iter_mapping(&req->def_iter, READ, page->mapping,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
@@ -444,10 +445,10 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
refcount_set(&req->usage, 1);
req->vnode = vnode;
req->key = key_get(afs_file_key(file));
- req->done = afs_file_read_done;
req->cleanup = afs_file_read_cleanup;
- req->pos = first->index;
- req->pos <<= PAGE_SHIFT;
+ req->cache.io_done = afs_file_read_done;
+ req->cache.pos = first->index;
+ req->cache.pos <<= PAGE_SHIFT;
/* Add pages to the LRU until it fails. We keep the pages ref'd and
* locked until the read is complete.
@@ -467,17 +468,17 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
break;
}
- req->nr_pages++;
- } while (req->nr_pages < n);
+ req->cache.nr_pages++;
+ } while (req->cache.nr_pages < n);
- if (req->nr_pages == 0) {
+ if (req->cache.nr_pages == 0) {
afs_put_read(req);
return 0;
}
- req->len = req->nr_pages * PAGE_SIZE;
+ req->cache.len = req->cache.nr_pages * PAGE_SIZE;
iov_iter_mapping(&req->def_iter, READ, file->f_mapping,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
req->iter = &req->def_iter;
ret = afs_fetch_data(vnode, req);
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index db80c2618778..c9789294fc68 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -361,7 +361,7 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
call->unmarshall++;
call->iter = req->iter;
- call->iov_len = min(req->actual_len, req->len);
+ call->iov_len = min(req->actual_len, req->cache.len);
/* Fall through */
/* extract the returned data */
@@ -374,17 +374,17 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
return ret;
call->iter = &call->def_iter;
- if (req->actual_len <= req->len)
+ if (req->actual_len <= req->cache.len)
goto no_more_data;
/* Discard any excess data the server gave us */
- afs_extract_discard(call, req->actual_len - req->len);
+ afs_extract_discard(call, req->actual_len - req->cache.len);
call->unmarshall = 3;
/* Fall through */
case 3:
_debug("extract discard %zu/%llu",
- iov_iter_count(call->iter), req->actual_len - req->len);
+ iov_iter_count(call->iter), req->actual_len - req->cache.len);
ret = afs_extract_data(call, true);
if (ret < 0)
@@ -417,8 +417,8 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
break;
}
- if (req->done)
- req->done(req);
+ if (req->cache.io_done)
+ req->cache.io_done(&req->cache);
_leave(" = 0 [done]");
return 0;
@@ -478,10 +478,10 @@ static int afs_fs_fetch_data64(struct afs_fs_cursor *fc,
bp[1] = htonl(vnode->fid.vid);
bp[2] = htonl(vnode->fid.vnode);
bp[3] = htonl(vnode->fid.unique);
- bp[4] = htonl(upper_32_bits(req->pos));
- bp[5] = htonl(lower_32_bits(req->pos));
+ bp[4] = htonl(upper_32_bits(req->cache.pos));
+ bp[5] = htonl(lower_32_bits(req->cache.pos));
bp[6] = 0;
- bp[7] = htonl(lower_32_bits(req->len));
+ bp[7] = htonl(lower_32_bits(req->cache.len));
afs_use_fs_server(call, fc->cbi);
trace_afs_make_fs_call(call, &vnode->fid);
@@ -505,9 +505,9 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
if (test_bit(AFS_SERVER_FL_IS_YFS, &fc->cbi->server->flags))
return yfs_fs_fetch_data(fc, scb, req);
- if (upper_32_bits(req->pos) ||
- upper_32_bits(req->len) ||
- upper_32_bits(req->pos + req->len))
+ if (upper_32_bits(req->cache.pos) ||
+ upper_32_bits(req->cache.len) ||
+ upper_32_bits(req->cache.pos + req->cache.len))
return afs_fs_fetch_data64(fc, scb, req);
_enter("");
@@ -528,8 +528,8 @@ int afs_fs_fetch_data(struct afs_fs_cursor *fc,
bp[1] = htonl(vnode->fid.vid);
bp[2] = htonl(vnode->fid.vnode);
bp[3] = htonl(vnode->fid.unique);
- bp[4] = htonl(lower_32_bits(req->pos));
- bp[5] = htonl(lower_32_bits(req->len));
+ bp[4] = htonl(lower_32_bits(req->cache.pos));
+ bp[5] = htonl(lower_32_bits(req->cache.len));
afs_use_fs_server(call, fc->cbi);
trace_afs_make_fs_call(call, &vnode->fid);
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 0cd9e998d52c..9f56acdc2ed9 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -218,8 +218,9 @@ static inline struct key *afs_file_key(struct file *file)
* Record of an outstanding read operation on a vnode.
*/
struct afs_read {
- loff_t pos; /* Where to start reading */
- loff_t len; /* How much we're asking for */
+ struct fscache_io_request cache;
+ struct iov_iter def_iter; /* Default iterator */
+ struct iov_iter *iter; /* Iterator to use */
loff_t actual_len; /* How much we're actually getting */
loff_t file_size; /* File size returned by server */
struct key *key; /* The key to use to reissue the read */
@@ -227,12 +228,7 @@ struct afs_read {
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
unsigned int call_debug_id;
- unsigned int nr_pages;
- int error;
- void (*done)(struct afs_read *);
- void (*cleanup)(struct afs_read *);
- struct iov_iter *iter; /* Iterator representing the buffer */
- struct iov_iter def_iter; /* Default iterator */
+ void (*cleanup)(struct afs_read *req);
};
/*
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 44dd4d0bad70..390fee44446c 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -51,9 +51,9 @@ static int afs_fill_page(struct file *file,
refcount_set(&req->usage, 1);
req->key = afs_file_key(file);
- req->pos = pos;
- req->len = len;
- req->nr_pages = 1;
+ req->cache.pos = pos;
+ req->cache.len = len;
+ req->cache.nr_pages = 1;
iov_iter_mapping(&req->def_iter, READ, vnode->vfs_inode.i_mapping,
pos, len);
req->iter = &req->def_iter;
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 0b744a117dde..6ea97233c0d2 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -471,7 +471,7 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
call->unmarshall++;
call->iter = req->iter;
- call->iov_len = min(req->actual_len, req->len);
+ call->iov_len = min(req->actual_len, req->cache.len);
/* Fall through */
/* extract the returned data */
@@ -484,17 +484,17 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
return ret;
call->iter = &call->def_iter;
- if (req->actual_len <= req->len)
+ if (req->actual_len <= req->cache.len)
goto no_more_data;
/* Discard any excess data the server gave us */
- afs_extract_discard(call, req->actual_len - req->len);
+ afs_extract_discard(call, req->actual_len - req->cache.len);
call->unmarshall = 3;
/* Fall through */
case 3:
_debug("extract discard %zu/%llu",
- iov_iter_count(call->iter), req->actual_len - req->len);
+ iov_iter_count(call->iter), req->actual_len - req->cache.len);
ret = afs_extract_data(call, true);
if (ret < 0)
@@ -531,8 +531,8 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
break;
}
- if (req->done)
- req->done(req);
+ if (req->cache.io_done)
+ req->cache.io_done(&req->cache);
_leave(" = 0 [done]");
return 0;
@@ -567,7 +567,7 @@ int yfs_fs_fetch_data(struct afs_fs_cursor *fc, struct afs_status_cb *scb,
_enter(",%x,{%llx:%llu},%llx,%llx",
key_serial(fc->key), vnode->fid.vid, vnode->fid.vnode,
- req->pos, req->len);
+ req->cache.pos, req->cache.len);
call = afs_alloc_flat_call(net, &yfs_RXYFSFetchData64,
sizeof(__be32) * 2 +
@@ -589,8 +589,8 @@ int yfs_fs_fetch_data(struct afs_fs_cursor *fc, struct afs_status_cb *scb,
bp = xdr_encode_u32(bp, YFSFETCHDATA64);
bp = xdr_encode_u32(bp, 0); /* RPC flags */
bp = xdr_encode_YFSFid(bp, &vnode->fid);
- bp = xdr_encode_u64(bp, req->pos);
- bp = xdr_encode_u64(bp, req->len);
+ bp = xdr_encode_u64(bp, req->cache.pos);
+ bp = xdr_encode_u64(bp, req->cache.len);
yfs_check_req(call, bp);
afs_use_fs_server(call, fc->cbi);
^ permalink raw reply related [flat|nested] 92+ messages in thread
* [RFC PATCH 53/61] afs: Note the amount transferred in fetch-data delivery
2020-05-04 17:07 ` David Howells
` (52 preceding siblings ...)
(?)
@ 2020-05-04 17:15 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Note the amount of data transferred in the fscache request op structure in
the delivery/decode routines for the various FetchData operations.
Also, we need to exclude the excess from this value and then we need to use
this in directory read rather than actual_len.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/dir.c | 9 ++++-----
fs/afs/fsclient.c | 5 +++++
fs/afs/yfsclient.c | 5 +++++
3 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 0b3f33269fdd..577c975c13b0 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -211,9 +211,8 @@ static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
pr_warn("DIR %llx:%llx f=%llx l=%llx al=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode,
req->file_size, req->cache.len, req->actual_len);
- pr_warn("DIR %llx %x %zx %zx\n",
- req->cache.pos, req->cache.nr_pages,
- req->iter->iov_offset, iov_iter_count(req->iter));
+ pr_warn("DIR %llx %x %llx\n",
+ req->cache.pos, req->cache.nr_pages, req->cache.transferred);
xas_for_each(&xas, page, last) {
if (xas_retry(&xas, page))
@@ -323,7 +322,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
nr_pages = (i_size + PAGE_SIZE - 1) / PAGE_SIZE;
- req->actual_len = i_size; /* May change */
+ req->cache.transferred = i_size; /* May change */
req->cache.len = nr_pages * PAGE_SIZE; /* We can ask for more than there is */
req->data_version = dvnode->status.data_version; /* May change */
iov_iter_mapping(&req->def_iter, READ, dvnode->vfs_inode.i_mapping,
@@ -548,7 +547,7 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
/* walk through the blocks in sequence */
ret = 0;
- while (ctx->pos < req->actual_len) {
+ while (ctx->pos < req->cache.transferred) {
blkoff = ctx->pos & ~(sizeof(union afs_xdr_dir_block) - 1);
/* Fetch the appropriate page from the directory and re-add it
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index c9789294fc68..62cc8072874b 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -417,6 +417,11 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
break;
}
+ /* Pass the call's ref on the read request descriptor to the completion
+ * handler.
+ */
+ req->cache.transferred = min(req->actual_len, req->cache.len);
+ set_bit(FSCACHE_IO_DATA_FROM_SERVER, &req->cache.flags);
if (req->cache.io_done)
req->cache.io_done(&req->cache);
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 6ea97233c0d2..fb3f006be31c 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -531,6 +531,11 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
break;
}
+ /* Pass the call's ref on the read request descriptor to the completion
+ * handler.
+ */
+ req->cache.transferred = min(req->actual_len, req->cache.len);
+ set_bit(FSCACHE_IO_DATA_FROM_SERVER, &req->cache.flags);
if (req->cache.io_done)
req->cache.io_done(&req->cache);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-04 17:07 ` David Howells
` (53 preceding siblings ...)
(?)
@ 2020-05-04 17:15 ` David Howells
2020-05-05 11:59 ` Matthew Wilcox
2020-05-06 7:57 ` David Howells
-1 siblings, 2 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
PG_fscache is going to be used to indicate that a page is being written to
the cache, and that the page should not be modified or released until it's
finished.
Make afs_invalidatepage() and afs_releasepage() wait for it.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/file.c | 13 +++++++++++++
fs/afs/write.c | 9 +++++++++
2 files changed, 22 insertions(+)
diff --git a/fs/afs/file.c b/fs/afs/file.c
index ea9f6d45d9ff..b25c5ab1f4e1 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -548,6 +548,11 @@ static void afs_invalidatepage(struct page *page, unsigned int offset,
/* we clean up only if the entire page is being invalidated */
if (offset == 0 && length == PAGE_SIZE) {
+#ifdef CONFIG_AFS_FSCACHE
+ if (PageFsCache(page))
+ wait_on_page_fscache(page);
+#endif
+
if (PagePrivate(page)) {
priv = page_private(page);
trace_afs_page_dirty(vnode, tracepoint_string("inval"),
@@ -575,6 +580,14 @@ static int afs_releasepage(struct page *page, gfp_t gfp_flags)
/* deny if page is being written to the cache and the caller hasn't
* elected to wait */
+#ifdef CONFIG_AFS_FSCACHE
+ if (PageFsCache(page)) {
+ if (!(gfp_flags & __GFP_DIRECT_RECLAIM) || !(gfp_flags & __GFP_FS))
+ return false;
+ wait_on_page_fscache(page);
+ }
+#endif
+
if (PagePrivate(page)) {
priv = page_private(page);
trace_afs_page_dirty(vnode, tracepoint_string("rel"),
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 390fee44446c..3632909fcd91 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -111,6 +111,10 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
SetPageUptodate(page);
}
+#ifdef CONFIG_AFS_FSCACHE
+ wait_on_page_fscache(page);
+#endif
+
/* page won't leak in error case: it eventually gets cleaned off LRU */
*pagep = page;
@@ -786,6 +790,11 @@ vm_fault_t afs_page_mkwrite(struct vm_fault *vmf)
/* Wait for the page to be written to the cache before we allow it to
* be modified. We then assume the entire page will need writing back.
*/
+#ifdef CONFIG_AFS_FSCACHE
+ if (PageFsCache(vmf->page) &&
+ wait_on_page_bit_killable(vmf->page, PG_fscache) < 0)
+ return VM_FAULT_RETRY;
+#endif
if (PageWriteback(vmf->page) &&
wait_on_page_bit_killable(vmf->page, PG_writeback) < 0)
^ permalink raw reply related [flat|nested] 92+ messages in thread* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-04 17:15 ` [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page David Howells
@ 2020-05-05 11:59 ` Matthew Wilcox
2020-05-06 7:57 ` David Howells
1 sibling, 0 replies; 92+ messages in thread
From: Matthew Wilcox @ 2020-05-05 11:59 UTC (permalink / raw)
To: David Howells
Cc: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton,
Alexander Viro, linux-afs, linux-nfs, linux-cifs, ceph-devel,
v9fs-developer, linux-fsdevel, linux-kernel
On Mon, May 04, 2020 at 06:15:39PM +0100, David Howells wrote:
> PG_fscache is going to be used to indicate that a page is being written to
> the cache, and that the page should not be modified or released until it's
> finished.
>
> Make afs_invalidatepage() and afs_releasepage() wait for it.
Well, why? Keeping a refcount on the page will prevent it from going
away while it's being written to storage. And the fact that it's
being written to this cache is no reason to delay the truncate of a file
(is it?) Similarly, I don't see why we need to wait for the page to make
it to the cache before we start to modify it. Certainly we'll need to
re-write it to the cache since the cache is now stale, but why should
we wait for the now-stale write to complete?
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-04 17:15 ` [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2020-05-05 11:59 ` Matthew Wilcox
@ 2020-05-06 7:57 ` David Howells
2020-05-06 11:09 ` Matthew Wilcox
` (2 more replies)
1 sibling, 3 replies; 92+ messages in thread
From: David Howells @ 2020-05-06 7:57 UTC (permalink / raw)
To: Matthew Wilcox
Cc: dhowells, Trond Myklebust, Anna Schumaker, Steve French,
Jeff Layton, Alexander Viro, linux-afs, linux-nfs, linux-cifs,
ceph-devel, v9fs-developer, linux-fsdevel, linux-kernel
Matthew Wilcox <willy@infradead.org> wrote:
> > PG_fscache is going to be used to indicate that a page is being written to
> > the cache, and that the page should not be modified or released until it's
> > finished.
> >
> > Make afs_invalidatepage() and afs_releasepage() wait for it.
>
> Well, why? Keeping a refcount on the page will prevent it from going
> away while it's being written to storage. And the fact that it's
> being written to this cache is no reason to delay the truncate of a file
> (is it?)
Won't that screw up ITER_MAPPING? Does that mean that ITER_MAPPING isn't
viable?
David
^ permalink raw reply [flat|nested] 92+ messages in thread* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-06 7:57 ` David Howells
@ 2020-05-06 11:09 ` Matthew Wilcox
2020-05-06 14:24 ` David Howells
2020-05-08 14:39 ` David Howells
2 siblings, 0 replies; 92+ messages in thread
From: Matthew Wilcox @ 2020-05-06 11:09 UTC (permalink / raw)
To: David Howells
Cc: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton,
Alexander Viro, linux-afs, linux-nfs, linux-cifs, ceph-devel,
v9fs-developer, linux-fsdevel, linux-kernel
On Wed, May 06, 2020 at 08:57:58AM +0100, David Howells wrote:
> Matthew Wilcox <willy@infradead.org> wrote:
>
> > > PG_fscache is going to be used to indicate that a page is being written to
> > > the cache, and that the page should not be modified or released until it's
> > > finished.
> > >
> > > Make afs_invalidatepage() and afs_releasepage() wait for it.
> >
> > Well, why? Keeping a refcount on the page will prevent it from going
> > away while it's being written to storage. And the fact that it's
> > being written to this cache is no reason to delay the truncate of a file
> > (is it?)
>
> Won't that screw up ITER_MAPPING? Does that mean that ITER_MAPPING isn't
> viable?
Can you remind me why ITER_MAPPING needs:
"The caller must guarantee that the pages are all present and they must be
locked using PG_locked, PG_writeback or PG_fscache to prevent them from
going away or being migrated whilst they're being accessed."
An elevated refcount prevents migration, and it also prevents the pages
from being freed. It doesn't prevent them from being truncated out of
the file, but it does ensure the pages aren't reallocated.
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-06 7:57 ` David Howells
2020-05-06 11:09 ` Matthew Wilcox
@ 2020-05-06 14:24 ` David Howells
2020-05-08 14:39 ` David Howells
2 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-06 14:24 UTC (permalink / raw)
To: Matthew Wilcox
Cc: dhowells, Trond Myklebust, Anna Schumaker, Steve French,
Jeff Layton, Alexander Viro, linux-afs, linux-nfs, linux-cifs,
ceph-devel, v9fs-developer, linux-fsdevel, linux-kernel
Matthew Wilcox <willy@infradead.org> wrote:
> > Won't that screw up ITER_MAPPING? Does that mean that ITER_MAPPING isn't
> > viable?
>
> Can you remind me why ITER_MAPPING needs:
>
> "The caller must guarantee that the pages are all present and they must be
> locked using PG_locked, PG_writeback or PG_fscache to prevent them from
> going away or being migrated whilst they're being accessed."
>
> An elevated refcount prevents migration, and it also prevents the pages
> from being freed. It doesn't prevent them from being truncated out of
> the file, but it does ensure the pages aren't reallocated.
ITER_MAPPING relies on the mapping to maintain the pointers to the pages so
that it can find them rather than being like ITER_BVEC where there's a
separate list.
Truncate removes the pages from the mapping - at which point ITER_MAPPING can
no longer find them.
David
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
2020-05-06 7:57 ` David Howells
@ 2020-05-08 14:39 ` David Howells
2020-05-06 14:24 ` David Howells
2020-05-08 14:39 ` David Howells
2 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-08 14:39 UTC (permalink / raw)
Cc: dhowells, Matthew Wilcox, Trond Myklebust, Anna Schumaker,
Steve French, Jeff Layton, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
David Howells <dhowells@redhat.com> wrote:
> ITER_MAPPING relies on the mapping to maintain the pointers to the pages so
> that it can find them rather than being like ITER_BVEC where there's a
> separate list.
>
> Truncate removes the pages from the mapping - at which point ITER_MAPPING can
> no longer find them.
It looks like ITER_MAPPING is fine with truncate, provided the invalidation
waits for the iterator to complete first:
int truncate_inode_page(struct address_space *mapping, struct page *page)
{
VM_BUG_ON_PAGE(PageTail(page), page);
if (page->mapping != mapping)
return -EIO;
truncate_cleanup_page(mapping, page);
delete_from_page_cache(page);
return 0;
}
In which case, ->invalidatepage() needs to wait for PG_fscache.
Similarly, it looks like ->releasepage() is fine, provided it waits for
PG_fscache also.
If I have to use ITER_BVEC, what's the advisability of using vmalloc() to
allocate the bio_vec array for a transient op? Such an array can reference up
to 1MiB on a 64-bit machine with 4KiB non-compound pages if it only allocates
up to a single page. I'm wondering what the teardown cost is, though, if all
the corresponding PTEs have to be erased from all CPUs.
David
^ permalink raw reply [flat|nested] 92+ messages in thread* Re: [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page
@ 2020-05-08 14:39 ` David Howells
0 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-08 14:39 UTC (permalink / raw)
Cc: dhowells, Matthew Wilcox, Trond Myklebust, Anna Schumaker,
Steve French, Jeff Layton, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
David Howells <dhowells@redhat.com> wrote:
> ITER_MAPPING relies on the mapping to maintain the pointers to the pages so
> that it can find them rather than being like ITER_BVEC where there's a
> separate list.
>
> Truncate removes the pages from the mapping - at which point ITER_MAPPING can
> no longer find them.
It looks like ITER_MAPPING is fine with truncate, provided the invalidation
waits for the iterator to complete first:
int truncate_inode_page(struct address_space *mapping, struct page *page)
{
VM_BUG_ON_PAGE(PageTail(page), page);
if (page->mapping != mapping)
return -EIO;
truncate_cleanup_page(mapping, page);
delete_from_page_cache(page);
return 0;
}
In which case, ->invalidatepage() needs to wait for PG_fscache.
Similarly, it looks like ->releasepage() is fine, provided it waits for
PG_fscache also.
If I have to use ITER_BVEC, what's the advisability of using vmalloc() to
allocate the bio_vec array for a transient op? Such an array can reference up
to 1MiB on a 64-bit machine with 4KiB non-compound pages if it only allocates
up to a single page. I'm wondering what the teardown cost is, though, if all
the corresponding PTEs have to be erased from all CPUs.
David
^ permalink raw reply [flat|nested] 92+ messages in thread
* [RFC PATCH 55/61] afs: Use new fscache I/O API
2020-05-04 17:07 ` David Howells
` (54 preceding siblings ...)
(?)
@ 2020-05-04 17:15 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Make AFS use the new fscache I/O API to read and write from the cache.
afs_readpage() now calls fscache_read_helper() once to try and create a
block around the page to be read.
afs_readpages() now calls fscache_read_helper() multiple times until its
list is exhausted or an error occurs.
afs_prefetch_for_write() is provided to be called from afs_write_begin() to
load the data that will be overwritten by the write into the cache,
extending the read as necessary. This guarantees that the page it returns
will be up to date, rendering it unnecessary for afs_write_end() to fill in
the gaps.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/dir.c | 1
fs/afs/file.c | 409 ++++++++++++++++++++++++++++------------------------
fs/afs/fsclient.c | 8 +
fs/afs/internal.h | 11 +
fs/afs/write.c | 91 +-----------
fs/afs/yfsclient.c | 8 +
6 files changed, 255 insertions(+), 273 deletions(-)
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 577c975c13b0..d04ab47e505c 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -305,6 +305,7 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
req->vnode = dvnode;
req->cleanup = afs_dir_read_cleanup;
req->cache.io_done = afs_dir_read_done;
+ fscache_init_io_request(&req->cache, NULL, NULL);
expand:
i_size = i_size_read(&dvnode->vfs_inode);
diff --git a/fs/afs/file.c b/fs/afs/file.c
index b25c5ab1f4e1..945930462492 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -197,60 +197,70 @@ int afs_release(struct inode *inode, struct file *file)
}
/*
- * Handle completion of a read operation.
+ * Dispose of our locks and refs on the pages if the read failed.
*/
-static void afs_file_read_done(struct fscache_io_request *fsreq)
+static void afs_file_read_cleanup(struct afs_read *req)
{
- struct afs_read *req = container_of(fsreq, struct afs_read, cache);
struct afs_vnode *vnode = req->vnode;
struct page *page;
pgoff_t index = req->cache.pos >> PAGE_SHIFT;
pgoff_t last = index + req->cache.nr_pages - 1;
- XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
+ _enter("%lx,%x,%llx", index, req->cache.nr_pages, req->cache.len);
- if (iov_iter_count(req->iter) > 0) {
- /* The read was short - clear the excess buffer. */
- _debug("afterclear %zx %zx %llx/%llx",
- req->iter->iov_offset,
- iov_iter_count(req->iter),
- req->actual_len, req->cache.len);
- iov_iter_zero(iov_iter_count(req->iter), req->iter);
- }
+ if (req->cache.nr_pages > 0) {
+ XA_STATE(xas, &vnode->vfs_inode.i_mapping->i_pages, index);
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ BUG_ON(xa_is_value(page));
+ BUG_ON(PageCompound(page));
- rcu_read_lock();
- xas_for_each(&xas, page, last) {
- page_endio(page, false, 0);
- put_page(page);
+ if (req->cache.error)
+ page_endio(page, false, req->cache.error);
+ else
+ unlock_page(page);
+ put_page(page);
+ }
+ rcu_read_unlock();
}
- rcu_read_unlock();
- task_io_account_read(req->cache.len);
- req->cleanup = NULL;
+ if (test_bit(AFS_READ_IN_PROGRESS, &req->flags)) {
+ clear_bit_unlock(AFS_READ_IN_PROGRESS, &req->flags);
+ wake_up_bit(&req->flags, AFS_READ_IN_PROGRESS);
+ }
}
/*
- * Dispose of our locks and refs on the pages if the read failed.
+ * Allocate a new read record.
*/
-static void afs_file_read_cleanup(struct afs_read *req)
+struct afs_read *afs_alloc_read(gfp_t gfp)
{
- struct page *page;
- pgoff_t index = req->cache.pos >> PAGE_SHIFT;
- pgoff_t last = index + req->cache.nr_pages - 1;
+ static atomic_t debug_ids;
+ struct afs_read *req;
- XA_STATE(xas, &req->iter->mapping->i_pages, index);
+ req = kzalloc(sizeof(struct afs_read), gfp);
+ if (req) {
+ refcount_set(&req->usage, 1);
+ req->debug_id = atomic_inc_return(&debug_ids);
+ __set_bit(AFS_READ_IN_PROGRESS, &req->flags);
+ }
- _enter("%lu,%u,%zu", index, req->cache.nr_pages, iov_iter_count(req->iter));
+ return req;
+}
- rcu_read_lock();
- xas_for_each(&xas, page, last) {
- BUG_ON(xa_is_value(page));
- BUG_ON(PageCompound(page));
+/*
+ *
+ */
+static void __afs_put_read(struct work_struct *work)
+{
+ struct afs_read *req = container_of(work, struct afs_read, cache.work);
- page_endio(page, false, req->cache.error);
- put_page(page);
- }
- rcu_read_unlock();
+ if (req->cleanup)
+ req->cleanup(req);
+ fscache_free_io_request(&req->cache);
+ key_put(req->key);
+ kfree(req);
}
/*
@@ -259,10 +269,13 @@ static void afs_file_read_cleanup(struct afs_read *req)
void afs_put_read(struct afs_read *req)
{
if (refcount_dec_and_test(&req->usage)) {
- if (req->cleanup)
- req->cleanup(req);
- key_put(req->key);
- kfree(req);
+ _debug("dead %u", req->debug_id);
+ if (in_softirq()) {
+ INIT_WORK(&req->cache.work, __afs_put_read);
+ queue_work(afs_wq, &req->cache.work);
+ } else {
+ __afs_put_read(&req->cache.work);
+ }
}
}
@@ -313,190 +326,130 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
return ret;
}
-/*
- * read page from file, directory or symlink, given a key to use
- */
-static int afs_page_filler(struct key *key, struct page *page)
+void afs_req_issue_op(struct fscache_io_request *fsreq)
{
- struct inode *inode = page->mapping->host;
- struct afs_vnode *vnode = AFS_FS_I(inode);
- struct afs_read *req;
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
int ret;
- _enter("{%x},{%lu},{%lu}", key_serial(key), inode->i_ino, page->index);
-
- BUG_ON(!PageLocked(page));
-
- ret = -ESTALE;
- if (test_bit(AFS_VNODE_DELETED, &vnode->flags))
- goto error;
-
- req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
- if (!req)
- goto enomem;
-
- refcount_set(&req->usage, 1);
- req->vnode = vnode;
- req->key = key_get(key);
- req->cache.nr_pages = 1;
- req->cache.pos = (loff_t)page->index << PAGE_SHIFT;
- req->cache.len = PAGE_SIZE;
- req->cache.io_done = afs_file_read_done;
- req->cleanup = afs_file_read_cleanup;
-
- get_page(page);
- iov_iter_mapping(&req->def_iter, READ, page->mapping,
+ iov_iter_mapping(&req->def_iter, READ, req->cache.mapping,
req->cache.pos, req->cache.len);
req->iter = &req->def_iter;
- ret = afs_fetch_data(vnode, req);
+ ret = afs_fetch_data(req->vnode, req);
if (ret < 0)
- goto fetch_error;
+ req->cache.error = ret;
+}
- afs_put_read(req);
- _leave(" = 0");
- return 0;
+void afs_req_done(struct fscache_io_request *fsreq)
+{
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
-fetch_error:
- switch (ret) {
- case -EINTR:
- case -ENOMEM:
- case -ERESTARTSYS:
- case -EAGAIN:
- afs_put_read(req);
- goto error;
- case -ENOENT:
- _debug("got NOENT from server - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
- /* Fall through */
- default:
- page_endio(page, false, ret);
- afs_put_read(req);
- _leave(" = %d", ret);
- return ret;
+ req->cleanup = NULL;
+ if (test_bit(AFS_READ_IN_PROGRESS, &req->flags)) {
+ clear_bit_unlock(AFS_READ_IN_PROGRESS, &req->flags);
+ wake_up_bit(&req->flags, AFS_READ_IN_PROGRESS);
}
+}
-enomem:
- ret = -ENOMEM;
-error:
- unlock_page(page);
- _leave(" = %d", ret);
- return ret;
+void afs_req_get(struct fscache_io_request *fsreq)
+{
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
+
+ afs_get_read(req);
}
+void afs_req_put(struct fscache_io_request *fsreq)
+{
+ struct afs_read *req = container_of(fsreq, struct afs_read, cache);
+
+ afs_put_read(req);
+}
+
+const struct fscache_io_request_ops afs_req_ops = {
+ .issue_op = afs_req_issue_op,
+ .done = afs_req_done,
+ .get = afs_req_get,
+ .put = afs_req_put,
+};
+
/*
* read page from file, directory or symlink, given a file to nominate the key
* to be used
*/
static int afs_readpage(struct file *file, struct page *page)
{
+ struct fscache_extent extent;
+ struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
+ struct afs_read *req;
struct key *key;
- int ret;
+ int ret = -ENOMEM;
+
+ _enter(",%lx", page->index);
if (file) {
- key = afs_file_key(file);
+ key = key_get(afs_file_key(file));
ASSERT(key != NULL);
- ret = afs_page_filler(key, page);
} else {
- struct inode *inode = page->mapping->host;
- key = afs_request_key(AFS_FS_S(inode->i_sb)->cell);
+ key = afs_request_key(vnode->volume->cell);
if (IS_ERR(key)) {
ret = PTR_ERR(key);
- } else {
- ret = afs_page_filler(key, page);
- key_put(key);
+ goto out;
}
}
- return ret;
-}
-/*
- * Read a contiguous set of pages.
- */
-static int afs_readpages_one(struct file *file, struct address_space *mapping,
- struct list_head *pages)
-{
- struct afs_vnode *vnode = AFS_FS_I(mapping->host);
- struct afs_read *req;
- struct list_head *p;
- struct page *first, *page;
- pgoff_t index;
- int ret, n;
-
- /* Count the number of contiguous pages at the front of the list. Note
- * that the list goes prev-wards rather than next-wards.
- */
- first = lru_to_page(pages);
- index = first->index + 1;
- n = 1;
- for (p = first->lru.prev; p != pages; p = p->prev) {
- page = list_entry(p, struct page, lru);
- if (page->index != index)
- break;
- index++;
- n++;
- }
-
- req = kzalloc(sizeof(struct afs_read), GFP_NOFS);
+ req = afs_alloc_read(GFP_NOFS);
if (!req)
- return -ENOMEM;
+ goto out_key;
- refcount_set(&req->usage, 1);
+ fscache_init_io_request(&req->cache, afs_vnode_cache(vnode), &afs_req_ops);
req->vnode = vnode;
- req->key = key_get(afs_file_key(file));
+ req->key = key;
req->cleanup = afs_file_read_cleanup;
- req->cache.io_done = afs_file_read_done;
- req->cache.pos = first->index;
- req->cache.pos <<= PAGE_SHIFT;
-
- /* Add pages to the LRU until it fails. We keep the pages ref'd and
- * locked until the read is complete.
- *
- * Note that it's possible for the file size to change whilst we're
- * doing this, but we rely on the server returning less than we asked
- * for if the file shrank. We also rely on this to deal with a partial
- * page at the end of the file.
- */
- do {
- page = lru_to_page(pages);
- list_del(&page->lru);
- index = page->index;
- if (add_to_page_cache_lru(page, mapping, index,
- readahead_gfp_mask(mapping))) {
- put_page(page);
- break;
- }
+ req->cache.mapping = page->mapping;
- req->cache.nr_pages++;
- } while (req->cache.nr_pages < n);
+ extent.start = page->index;
+ extent.block_end = page->index + 1;
+ extent.limit = ULONG_MAX;
- if (req->cache.nr_pages == 0) {
- afs_put_read(req);
- return 0;
- }
+ ret = fscache_read_helper(&req->cache, &extent, &page, NULL,
+ FSCACHE_READ_LOCKED_PAGE, 0);
+ afs_put_read(req);
+ return ret;
- req->cache.len = req->cache.nr_pages * PAGE_SIZE;
- iov_iter_mapping(&req->def_iter, READ, file->f_mapping,
- req->cache.pos, req->cache.len);
- req->iter = &req->def_iter;
+out_key:
+ key_put(key);
+out:
+ return ret;
+}
- ret = afs_fetch_data(vnode, req);
- if (ret < 0)
- goto error;
+/*
+ * Determine the extent of contiguous pages at the front of the list.
+ * Note that the list goes prev-wards rather than next-wards.
+ *
+ * We also determine the last page we can include in a transaction - we stop
+ * if there's a non-contiguity in the page list, but we include the gap.
+ */
+static void afs_count_contig(struct list_head *pages,
+ struct fscache_extent *extent)
+{
+ struct list_head *p;
+ struct page *first = lru_to_page(pages), *page;
- afs_put_read(req);
- return 0;
+ extent->start = first->index;
+ extent->block_end = first->index + 1;
+ extent->limit = ULONG_MAX;
-error:
- if (ret == -ENOENT) {
- _debug("got NOENT from server - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
+ for (p = first->lru.prev; p != pages; p = p->prev) {
+ page = list_entry(p, struct page, lru);
+ if (page->index != extent->block_end) {
+ extent->limit = page->index;
+ break;
+ }
+ extent->block_end = page->index + 1;
}
- afs_put_read(req);
- return ret;
+ _leave(" [%lx,%lx,%lx]",
+ extent->start, extent->block_end, extent->limit);
}
/*
@@ -505,14 +458,12 @@ static int afs_readpages_one(struct file *file, struct address_space *mapping,
static int afs_readpages(struct file *file, struct address_space *mapping,
struct list_head *pages, unsigned nr_pages)
{
- struct key *key = afs_file_key(file);
+ struct fscache_extent extent;
struct afs_vnode *vnode;
+ struct afs_read *req;
int ret = 0;
- _enter("{%d},{%lu},,%d",
- key_serial(key), mapping->host->i_ino, nr_pages);
-
- ASSERT(key != NULL);
+ _enter(",{%lu},,%x", mapping->host->i_ino, nr_pages);
vnode = AFS_FS_I(mapping->host);
if (test_bit(AFS_VNODE_DELETED, &vnode->flags)) {
@@ -520,9 +471,26 @@ static int afs_readpages(struct file *file, struct address_space *mapping,
return -ESTALE;
}
- /* attempt to read as many of the pages as possible */
while (!list_empty(pages)) {
- ret = afs_readpages_one(file, mapping, pages);
+ /* Determine the size of the next contiguous run of pages and
+ * find out what size of download will be required to pad it
+ * out to a whole number of cache blocks.
+ */
+ afs_count_contig(pages, &extent);
+ req = afs_alloc_read(GFP_NOFS);
+ if (!req)
+ return -ENOMEM;
+
+ fscache_init_io_request(&req->cache, afs_vnode_cache(vnode),
+ &afs_req_ops);
+ req->vnode = AFS_FS_I(mapping->host);
+ req->key = key_get(afs_file_key(file));
+ req->cleanup = afs_file_read_cleanup;
+ req->cache.mapping = mapping;
+
+ ret = fscache_read_helper(&req->cache, &extent, NULL, pages,
+ FSCACHE_READ_PAGE_LIST, 0);
+ afs_put_read(req);
if (ret < 0)
break;
}
@@ -531,6 +499,71 @@ static int afs_readpages(struct file *file, struct address_space *mapping,
return ret;
}
+/*
+ * Prefetch data into the cache prior to writing, returning the requested page
+ * to the caller, with the lock held, upon completion of the write.
+ */
+struct page *afs_prefetch_for_write(struct file *file,
+ struct address_space *mapping,
+ pgoff_t index,
+ unsigned int aop_flags)
+{
+ struct fscache_extent extent;
+ struct afs_vnode *vnode = AFS_FS_I(mapping->host);
+ struct afs_read *req;
+ struct page *page;
+ int ret = 0;
+
+ _enter("{%lu},%lx", mapping->host->i_ino, index);
+
+ if (test_bit(AFS_VNODE_DELETED, &vnode->flags)) {
+ _leave(" = -ESTALE");
+ return ERR_PTR(-ESTALE);
+ }
+
+ page = pagecache_get_page(mapping, index, FGP_WRITE, 0);
+ if (page) {
+ if (PageUptodate(page)) {
+ lock_page(page);
+ if (PageUptodate(page))
+ goto have_page;
+ unlock_page(page);
+ }
+ }
+
+ extent.start = index;
+ extent.block_end = index + 1;
+ extent.limit = ULONG_MAX;
+
+ req = afs_alloc_read(GFP_NOFS);
+ if (!req)
+ return ERR_PTR(-ENOMEM);
+
+ fscache_init_io_request(&req->cache, afs_vnode_cache(vnode), &afs_req_ops);
+ req->vnode = AFS_FS_I(mapping->host);
+ req->key = key_get(afs_file_key(file));
+ req->cleanup = afs_file_read_cleanup;
+ req->cache.mapping = mapping;
+
+ ret = fscache_read_helper(&req->cache, &extent, &page, NULL,
+ FSCACHE_READ_FOR_WRITE, aop_flags);
+ if (ret == 0)
+ /* Synchronicity required */
+ ret = wait_on_bit(&req->flags, AFS_READ_IN_PROGRESS, TASK_KILLABLE);
+
+ afs_put_read(req);
+
+ if (ret < 0) {
+ if (page)
+ put_page(page);
+ return ERR_PTR(ret);
+ }
+
+have_page:
+ wait_for_stable_page(page);
+ return page;
+}
+
/*
* invalidate part or all of a page
* - release a page and clean up its private data if offset is 0 (indicating
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 62cc8072874b..0d0fbc594b52 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -422,8 +422,11 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
*/
req->cache.transferred = min(req->actual_len, req->cache.len);
set_bit(FSCACHE_IO_DATA_FROM_SERVER, &req->cache.flags);
- if (req->cache.io_done)
+ if (req->cache.io_done) {
req->cache.io_done(&req->cache);
+ afs_put_read(req);
+ call->read_request = NULL;
+ }
_leave(" = 0 [done]");
return 0;
@@ -433,7 +436,8 @@ static void afs_fetch_data_destructor(struct afs_call *call)
{
struct afs_read *req = call->read_request;
- afs_put_read(req);
+ if (req)
+ afs_put_read(req);
afs_flat_call_destructor(call);
}
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 9f56acdc2ed9..70b8437d391f 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -227,7 +227,10 @@ struct afs_read {
struct afs_vnode *vnode; /* The file being read into. */
afs_dataversion_t data_version; /* Version number returned by server */
refcount_t usage;
+ unsigned int debug_id;
unsigned int call_debug_id;
+ unsigned long flags;
+#define AFS_READ_IN_PROGRESS 0 /* Set whilst a read is in progress */
void (*cleanup)(struct afs_read *req);
};
@@ -906,6 +909,7 @@ extern void afs_dynroot_depopulate(struct super_block *);
/*
* file.c
*/
+extern const struct fscache_io_request_ops afs_req_ops;
extern const struct address_space_operations afs_fs_aops;
extern const struct inode_operations afs_file_inode_operations;
extern const struct file_operations afs_file_operations;
@@ -915,7 +919,14 @@ extern void afs_put_wb_key(struct afs_wb_key *);
extern int afs_open(struct inode *, struct file *);
extern int afs_release(struct inode *, struct file *);
extern int afs_fetch_data(struct afs_vnode *, struct afs_read *);
+extern struct afs_read *afs_alloc_read(gfp_t);
extern void afs_put_read(struct afs_read *);
+extern void afs_req_issue_op(struct fscache_io_request *);
+extern void afs_req_done(struct fscache_io_request *);
+extern void afs_req_get(struct fscache_io_request *);
+extern void afs_req_put(struct fscache_io_request *);
+extern struct page *afs_prefetch_for_write(struct file *, struct address_space *,
+ pgoff_t, unsigned int);
static inline struct afs_read *afs_get_read(struct afs_read *req)
{
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 3632909fcd91..312d8f07533e 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -22,57 +22,6 @@ int afs_set_page_dirty(struct page *page)
return __set_page_dirty_nobuffers(page);
}
-/*
- * partly or wholly fill a page that's under preparation for writing
- */
-static int afs_fill_page(struct file *file,
- loff_t pos, unsigned int len, struct page *page)
-{
- struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
- struct afs_read *req;
- size_t p;
- void *data;
- int ret;
-
- _enter(",,%llu", (unsigned long long)pos);
-
- if (pos >= vnode->vfs_inode.i_size) {
- p = pos & ~PAGE_MASK;
- ASSERTCMP(p + len, <=, PAGE_SIZE);
- data = kmap(page);
- memset(data + p, 0, len);
- kunmap(page);
- return 0;
- }
-
- req = kzalloc(sizeof(struct afs_read), GFP_KERNEL);
- if (!req)
- return -ENOMEM;
-
- refcount_set(&req->usage, 1);
- req->key = afs_file_key(file);
- req->cache.pos = pos;
- req->cache.len = len;
- req->cache.nr_pages = 1;
- iov_iter_mapping(&req->def_iter, READ, vnode->vfs_inode.i_mapping,
- pos, len);
- req->iter = &req->def_iter;
-
- ret = afs_fetch_data(vnode, req);
- afs_put_read(req);
- if (ret < 0) {
- if (ret == -ENOENT) {
- _debug("got NOENT from server"
- " - marking file deleted and stale");
- set_bit(AFS_VNODE_DELETED, &vnode->flags);
- ret = -ESTALE;
- }
- }
-
- _leave(" = %d", ret);
- return ret;
-}
-
/*
* prepare to perform part of a write to a page
*/
@@ -96,20 +45,15 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
*/
BUILD_BUG_ON(PAGE_SIZE > 32768 && sizeof(page->private) < 8);
- page = grab_cache_page_write_begin(mapping, index, flags);
- if (!page)
- return -ENOMEM;
+ /* Prefetch area to be written into the cache if we're caching this
+ * file. We need to do this before we get a lock on the page in case
+ * there's more than one writer competing for the same cache block.
+ */
+ page = afs_prefetch_for_write(file, mapping, index, flags);
+ if (IS_ERR(page))
+ return PTR_ERR(page);
- if (!PageUptodate(page) && len != PAGE_SIZE) {
- ret = afs_fill_page(file, pos & PAGE_MASK, PAGE_SIZE, page);
- if (ret < 0) {
- unlock_page(page);
- put_page(page);
- _leave(" = %d [prep]", ret);
- return ret;
- }
- SetPageUptodate(page);
- }
+ ASSERT(PageUptodate(page));
#ifdef CONFIG_AFS_FSCACHE
wait_on_page_fscache(page);
@@ -189,7 +133,6 @@ int afs_write_end(struct file *file, struct address_space *mapping,
{
struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
loff_t i_size, maybe_i_size;
- int ret;
_enter("{%llx:%llu},{%lx}",
vnode->fid.vid, vnode->fid.vnode, page->index);
@@ -205,29 +148,15 @@ int afs_write_end(struct file *file, struct address_space *mapping,
spin_unlock(&vnode->wb_lock);
}
- if (!PageUptodate(page)) {
- if (copied < len) {
- /* Try and load any missing data from the server. The
- * unmarshalling routine will take care of clearing any
- * bits that are beyond the EOF.
- */
- ret = afs_fill_page(file, pos + copied,
- len - copied, page);
- if (ret < 0)
- goto out;
- }
- SetPageUptodate(page);
- }
+ ASSERT(PageUptodate(page));
set_page_dirty(page);
if (PageDirty(page))
_debug("dirtied");
- ret = copied;
-out:
unlock_page(page);
put_page(page);
- return ret;
+ return copied;
}
/*
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index fb3f006be31c..74fce9736796 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -536,8 +536,11 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
*/
req->cache.transferred = min(req->actual_len, req->cache.len);
set_bit(FSCACHE_IO_DATA_FROM_SERVER, &req->cache.flags);
- if (req->cache.io_done)
+ if (req->cache.io_done) {
req->cache.io_done(&req->cache);
+ afs_put_read(req);
+ call->read_request = NULL;
+ }
_leave(" = 0 [done]");
return 0;
@@ -545,7 +548,8 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
static void yfs_fetch_data_destructor(struct afs_call *call)
{
- afs_put_read(call->read_request);
+ if (call->read_request)
+ afs_put_read(call->read_request);
afs_flat_call_destructor(call);
}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 56/61] afs: Copy local writes to the cache when writing to the server
2020-05-04 17:07 ` David Howells
` (55 preceding siblings ...)
(?)
@ 2020-05-04 17:15 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:15 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
When writing to the server from afs_writepage() or afs_writepages(), copy
the data to the cache object too.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/write.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 119 insertions(+), 5 deletions(-)
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 312d8f07533e..38637dc6043c 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -13,6 +13,9 @@
#include <linux/pagevec.h>
#include "internal.h"
+static void afs_write_to_cache(struct afs_vnode *vnode,
+ pgoff_t start, pgoff_t end, loff_t a, loff_t b);
+
/*
* mark a page as having been made dirty and thus needing writeback
*/
@@ -389,6 +392,8 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
count = 1;
if (test_set_page_writeback(primary_page))
BUG();
+ if (TestSetPageFsCache(primary_page))
+ BUG();
/* Find all consecutive lockable dirty pages that have contiguous
* written regions, stopping when we find a page that is not
@@ -437,7 +442,8 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
break;
if (!trylock_page(page))
break;
- if (!PageDirty(page) || PageWriteback(page)) {
+ if (!PageDirty(page) || PageWriteback(page) ||
+ PageFsCache(page)) {
unlock_page(page);
break;
}
@@ -459,6 +465,8 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
BUG();
if (test_set_page_writeback(page))
BUG();
+ if (TestSetPageFsCache(page))
+ BUG();
unlock_page(page);
put_page(page);
}
@@ -489,8 +497,13 @@ static int afs_write_back_from_locked_page(struct address_space *mapping,
b = last;
b <<= PAGE_SHIFT;
b += to;
- iov_iter_mapping(&iter, WRITE, mapping, a, b - a);
+ /* Speculatively write to the cache. We have to fix this up later if
+ * the store fails.
+ */
+ afs_write_to_cache(vnode, first, last + 1, a, b);
+
+ iov_iter_mapping(&iter, WRITE, mapping, a, b - a);
ret = afs_store_data(vnode, &iter, a, first, last);
switch (ret) {
case 0:
@@ -543,6 +556,10 @@ int afs_writepage(struct page *page, struct writeback_control *wbc)
_enter("{%lx},", page->index);
+#ifdef CONFIG_AFS_FSCACHE
+ wait_on_page_fscache(page);
+#endif
+
ret = afs_write_back_from_locked_page(page->mapping, wbc, page,
wbc->range_end >> PAGE_SHIFT);
if (ret < 0) {
@@ -570,7 +587,7 @@ static int afs_writepages_region(struct address_space *mapping,
do {
n = find_get_pages_range_tag(mapping, &index, end,
- PAGECACHE_TAG_DIRTY, 1, &page);
+ PAGECACHE_TAG_DIRTY, 1, &page);
if (!n)
break;
@@ -595,10 +612,14 @@ static int afs_writepages_region(struct address_space *mapping,
continue;
}
- if (PageWriteback(page)) {
+ if (PageWriteback(page) || PageFsCache(page)) {
unlock_page(page);
- if (wbc->sync_mode != WB_SYNC_NONE)
+ if (wbc->sync_mode != WB_SYNC_NONE) {
wait_on_page_writeback(page);
+#ifdef CONFIG_AFS_FSCACHE
+ wait_on_page_fscache(page);
+#endif
+ }
put_page(page);
continue;
}
@@ -818,3 +839,96 @@ int afs_launder_page(struct page *page)
ClearPagePrivate(page);
return ret;
}
+
+/*
+ * Clear the PG_fscache flag from a sequence of pages and wake up anyone who's
+ * waiting. The last page is included in the sequence.
+ */
+static void afs_clear_fscache_bits(struct address_space *mapping,
+ pgoff_t start, pgoff_t last)
+{
+ struct page *page;
+
+ XA_STATE(xas, &mapping->i_pages, start);
+
+ rcu_read_lock();
+ xas_for_each(&xas, page, last) {
+ unlock_page_fscache(page);
+ }
+ rcu_read_unlock();
+}
+
+/*
+ * Deal with the completion of writing the data to the cache.
+ */
+static void afs_write_to_cache_done(struct fscache_io_request *_req)
+{
+ struct afs_read *req = container_of(_req, struct afs_read, cache);
+ pgoff_t index = req->cache.pos >> PAGE_SHIFT;
+ pgoff_t last = index + req->cache.nr_pages - 1;
+
+ _enter("%lx,%x,%llx", index, req->cache.nr_pages, req->cache.transferred);
+
+ afs_clear_fscache_bits(req->cache.mapping, index, last);
+
+ if (req->cache.error && req->cache.error != -ENOBUFS) {
+ struct afs_vnode *vnode = req->vnode;
+ struct afs_vnode_cache_aux aux = {
+ .data_version = vnode->status.data_version,
+ };
+ _debug("inval wr %d", req->cache.error);
+ fscache_invalidate(req->cache.cookie, &aux,
+ i_size_read(&vnode->vfs_inode), 0);
+ }
+}
+
+static const struct fscache_io_request_ops afs_write_req_ops = {
+ .get = afs_req_get,
+ .put = afs_req_put,
+};
+
+/*
+ * Save the write to the cache also.
+ */
+static void afs_write_to_cache(struct afs_vnode *vnode,
+ pgoff_t start, pgoff_t end, loff_t a, loff_t b)
+{
+ struct afs_read *req;
+ struct iov_iter iter;
+ unsigned int x;
+
+ struct fscache_extent extent = {
+ .start = start,
+ .block_end = end,
+ .limit = end,
+ };
+
+ _enter("%lx,%lx,%llx,%llx", start, end, a, b);
+
+ x = fscache_shape_extent(afs_vnode_cache(vnode), &extent,
+ i_size_read(&vnode->vfs_inode), true);
+ if (!(x & FSCACHE_WRITE_TO_CACHE))
+ goto abandon;
+
+ req = afs_alloc_read(GFP_KERNEL);
+ if (!req)
+ goto abandon;
+
+ fscache_init_io_request(&req->cache, afs_vnode_cache(vnode),
+ &afs_write_req_ops);
+ req->vnode = vnode;
+ req->cache.pos = round_down(a, extent.dio_block_size);
+ req->cache.len = round_up(b, extent.dio_block_size) - req->cache.pos;
+ req->cache.nr_pages = end - start;
+ req->cache.mapping = vnode->vfs_inode.i_mapping;
+ req->cache.io_done = &afs_write_to_cache_done;
+
+ iov_iter_mapping(&iter, WRITE, req->cache.mapping,
+ req->cache.pos, req->cache.len);
+ fscache_write(&req->cache, &iter);
+ afs_put_read(req);
+ return;
+
+abandon:
+ afs_clear_fscache_bits(vnode->vfs_inode.i_mapping, start, end);
+}
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 57/61] afs: Invoke fscache_resize_cookie() when handling ATTR_SIZE for setattr
2020-05-04 17:07 ` David Howells
` (56 preceding siblings ...)
(?)
@ 2020-05-04 17:16 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:16 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Invoke fscache_resize_cookie() to adjust the size of the backing cache
object when setattr is called with ATTR_SIZE. This discards any data that
then lies beyond the revised EOF and frees up space.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/afs/inode.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 59dc179b40cb..21a48ab74d2b 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -838,6 +838,8 @@ int afs_setattr(struct dentry *dentry, struct iattr *attr)
if (!scb)
goto error;
+ fscache_use_cookie(afs_vnode_cache(vnode), true);
+
/* flush any dirty data outstanding on a regular file */
if (S_ISREG(vnode->vfs_inode.i_mode))
filemap_write_and_wait(vnode->vfs_inode.i_mapping);
@@ -848,7 +850,7 @@ int afs_setattr(struct dentry *dentry, struct iattr *attr)
key = afs_request_key(vnode->volume->cell);
if (IS_ERR(key)) {
ret = PTR_ERR(key);
- goto error_scb;
+ goto error_unuse;
}
}
@@ -873,7 +875,11 @@ int afs_setattr(struct dentry *dentry, struct iattr *attr)
if (!(attr->ia_valid & ATTR_FILE))
key_put(key);
-error_scb:
+ if ((attr->ia_valid & ATTR_SIZE) && ret == 0)
+ fscache_resize_cookie(afs_vnode_cache(vnode), scb->status.size);
+
+error_unuse:
+ fscache_unuse_cookie(afs_vnode_cache(vnode), NULL, NULL);
kfree(scb);
error:
_leave(" = %d", ret);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 58/61] fscache: Rewrite the main document
2020-05-04 17:07 ` David Howells
` (57 preceding siblings ...)
(?)
@ 2020-05-04 17:16 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:16 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Rewrite the main document to reflect the new API.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/filesystems/caching/fscache.txt | 51 ++++++++-----------------
1 file changed, 17 insertions(+), 34 deletions(-)
diff --git a/Documentation/filesystems/caching/fscache.txt b/Documentation/filesystems/caching/fscache.txt
index 50f0a5757f48..dbfa0ece0ce8 100644
--- a/Documentation/filesystems/caching/fscache.txt
+++ b/Documentation/filesystems/caching/fscache.txt
@@ -83,9 +83,6 @@ then serving the pages out of that cache rather than the netfs inode because:
one-off access of a small portion of it (such as might be done with the
"file" program).
-It instead serves the cache out in PAGE_SIZE chunks as and when requested by
-the netfs('s) using it.
-
FS-Cache provides the following facilities:
@@ -109,22 +106,22 @@ FS-Cache provides the following facilities:
recursive, stack space is limited, and indices can only be children of
indices.
- (7) Data I/O is done direct to and from the netfs's pages. The netfs
- indicates that page A is at index B of the data-file represented by cookie
- C, and that it should be read or written. The cache backend may or may
- not start I/O on that page, but if it does, a netfs callback will be
- invoked to indicate completion. The I/O may be either synchronous or
- asynchronous.
+ (7) The cache provides two basic I/O operations: write to the cache and read
+ from the cache. These may be done synchronously or asynchronously and may
+ involve direct I/O. The position and length of the request have to be
+ rounded to the I/O block size of the cache.
+
+ (8) The cache doesn't keep track of any of the netfs state and retains no
+ pointers back into the netfs. The netfs is entirely responsible for
+ telling the cache what to do. A number of helpers are provided to manage
+ the interaction.
(8) Cookies can be "retired" upon release. At this point FS-Cache will mark
them as obsolete and the index hierarchy rooted at that point will get
recycled.
- (9) The netfs provides a "match" function for index searches. In addition to
- saying whether a match was made or not, this can also specify that an
- entry should be updated or deleted.
-
-(10) As much as possible is done asynchronously.
+ (9) Coherency data and index keys are stored in the cookie. This is used by
+ the cache to determine whether the stored data is still valid.
FS-Cache maintains a virtual indexing tree in which all indices, files, objects
@@ -144,33 +141,19 @@ caches.
+------------+ +---------------+ +----------+
| | | | | |
00001 00002 00007 00125 vol00001 vol00002
- | | | | |
- +---+---+ +-----+ +---+ +------+------+ +-----+----+
- | | | | | | | | | | | | |
-PG0 PG1 PG2 PG0 XATTR PG0 PG1 DIRENT DIRENT DIRENT R/W R/O Bak
- | |
- PG0 +-------+
- | |
- 00001 00003
- |
- +---+---+
- | | |
- PG0 PG1 PG2
+ |
+ +-------+
+ | |
+ 00001 00003
In the example above, you can see two netfs's being backed: NFS and AFS. These
have different index hierarchies:
(*) The NFS primary index contains per-server indices. Each server index is
- indexed by NFS file handles to get data file objects. Each data file
- objects can have an array of pages, but may also have further child
- objects, such as extended attributes and directory entries. Extended
- attribute objects themselves have page-array contents.
+ indexed by NFS file handles to get data objects.
(*) The AFS primary index contains per-cell indices. Each cell index contains
- per-logical-volume indices. Each of volume index contains up to three
- indices for the read-write, read-only and backup mirrors of those volumes.
- Each of these contains vnode data file objects, each of which contains an
- array of pages.
+ logical volume indices and each of those contains vnode data file objects.
The very top index is the FS-Cache master index in which individual netfs's
have entries.
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 59/61] fscache: Remove the obsolete API bits from the documentation
2020-05-04 17:07 ` David Howells
` (58 preceding siblings ...)
(?)
@ 2020-05-04 17:16 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:16 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Remove the obsolete API bits from the documentation preparatory to writing
up the new API.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/filesystems/caching/backend-api.txt | 424 ----------------
Documentation/filesystems/caching/netfs-api.txt | 554 +--------------------
Documentation/filesystems/caching/object.txt | 240 ---------
Documentation/filesystems/caching/operations.txt | 213 --------
4 files changed, 21 insertions(+), 1410 deletions(-)
delete mode 100644 Documentation/filesystems/caching/operations.txt
diff --git a/Documentation/filesystems/caching/backend-api.txt b/Documentation/filesystems/caching/backend-api.txt
index c418280c915f..710d10408fcb 100644
--- a/Documentation/filesystems/caching/backend-api.txt
+++ b/Documentation/filesystems/caching/backend-api.txt
@@ -108,27 +108,14 @@ RELEVANT DATA STRUCTURES
(*) Index/Data file FS-Cache representation cookie:
struct fscache_cookie {
- struct fscache_object_def *def;
- struct fscache_netfs *netfs;
- void *netfs_data;
...
};
- The fields that might be of use to the backend describe the object
- definition, the netfs definition and the netfs's data for this cookie.
- The object definition contain functions supplied by the netfs for loading
- and matching index entries; these are required to provide some of the
- cache operations.
-
(*) In-cache object representation:
struct fscache_object {
int debug_id;
- enum {
- FSCACHE_OBJECT_RECYCLING,
- ...
- } state;
spinlock_t lock
struct fscache_cache *cache;
struct fscache_cookie *cookie;
@@ -145,71 +132,8 @@ RELEVANT DATA STRUCTURES
using "OBJ%x" to be consistent with FS-Cache.
Each object contains a pointer to the cookie that represents the object it
- is backing. An object should retired when put_object() is called if it is
- in state FSCACHE_OBJECT_RECYCLING. The fscache_object struct should be
- initialised by calling fscache_object_init(object).
-
-
- (*) FS-Cache operation record:
-
- struct fscache_operation {
- atomic_t usage;
- struct fscache_object *object;
- unsigned long flags;
- #define FSCACHE_OP_EXCLUSIVE
- void (*processor)(struct fscache_operation *op);
- void (*release)(struct fscache_operation *op);
- ...
- };
-
- FS-Cache has a pool of threads that it uses to give CPU time to the
- various asynchronous operations that need to be done as part of driving
- the cache. These are represented by the above structure. The processor
- method is called to give the op CPU time, and the release method to get
- rid of it when its usage count reaches 0.
-
- An operation can be made exclusive upon an object by setting the
- appropriate flag before enqueuing it with fscache_enqueue_operation(). If
- an operation needs more processing time, it should be enqueued again.
-
-
- (*) FS-Cache retrieval operation record:
-
- struct fscache_retrieval {
- struct fscache_operation op;
- struct address_space *mapping;
- struct list_head *to_do;
- ...
- };
-
- A structure of this type is allocated by FS-Cache to record retrieval and
- allocation requests made by the netfs. This struct is then passed to the
- backend to do the operation. The backend may get extra refs to it by
- calling fscache_get_retrieval() and refs may be discarded by calling
- fscache_put_retrieval().
-
- A retrieval operation can be used by the backend to do retrieval work. To
- do this, the retrieval->op.processor method pointer should be set
- appropriately by the backend and fscache_enqueue_retrieval() called to
- submit it to the thread pool. CacheFiles, for example, uses this to queue
- page examination when it detects PG_lock being cleared.
-
- The to_do field is an empty list available for the cache backend to use as
- it sees fit.
-
-
- (*) FS-Cache storage operation record:
-
- struct fscache_storage {
- struct fscache_operation op;
- pgoff_t store_limit;
- ...
- };
-
- A structure of this type is allocated by FS-Cache to record outstanding
- writes to be made. FS-Cache itself enqueues this operation and invokes
- the write_page() method on the object at appropriate times to effect
- storage.
+ is backing. The fscache_object struct should be initialised by calling
+ fscache_object_init(object).
================
@@ -238,11 +162,6 @@ performed on the denizens of the cache. These are held in a structure of type:
cookie in a particular cache. fscache_object_init() should be called on
the object to initialise it prior to returning.
- This function may also be used to parse the index key to be used for
- multiple lookup calls to turn it into a more convenient form. FS-Cache
- will call the lookup_complete() method to allow the cache to release the
- form once lookup is complete or aborted.
-
(*) Look up and create object [mandatory]:
@@ -252,16 +171,6 @@ performed on the denizens of the cache. These are held in a structure of type:
allocated and attached to the cookie. This should instantiate that object
in the cache if it can.
- The method should call fscache_object_lookup_negative() as soon as
- possible if it determines the object doesn't exist in the cache. If the
- object is found to exist and the netfs indicates that it is valid then
- fscache_obtained_object() should be called once the object is in a
- position to have data stored in it. Similarly, fscache_obtained_object()
- should also be called once a non-present object has been created.
-
- If a lookup error occurs, fscache_object_lookup_error() should be called
- to abort the lookup of that object.
-
(*) Release lookup data [mandatory]:
@@ -276,17 +185,7 @@ performed on the denizens of the cache. These are held in a structure of type:
struct fscache_object *(*grab_object)(struct fscache_object *object)
This method is called to increment the reference count on an object. It
- may fail (for instance if the cache is being withdrawn) by returning NULL.
- It should return the object pointer if successful.
-
-
- (*) Lock/Unlock object [mandatory]:
-
- void (*lock_object)(struct fscache_object *object)
- void (*unlock_object)(struct fscache_object *object)
-
- These methods are used to exclusively lock an object. It must be possible
- to schedule with the lock held, so a spinlock isn't sufficient.
+ should return the object pointer.
(*) Pin/Unpin object [optional]:
@@ -299,34 +198,19 @@ performed on the denizens of the cache. These are held in a structure of type:
enough space in the cache to permit this.
- (*) Check coherency state of an object [mandatory]:
-
- int (*check_consistency)(struct fscache_object *object)
-
- This method is called to have the cache check the saved auxiliary data of
- the object against the netfs's idea of the state. 0 should be returned
- if they're consistent and -ESTALE otherwise. -ENOMEM and -ERESTARTSYS
- may also be returned.
-
(*) Update object [mandatory]:
int (*update_object)(struct fscache_object *object)
This is called to update the index entry for the specified object. The
- new information should be in object->cookie->netfs_data. This can be
- obtained by calling object->cookie->def->get_aux()/get_attr().
+ new information should be in object->cookie->aux and ->object_size.
(*) Invalidate data object [mandatory]:
int (*invalidate_object)(struct fscache_operation *op)
- This is called to invalidate a data object (as pointed to by op->object).
- All the data stored for this object should be discarded and an
- attr_changed operation should be performed. The caller will follow up
- with an object update operation.
-
- fscache_op_complete() must be called on op before returning.
+ This is called to invalidate a data object.
(*) Discard object [mandatory]:
@@ -334,8 +218,7 @@ performed on the denizens of the cache. These are held in a structure of type:
void (*drop_object)(struct fscache_object *object)
This method is called to indicate that an object has been unbound from its
- cookie, and that the cache should release the object's resources and
- retire it if it's in state FSCACHE_OBJECT_RECYCLING.
+ cookie.
This method should not attempt to release any references held by the
caller. The caller will invoke the put_object() method as appropriate.
@@ -351,41 +234,12 @@ performed on the denizens of the cache. These are held in a structure of type:
(*) Synchronise a cache [mandatory]:
- void (*sync)(struct fscache_cache *cache)
+ void (*sync_cache)(struct fscache_cache *cache)
This is called to ask the backend to synchronise a cache with its backing
device.
- (*) Dissociate a cache [mandatory]:
-
- void (*dissociate_pages)(struct fscache_cache *cache)
-
- This is called to ask a cache to perform any page dissociations as part of
- cache withdrawal.
-
-
- (*) Notification that the attributes on a netfs file changed [mandatory]:
-
- int (*attr_changed)(struct fscache_object *object);
-
- This is called to indicate to the cache that certain attributes on a netfs
- file have changed (for example the maximum size a file may reach). The
- cache can read these from the netfs by calling the cookie's get_attr()
- method.
-
- The cache may use the file size information to reserve space on the cache.
- It should also call fscache_set_store_limit() to indicate to FS-Cache the
- highest byte it's willing to store for an object.
-
- This method may return -ve if an error occurred or the cache object cannot
- be expanded. In such a case, the object will be withdrawn from service.
-
- This operation is run asynchronously from FS-Cache's thread pool, and
- storage and retrieval operations from the netfs are excluded during the
- execution of this operation.
-
-
(*) Reserve cache space for an object's data [optional]:
int (*reserve_space)(struct fscache_object *object, loff_t size);
@@ -404,125 +258,6 @@ performed on the denizens of the cache. These are held in a structure of type:
size if larger than that already.
- (*) Request page be read from cache [mandatory]:
-
- int (*read_or_alloc_page)(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-
- This is called to attempt to read a netfs page from the cache, or to
- reserve a backing block if not. FS-Cache will have done as much checking
- as it can before calling, but most of the work belongs to the backend.
-
- If there's no page in the cache, then -ENODATA should be returned if the
- backend managed to reserve a backing block; -ENOBUFS or -ENOMEM if it
- didn't.
-
- If there is suitable data in the cache, then a read operation should be
- queued and 0 returned. When the read finishes, fscache_end_io() should be
- called.
-
- The fscache_mark_pages_cached() should be called for the page if any cache
- metadata is retained. This will indicate to the netfs that the page needs
- explicit uncaching. This operation takes a pagevec, thus allowing several
- pages to be marked at once.
-
- The retrieval record pointed to by op should be retained for each page
- queued and released when I/O on the page has been formally ended.
- fscache_get/put_retrieval() are available for this purpose.
-
- The retrieval record may be used to get CPU time via the FS-Cache thread
- pool. If this is desired, the op->op.processor should be set to point to
- the appropriate processing routine, and fscache_enqueue_retrieval() should
- be called at an appropriate point to request CPU time. For instance, the
- retrieval routine could be enqueued upon the completion of a disk read.
- The to_do field in the retrieval record is provided to aid in this.
-
- If an I/O error occurs, fscache_io_error() should be called and -ENOBUFS
- returned if possible or fscache_end_io() called with a suitable error
- code.
-
- fscache_put_retrieval() should be called after a page or pages are dealt
- with. This will complete the operation when all pages are dealt with.
-
-
- (*) Request pages be read from cache [mandatory]:
-
- int (*read_or_alloc_pages)(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-
- This is like the read_or_alloc_page() method, except it is handed a list
- of pages instead of one page. Any pages on which a read operation is
- started must be added to the page cache for the specified mapping and also
- to the LRU. Such pages must also be removed from the pages list and
- *nr_pages decremented per page.
-
- If there was an error such as -ENOMEM, then that should be returned; else
- if one or more pages couldn't be read or allocated, then -ENOBUFS should
- be returned; else if one or more pages couldn't be read, then -ENODATA
- should be returned. If all the pages are dispatched then 0 should be
- returned.
-
-
- (*) Request page be allocated in the cache [mandatory]:
-
- int (*allocate_page)(struct fscache_retrieval *op,
- struct page *page,
- gfp_t gfp)
-
- This is like the read_or_alloc_page() method, except that it shouldn't
- read from the cache, even if there's data there that could be retrieved.
- It should, however, set up any internal metadata required such that
- the write_page() method can write to the cache.
-
- If there's no backing block available, then -ENOBUFS should be returned
- (or -ENOMEM if there were other problems). If a block is successfully
- allocated, then the netfs page should be marked and 0 returned.
-
-
- (*) Request pages be allocated in the cache [mandatory]:
-
- int (*allocate_pages)(struct fscache_retrieval *op,
- struct list_head *pages,
- unsigned *nr_pages,
- gfp_t gfp)
-
- This is an multiple page version of the allocate_page() method. pages and
- nr_pages should be treated as for the read_or_alloc_pages() method.
-
-
- (*) Request page be written to cache [mandatory]:
-
- int (*write_page)(struct fscache_storage *op,
- struct page *page);
-
- This is called to write from a page on which there was a previously
- successful read_or_alloc_page() call or similar. FS-Cache filters out
- pages that don't have mappings.
-
- This method is called asynchronously from the FS-Cache thread pool. It is
- not required to actually store anything, provided -ENODATA is then
- returned to the next read of this page.
-
- If an error occurred, then a negative error code should be returned,
- otherwise zero should be returned. FS-Cache will take appropriate action
- in response to an error, such as withdrawing this object.
-
- If this method returns success then FS-Cache will inform the netfs
- appropriately.
-
-
- (*) Discard retained per-page metadata [mandatory]:
-
- void (*uncache_page)(struct fscache_object *object, struct page *page)
-
- This is called when a netfs page is being evicted from the pagecache. The
- cache backend should tear down any internal representation or tracking it
- maintains for this page.
-
-
==================
FS-CACHE UTILITIES
==================
@@ -541,83 +276,6 @@ FS-Cache provides some utilities that a cache backend may make use of:
This does not actually withdraw the cache. That must be done separately.
- (*) Invoke the retrieval I/O completion function:
-
- void fscache_end_io(struct fscache_retrieval *op, struct page *page,
- int error);
-
- This is called to note the end of an attempt to retrieve a page. The
- error value should be 0 if successful and an error otherwise.
-
-
- (*) Record that one or more pages being retrieved or allocated have been dealt
- with:
-
- void fscache_retrieval_complete(struct fscache_retrieval *op,
- int n_pages);
-
- This is called to record the fact that one or more pages have been dealt
- with and are no longer the concern of this operation. When the number of
- pages remaining in the operation reaches 0, the operation will be
- completed.
-
-
- (*) Record operation completion:
-
- void fscache_op_complete(struct fscache_operation *op);
-
- This is called to record the completion of an operation. This deducts
- this operation from the parent object's run state, potentially permitting
- one or more pending operations to start running.
-
-
- (*) Set highest store limit:
-
- void fscache_set_store_limit(struct fscache_object *object,
- loff_t i_size);
-
- This sets the limit FS-Cache imposes on the highest byte it's willing to
- try and store for a netfs. Any page over this limit is automatically
- rejected by fscache_read_alloc_page() and co with -ENOBUFS.
-
-
- (*) Mark pages as being cached:
-
- void fscache_mark_pages_cached(struct fscache_retrieval *op,
- struct pagevec *pagevec);
-
- This marks a set of pages as being cached. After this has been called,
- the netfs must call fscache_uncache_page() to unmark the pages.
-
-
- (*) Perform coherency check on an object:
-
- enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
- const void *data,
- uint16_t datalen);
-
- This asks the netfs to perform a coherency check on an object that has
- just been looked up. The cookie attached to the object will determine the
- netfs to use. data and datalen should specify where the auxiliary data
- retrieved from the cache can be found.
-
- One of three values will be returned:
-
- (*) FSCACHE_CHECKAUX_OKAY
-
- The coherency data indicates the object is valid as is.
-
- (*) FSCACHE_CHECKAUX_NEEDS_UPDATE
-
- The coherency data needs updating, but otherwise the object is
- valid.
-
- (*) FSCACHE_CHECKAUX_OBSOLETE
-
- The coherency data indicates that the object is obsolete and should
- be discarded.
-
-
(*) Initialise a freshly allocated object:
void fscache_object_init(struct fscache_object *object);
@@ -635,47 +293,6 @@ FS-Cache provides some utilities that a cache backend may make use of:
all the objects.
- (*) Indicate negative lookup on an object:
-
- void fscache_object_lookup_negative(struct fscache_object *object);
-
- This is called to indicate to FS-Cache that a lookup process for an object
- found a negative result.
-
- This changes the state of an object to permit reads pending on lookup
- completion to go off and start fetching data from the netfs server as it's
- known at this point that there can't be any data in the cache.
-
- This may be called multiple times on an object. Only the first call is
- significant - all subsequent calls are ignored.
-
-
- (*) Indicate an object has been obtained:
-
- void fscache_obtained_object(struct fscache_object *object);
-
- This is called to indicate to FS-Cache that a lookup process for an object
- produced a positive result, or that an object was created. This should
- only be called once for any particular object.
-
- This changes the state of an object to indicate:
-
- (1) if no call to fscache_object_lookup_negative() has been made on
- this object, that there may be data available, and that reads can
- now go and look for it; and
-
- (2) that writes may now proceed against this object.
-
-
- (*) Indicate that object lookup failed:
-
- void fscache_object_lookup_error(struct fscache_object *object);
-
- This marks an object as having encountered a fatal error (usually EIO)
- and causes it to move into a state whereby it will be withdrawn as soon
- as possible.
-
-
(*) Indicate that a stale object was found and discarded:
void fscache_object_retrying_stale(struct fscache_object *object);
@@ -697,30 +314,3 @@ FS-Cache provides some utilities that a cache backend may make use of:
FSCACHE_OBJECT_NO_SPACE - there was insufficient cache space
FSCACHE_OBJECT_WAS_RETIRED - the object was retired when relinquished.
FSCACHE_OBJECT_WAS_CULLED - the object was culled to make space.
-
-
- (*) Get and release references on a retrieval record:
-
- void fscache_get_retrieval(struct fscache_retrieval *op);
- void fscache_put_retrieval(struct fscache_retrieval *op);
-
- These two functions are used to retain a retrieval record while doing
- asynchronous data retrieval and block allocation.
-
-
- (*) Enqueue a retrieval record for processing.
-
- void fscache_enqueue_retrieval(struct fscache_retrieval *op);
-
- This enqueues a retrieval record for processing by the FS-Cache thread
- pool. One of the threads in the pool will invoke the retrieval record's
- op->op.processor callback function. This function may be called from
- within the callback function.
-
-
- (*) List of object state names:
-
- const char *fscache_object_states[];
-
- For debugging purposes, this may be used to turn the state that an object
- is in into a text string for display purposes.
diff --git a/Documentation/filesystems/caching/netfs-api.txt b/Documentation/filesystems/caching/netfs-api.txt
index ba968e8f5704..a39de0a4f336 100644
--- a/Documentation/filesystems/caching/netfs-api.txt
+++ b/Documentation/filesystems/caching/netfs-api.txt
@@ -21,24 +21,19 @@ This API is declared in <linux/fscache.h>.
This document contains the following sections:
- (1) Network filesystem definition
- (2) Index definition
- (3) Object definition
- (4) Network filesystem (un)registration
- (5) Cache tag lookup
- (6) Index registration
- (7) Data file registration
- (8) Miscellaneous object registration
- (9) Setting the data file size
- (10) Page alloc/read/write
- (11) Page uncaching
- (12) Index and data file consistency
- (13) Cookie enablement
- (14) Miscellaneous cookie operations
- (15) Cookie unregistration
- (16) Index invalidation
- (17) Data file invalidation
- (18) FS-Cache specific page flags.
+ - Network filesystem definition
+ - Index definition
+ - Network filesystem (un)registration
+ - Cache tag lookup
+ - Index registration
+ - Data file registration
+ - Miscellaneous object registration
+ - Index and data file consistency
+ - Miscellaneous cookie operations
+ - Cookie unregistration
+ - Index invalidation
+ - Data file invalidation
+ - FS-Cache specific page flag
=============================
@@ -114,128 +109,6 @@ There are some limits on indices:
function is recursive. Too many layers will run the kernel out of stack.
-=================
-OBJECT DEFINITION
-=================
-
-To define an object, a structure of the following type should be filled out:
-
- struct fscache_cookie_def
- {
- uint8_t name[16];
- uint8_t type;
-
- struct fscache_cache_tag *(*select_cache)(
- const void *parent_netfs_data,
- const void *cookie_netfs_data);
-
- enum fscache_checkaux (*check_aux)(void *cookie_netfs_data,
- const void *data,
- uint16_t datalen,
- loff_t object_size);
-
- void (*get_context)(void *cookie_netfs_data, void *context);
-
- void (*put_context)(void *cookie_netfs_data, void *context);
-
- void (*mark_pages_cached)(void *cookie_netfs_data,
- struct address_space *mapping,
- struct pagevec *cached_pvec);
- };
-
-This has the following fields:
-
- (1) The type of the object [mandatory].
-
- This is one of the following values:
-
- (*) FSCACHE_COOKIE_TYPE_INDEX
-
- This defines an index, which is a special FS-Cache type.
-
- (*) FSCACHE_COOKIE_TYPE_DATAFILE
-
- This defines an ordinary data file.
-
- (*) Any other value between 2 and 255
-
- This defines an extraordinary object such as an XATTR.
-
- (2) The name of the object type (NUL terminated unless all 16 chars are used)
- [optional].
-
- (3) A function to select the cache in which to store an index [optional].
-
- This function is invoked when an index needs to be instantiated in a cache
- during the instantiation of a non-index object. Only the immediate index
- parent for the non-index object will be queried. Any indices above that
- in the hierarchy may be stored in multiple caches. This function does not
- need to be supplied for any non-index object or any index that will only
- have index children.
-
- If this function is not supplied or if it returns NULL then the first
- cache in the parent's list will be chosen, or failing that, the first
- cache in the master list.
-
- (4) A function to check the auxiliary data [optional].
-
- This function will be called to check that a match found in the cache for
- this object is valid. For instance with AFS it could check the auxiliary
- data against the data version number returned by the server to determine
- whether the index entry in a cache is still valid.
-
- If this function is absent, it will be assumed that matching objects in a
- cache are always valid.
-
- The function is also passed the cache's idea of the object size and may
- use this to manage coherency also.
-
- If present, the function should return one of the following values:
-
- (*) FSCACHE_CHECKAUX_OKAY - the entry is okay as is
- (*) FSCACHE_CHECKAUX_NEEDS_UPDATE - the entry requires update
- (*) FSCACHE_CHECKAUX_OBSOLETE - the entry should be deleted
-
- This function can also be used to extract data from the auxiliary data in
- the cache and copy it into the netfs's structures.
-
- (5) A pair of functions to manage contexts for the completion callback
- [optional].
-
- The cache read/write functions are passed a context which is then passed
- to the I/O completion callback function. To ensure this context remains
- valid until after the I/O completion is called, two functions may be
- provided: one to get an extra reference on the context, and one to drop a
- reference to it.
-
- If the context is not used or is a type of object that won't go out of
- scope, then these functions are not required. These functions are not
- required for indices as indices may not contain data. These functions may
- be called in interrupt context and so may not sleep.
-
- (6) A function to mark a page as retaining cache metadata [optional].
-
- This is called by the cache to indicate that it is retaining in-memory
- information for this page and that the netfs should uncache the page when
- it has finished. This does not indicate whether there's data on the disk
- or not. Note that several pages at once may be presented for marking.
-
- The PG_fscache bit is set on the pages before this function would be
- called, so the function need not be provided if this is sufficient.
-
- This function is not required for indices as they're not permitted data.
-
- (7) A function to unmark all the pages retaining cache metadata [mandatory].
-
- This is called by FS-Cache to indicate that a backing store is being
- unbound from a cookie and that all the marks on the pages should be
- cleared to prevent confusion. Note that the cache will have torn down all
- its tracking information so that the pages don't need to be explicitly
- uncached.
-
- This function is not required for indices as they're not permitted data.
-
-
===================================
NETWORK FILESYSTEM (UN)REGISTRATION
===================================
@@ -319,10 +192,6 @@ The netfs may pass an arbitrary value in netfs_data and this will be presented
to it in the event of any calling back. This may also be used in tracing or
logging of messages.
-The cache tracks the size of the data attached to an object and this set to be
-object_size. For indices, this should be 0. This value will be passed to the
-->check_aux() callback.
-
Note that this function never returns an error - all errors are handled
internally. It may, however, return NULL to indicate no cookie. It is quite
acceptable to pass this token back to this function as the parent to another
@@ -334,10 +203,6 @@ object needs to be created somewhere down the hierarchy. Furthermore, an index
may be created in several different caches independently at different times.
This is all handled transparently, and the netfs doesn't see any of it.
-A cookie will be created in the disabled state if enabled is false. A cookie
-must be enabled to do anything with it. A disabled cookie can be enabled by
-calling fscache_enable_cookie() (see below).
-
For example, with AFS, a cell would be added to the primary index. This index
entry would have a dependent inode containing volume mappings within this cell:
@@ -396,317 +261,10 @@ Miscellaneous objects might be used to store extended attributes or directory
entries for example.
-==========================
-SETTING THE DATA FILE SIZE
-==========================
-
-The fifth step is to set the physical attributes of the file, such as its size.
-This doesn't automatically reserve any space in the cache, but permits the
-cache to adjust its metadata for data tracking appropriately:
-
- int fscache_attr_changed(struct fscache_cookie *cookie);
-
-The cache will return -ENOBUFS if there is no backing cache or if there is no
-space to allocate any extra metadata required in the cache.
-
-Note that attempts to read or write data pages in the cache over this size may
-be rebuffed with -ENOBUFS.
-
-This operation schedules an attribute adjustment to happen asynchronously at
-some point in the future, and as such, it may happen after the function returns
-to the caller. The attribute adjustment excludes read and write operations.
-
-
-=====================
-PAGE ALLOC/READ/WRITE
-=====================
-
-And the sixth step is to store and retrieve pages in the cache. There are
-three functions that are used to do this.
-
-Note:
-
- (1) A page should not be re-read or re-allocated without uncaching it first.
-
- (2) A read or allocated page must be uncached when the netfs page is released
- from the pagecache.
-
- (3) A page should only be written to the cache if previous read or allocated.
-
-This permits the cache to maintain its page tracking in proper order.
-
-
-PAGE READ
----------
-
-Firstly, the netfs should ask FS-Cache to examine the caches and read the
-contents cached for a particular page of a particular file if present, or else
-allocate space to store the contents if not:
-
- typedef
- void (*fscache_rw_complete_t)(struct page *page,
- void *context,
- int error);
-
- int fscache_read_or_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp);
-
-The cookie argument must specify a cookie for an object that isn't an index,
-the page specified will have the data loaded into it (and is also used to
-specify the page number), and the gfp argument is used to control how any
-memory allocations made are satisfied.
-
-If the cookie indicates the inode is not cached:
-
- (1) The function will return -ENOBUFS.
-
-Else if there's a copy of the page resident in the cache:
-
- (1) The mark_pages_cached() cookie operation will be called on that page.
-
- (2) The function will submit a request to read the data from the cache's
- backing device directly into the page specified.
-
- (3) The function will return 0.
-
- (4) When the read is complete, end_io_func() will be invoked with:
-
- (*) The netfs data supplied when the cookie was created.
-
- (*) The page descriptor.
-
- (*) The context argument passed to the above function. This will be
- maintained with the get_context/put_context functions mentioned above.
-
- (*) An argument that's 0 on success or negative for an error code.
-
- If an error occurs, it should be assumed that the page contains no usable
- data. fscache_readpages_cancel() may need to be called.
-
- end_io_func() will be called in process context if the read is results in
- an error, but it might be called in interrupt context if the read is
- successful.
-
-Otherwise, if there's not a copy available in cache, but the cache may be able
-to store the page:
-
- (1) The mark_pages_cached() cookie operation will be called on that page.
-
- (2) A block may be reserved in the cache and attached to the object at the
- appropriate place.
-
- (3) The function will return -ENODATA.
-
-This function may also return -ENOMEM or -EINTR, in which case it won't have
-read any data from the cache.
-
-
-PAGE ALLOCATE
--------------
-
-Alternatively, if there's not expected to be any data in the cache for a page
-because the file has been extended, a block can simply be allocated instead:
-
- int fscache_alloc_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp);
-
-This is similar to the fscache_read_or_alloc_page() function, except that it
-never reads from the cache. It will return 0 if a block has been allocated,
-rather than -ENODATA as the other would. One or the other must be performed
-before writing to the cache.
-
-The mark_pages_cached() cookie operation will be called on the page if
-successful.
-
-
-PAGE WRITE
-----------
-
-Secondly, if the netfs changes the contents of the page (either due to an
-initial download or if a user performs a write), then the page should be
-written back to the cache:
-
- int fscache_write_page(struct fscache_cookie *cookie,
- struct page *page,
- loff_t object_size,
- gfp_t gfp);
-
-The cookie argument must specify a data file cookie, the page specified should
-contain the data to be written (and is also used to specify the page number),
-object_size is the revised size of the object and the gfp argument is used to
-control how any memory allocations made are satisfied.
-
-The page must have first been read or allocated successfully and must not have
-been uncached before writing is performed.
-
-If the cookie indicates the inode is not cached then:
-
- (1) The function will return -ENOBUFS.
-
-Else if space can be allocated in the cache to hold this page:
-
- (1) PG_fscache_write will be set on the page.
-
- (2) The function will submit a request to write the data to cache's backing
- device directly from the page specified.
-
- (3) The function will return 0.
-
- (4) When the write is complete PG_fscache_write is cleared on the page and
- anyone waiting for that bit will be woken up.
-
-Else if there's no space available in the cache, -ENOBUFS will be returned. It
-is also possible for the PG_fscache_write bit to be cleared when no write took
-place if unforeseen circumstances arose (such as a disk error).
-
-Writing takes place asynchronously.
-
-
-MULTIPLE PAGE READ
-------------------
-
-A facility is provided to read several pages at once, as requested by the
-readpages() address space operation:
-
- int fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
- struct address_space *mapping,
- struct list_head *pages,
- int *nr_pages,
- fscache_rw_complete_t end_io_func,
- void *context,
- gfp_t gfp);
-
-This works in a similar way to fscache_read_or_alloc_page(), except:
-
- (1) Any page it can retrieve data for is removed from pages and nr_pages and
- dispatched for reading to the disk. Reads of adjacent pages on disk may
- be merged for greater efficiency.
-
- (2) The mark_pages_cached() cookie operation will be called on several pages
- at once if they're being read or allocated.
-
- (3) If there was an general error, then that error will be returned.
-
- Else if some pages couldn't be allocated or read, then -ENOBUFS will be
- returned.
-
- Else if some pages couldn't be read but were allocated, then -ENODATA will
- be returned.
-
- Otherwise, if all pages had reads dispatched, then 0 will be returned, the
- list will be empty and *nr_pages will be 0.
-
- (4) end_io_func will be called once for each page being read as the reads
- complete. It will be called in process context if error != 0, but it may
- be called in interrupt context if there is no error.
-
-Note that a return of -ENODATA, -ENOBUFS or any other error does not preclude
-some of the pages being read and some being allocated. Those pages will have
-been marked appropriately and will need uncaching.
-
-
-CANCELLATION OF UNREAD PAGES
-----------------------------
-
-If one or more pages are passed to fscache_read_or_alloc_pages() but not then
-read from the cache and also not read from the underlying filesystem then
-those pages will need to have any marks and reservations removed. This can be
-done by calling:
-
- void fscache_readpages_cancel(struct fscache_cookie *cookie,
- struct list_head *pages);
-
-prior to returning to the caller. The cookie argument should be as passed to
-fscache_read_or_alloc_pages(). Every page in the pages list will be examined
-and any that have PG_fscache set will be uncached.
-
-
-==============
-PAGE UNCACHING
-==============
-
-To uncache a page, this function should be called:
-
- void fscache_uncache_page(struct fscache_cookie *cookie,
- struct page *page);
-
-This function permits the cache to release any in-memory representation it
-might be holding for this netfs page. This function must be called once for
-each page on which the read or write page functions above have been called to
-make sure the cache's in-memory tracking information gets torn down.
-
-Note that pages can't be explicitly deleted from the a data file. The whole
-data file must be retired (see the relinquish cookie function below).
-
-Furthermore, note that this does not cancel the asynchronous read or write
-operation started by the read/alloc and write functions, so the page
-invalidation functions must use:
-
- bool fscache_check_page_write(struct fscache_cookie *cookie,
- struct page *page);
-
-to see if a page is being written to the cache, and:
-
- void fscache_wait_on_page_write(struct fscache_cookie *cookie,
- struct page *page);
-
-to wait for it to finish if it is.
-
-
-When releasepage() is being implemented, a special FS-Cache function exists to
-manage the heuristics of coping with vmscan trying to eject pages, which may
-conflict with the cache trying to write pages to the cache (which may itself
-need to allocate memory):
-
- bool fscache_maybe_release_page(struct fscache_cookie *cookie,
- struct page *page,
- gfp_t gfp);
-
-This takes the netfs cookie, and the page and gfp arguments as supplied to
-releasepage(). It will return false if the page cannot be released yet for
-some reason and if it returns true, the page has been uncached and can now be
-released.
-
-To make a page available for release, this function may wait for an outstanding
-storage request to complete, or it may attempt to cancel the storage request -
-in which case the page will not be stored in the cache this time.
-
-
-BULK INODE PAGE UNCACHE
------------------------
-
-A convenience routine is provided to perform an uncache on all the pages
-attached to an inode. This assumes that the pages on the inode correspond on a
-1:1 basis with the pages in the cache.
-
- void fscache_uncache_all_inode_pages(struct fscache_cookie *cookie,
- struct inode *inode);
-
-This takes the netfs cookie that the pages were cached with and the inode that
-the pages are attached to. This function will wait for pages to finish being
-written to the cache and for the cache to finish with the page generally. No
-error is returned.
-
-
===============================
INDEX AND DATA FILE CONSISTENCY
===============================
-To find out whether auxiliary data for an object is up to data within the
-cache, the following function can be called:
-
- int fscache_check_consistency(struct fscache_cookie *cookie,
- const void *aux_data);
-
-This will call back to the netfs to check whether the auxiliary data associated
-with a cookie is correct; if aux_data is non-NULL, it will update the auxiliary
-data buffer first. It returns 0 if it is and -ESTALE if it isn't; it may also
-return -ENOMEM and -ERESTARTSYS.
-
To request an update of the index data for an index or other object, the
following function should be called:
@@ -721,56 +279,6 @@ Note that partial updates may happen automatically at other times, such as when
data blocks are added to a data file object.
-=================
-COOKIE ENABLEMENT
-=================
-
-Cookies exist in one of two states: enabled and disabled. If a cookie is
-disabled, it ignores all attempts to acquire child cookies; check, update or
-invalidate its state; allocate, read or write backing pages - though it is
-still possible to uncache pages and relinquish the cookie.
-
-The initial enablement state is set by fscache_acquire_cookie(), but the cookie
-can be enabled or disabled later. To disable a cookie, call:
-
- void fscache_disable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- bool invalidate);
-
-If the cookie is not already disabled, this locks the cookie against other
-enable and disable ops, marks the cookie as being disabled, discards or
-invalidates any backing objects and waits for cessation of activity on any
-associated object before unlocking the cookie.
-
-All possible failures are handled internally. The caller should consider
-calling fscache_uncache_all_inode_pages() afterwards to make sure all page
-markings are cleared up.
-
-Cookies can be enabled or reenabled with:
-
- void fscache_enable_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
- loff_t object_size,
- bool (*can_enable)(void *data),
- void *data)
-
-If the cookie is not already enabled, this locks the cookie against other
-enable and disable ops, invokes can_enable() and, if the cookie is not an index
-cookie, will begin the procedure of acquiring backing objects.
-
-The optional can_enable() function is passed the data argument and returns a
-ruling as to whether or not enablement should actually be permitted to begin.
-
-All possible failures are handled internally. The cookie will only be marked
-as enabled if provisional backing objects are allocated.
-
-The object's data size is updated from object_size and is passed to the
-->check_aux() function.
-
-In both cases, the cookie's auxiliary data buffer is updated from aux_data if
-that is non-NULL inside the enablement lock before proceeding.
-
-
===============================
MISCELLANEOUS COOKIE OPERATIONS
===============================
@@ -790,24 +298,6 @@ There are a number of operations that can be used to control cookies:
-ENOSPC if there isn't enough space to honour the operation, -ENOMEM or
-EIO if there's any other problem.
- (*) Data space reservation:
-
- int fscache_reserve_space(struct fscache_cookie *cookie, loff_t size);
-
- This permits a netfs to request cache space be reserved to store up to the
- given amount of a file. It is permitted to ask for more than the current
- size of the file to allow for future file expansion.
-
- If size is given as zero then the reservation will be cancelled.
-
- The function will return 0 if successful, -ENOBUFS in the cookie isn't
- backed by a cache, -EOPNOTSUPP if the cache doesn't support reservations,
- -ENOSPC if there isn't enough space to honour the operation, -ENOMEM or
- -EIO if there's any other problem.
-
- Note that this doesn't pin an object in a cache; it can still be culled to
- make space if it's not in use.
-
=====================
COOKIE UNREGISTRATION
@@ -857,7 +347,6 @@ can be called:
void fscache_invalidate(struct fscache_cookie *cookie);
-This can be called with spinlocks held as it defers the work to a thread pool.
All extant storage, retrieval and attribute change ops at this point are
cancelled and discarded. Some future operations will be rejected until the
cache has had a chance to insert a barrier in the operations queue. After
@@ -866,12 +355,6 @@ that, operations will be queued again behind the invalidation operation.
The invalidation operation will perform an attribute change operation and an
auxiliary data update operation as it is very likely these will have changed.
-Using the following function, the netfs can wait for the invalidation operation
-to have reached a point at which it can start submitting ordinary operations
-once again:
-
- void fscache_wait_on_invalidate(struct fscache_cookie *cookie);
-
===========================
FS-CACHE SPECIFIC PAGE FLAG
@@ -880,20 +363,11 @@ FS-CACHE SPECIFIC PAGE FLAG
FS-Cache makes use of a page flag, PG_private_2, for its own purpose. This is
given the alternative name PG_fscache.
-PG_fscache is used to indicate that the page is known by the cache, and that
-the cache must be informed if the page is going to go away. It's an indication
-to the netfs that the cache has an interest in this page, where an interest may
-be a pointer to it, resources allocated or reserved for it, or I/O in progress
-upon it.
-
The netfs can use this information in methods such as releasepage() to
determine whether it needs to uncache a page or update it.
Furthermore, if this bit is set, releasepage() and invalidatepage() operations
-will be called on a page to get rid of it, even if PG_private is not set. This
-allows caching to attempted on a page before read_cache_pages() to be called
-after fscache_read_or_alloc_pages() as the former will try and release pages it
-was given under certain circumstances.
+will be called on a page to get rid of it, even if PG_private is not set.
This bit does not overlap with such as PG_private. This means that FS-Cache
can be used with a filesystem that uses the block buffering code.
diff --git a/Documentation/filesystems/caching/object.txt b/Documentation/filesystems/caching/object.txt
index 100ff41127e4..65848e3d0a3f 100644
--- a/Documentation/filesystems/caching/object.txt
+++ b/Documentation/filesystems/caching/object.txt
@@ -8,15 +8,6 @@ Contents:
(*) Representation
- (*) Object management state machine.
-
- - Provision of cpu time.
- - Locking simplification.
-
- (*) The set of states.
-
- (*) The set of events.
-
==============
REPRESENTATION
@@ -87,234 +78,3 @@ may also be entirely unrepresented.
As far as the netfs API goes, the netfs is only actually permitted to see
pointers to the cookies. The cookies themselves and any objects attached to
those cookies are hidden from it.
-
-
-===============================
-OBJECT MANAGEMENT STATE MACHINE
-===============================
-
-Within FS-Cache, each active object is managed by its own individual state
-machine. The state for an object is kept in the fscache_object struct, in
-object->state. A cookie may point to a set of objects that are in different
-states.
-
-Each state has an action associated with it that is invoked when the machine
-wakes up in that state. There are four logical sets of states:
-
- (1) Preparation: states that wait for the parent objects to become ready. The
- representations are hierarchical, and it is expected that an object must
- be created or accessed with respect to its parent object.
-
- (2) Initialisation: states that perform lookups in the cache and validate
- what's found and that create on disk any missing metadata.
-
- (3) Normal running: states that allow netfs operations on objects to proceed
- and that update the state of objects.
-
- (4) Termination: states that detach objects from their netfs cookies, that
- delete objects from disk, that handle disk and system errors and that free
- up in-memory resources.
-
-
-In most cases, transitioning between states is in response to signalled events.
-When a state has finished processing, it will usually set the mask of events in
-which it is interested (object->event_mask) and relinquish the worker thread.
-Then when an event is raised (by calling fscache_raise_event()), if the event
-is not masked, the object will be queued for processing (by calling
-fscache_enqueue_object()).
-
-
-PROVISION OF CPU TIME
----------------------
-
-The work to be done by the various states was given CPU time by the threads of
-the slow work facility. This was used in preference to the workqueue facility
-because:
-
- (1) Threads may be completely occupied for very long periods of time by a
- particular work item. These state actions may be doing sequences of
- synchronous, journalled disk accesses (lookup, mkdir, create, setxattr,
- getxattr, truncate, unlink, rmdir, rename).
-
- (2) Threads may do little actual work, but may rather spend a lot of time
- sleeping on I/O. This means that single-threaded and 1-per-CPU-threaded
- workqueues don't necessarily have the right numbers of threads.
-
-
-LOCKING SIMPLIFICATION
-----------------------
-
-Because only one worker thread may be operating on any particular object's
-state machine at once, this simplifies the locking, particularly with respect
-to disconnecting the netfs's representation of a cache object (fscache_cookie)
-from the cache backend's representation (fscache_object) - which may be
-requested from either end.
-
-
-=================
-THE SET OF STATES
-=================
-
-The object state machine has a set of states that it can be in. There are
-preparation states in which the object sets itself up and waits for its parent
-object to transit to a state that allows access to its children:
-
- (1) State FSCACHE_OBJECT_INIT.
-
- Initialise the object and wait for the parent object to become active. In
- the cache, it is expected that it will not be possible to look an object
- up from the parent object, until that parent object itself has been looked
- up.
-
-There are initialisation states in which the object sets itself up and accesses
-disk for the object metadata:
-
- (2) State FSCACHE_OBJECT_LOOKING_UP.
-
- Look up the object on disk, using the parent as a starting point.
- FS-Cache expects the cache backend to probe the cache to see whether this
- object is represented there, and if it is, to see if it's valid (coherency
- management).
-
- The cache should call fscache_object_lookup_negative() to indicate lookup
- failure for whatever reason, and should call fscache_obtained_object() to
- indicate success.
-
- At the completion of lookup, FS-Cache will let the netfs go ahead with
- read operations, no matter whether the file is yet cached. If not yet
- cached, read operations will be immediately rejected with ENODATA until
- the first known page is uncached - as to that point there can be no data
- to be read out of the cache for that file that isn't currently also held
- in the pagecache.
-
- (3) State FSCACHE_OBJECT_CREATING.
-
- Create an object on disk, using the parent as a starting point. This
- happens if the lookup failed to find the object, or if the object's
- coherency data indicated what's on disk is out of date. In this state,
- FS-Cache expects the cache to create
-
- The cache should call fscache_obtained_object() if creation completes
- successfully, fscache_object_lookup_negative() otherwise.
-
- At the completion of creation, FS-Cache will start processing write
- operations the netfs has queued for an object. If creation failed, the
- write ops will be transparently discarded, and nothing recorded in the
- cache.
-
-There are some normal running states in which the object spends its time
-servicing netfs requests:
-
- (4) State FSCACHE_OBJECT_AVAILABLE.
-
- A transient state in which pending operations are started, child objects
- are permitted to advance from FSCACHE_OBJECT_INIT state, and temporary
- lookup data is freed.
-
- (5) State FSCACHE_OBJECT_ACTIVE.
-
- The normal running state. In this state, requests the netfs makes will be
- passed on to the cache.
-
- (6) State FSCACHE_OBJECT_INVALIDATING.
-
- The object is undergoing invalidation. When the state comes here, it
- discards all pending read, write and attribute change operations as it is
- going to clear out the cache entirely and reinitialise it. It will then
- continue to the FSCACHE_OBJECT_UPDATING state.
-
- (7) State FSCACHE_OBJECT_UPDATING.
-
- The state machine comes here to update the object in the cache from the
- netfs's records. This involves updating the auxiliary data that is used
- to maintain coherency.
-
-And there are terminal states in which an object cleans itself up, deallocates
-memory and potentially deletes stuff from disk:
-
- (8) State FSCACHE_OBJECT_LC_DYING.
-
- The object comes here if it is dying because of a lookup or creation
- error. This would be due to a disk error or system error of some sort.
- Temporary data is cleaned up, and the parent is released.
-
- (9) State FSCACHE_OBJECT_DYING.
-
- The object comes here if it is dying due to an error, because its parent
- cookie has been relinquished by the netfs or because the cache is being
- withdrawn.
-
- Any child objects waiting on this one are given CPU time so that they too
- can destroy themselves. This object waits for all its children to go away
- before advancing to the next state.
-
-(10) State FSCACHE_OBJECT_ABORT_INIT.
-
- The object comes to this state if it was waiting on its parent in
- FSCACHE_OBJECT_INIT, but its parent died. The object will destroy itself
- so that the parent may proceed from the FSCACHE_OBJECT_DYING state.
-
-(11) State FSCACHE_OBJECT_RELEASING.
-(12) State FSCACHE_OBJECT_RECYCLING.
-
- The object comes to one of these two states when dying once it is rid of
- all its children, if it is dying because the netfs relinquished its
- cookie. In the first state, the cached data is expected to persist, and
- in the second it will be deleted.
-
-(13) State FSCACHE_OBJECT_WITHDRAWING.
-
- The object transits to this state if the cache decides it wants to
- withdraw the object from service, perhaps to make space, but also due to
- error or just because the whole cache is being withdrawn.
-
-(14) State FSCACHE_OBJECT_DEAD.
-
- The object transits to this state when the in-memory object record is
- ready to be deleted. The object processor shouldn't ever see an object in
- this state.
-
-
-THE SET OF EVENTS
------------------
-
-There are a number of events that can be raised to an object state machine:
-
- (*) FSCACHE_OBJECT_EV_UPDATE
-
- The netfs requested that an object be updated. The state machine will ask
- the cache backend to update the object, and the cache backend will ask the
- netfs for details of the change through its cookie definition ops.
-
- (*) FSCACHE_OBJECT_EV_CLEARED
-
- This is signalled in two circumstances:
-
- (a) when an object's last child object is dropped and
-
- (b) when the last operation outstanding on an object is completed.
-
- This is used to proceed from the dying state.
-
- (*) FSCACHE_OBJECT_EV_ERROR
-
- This is signalled when an I/O error occurs during the processing of some
- object.
-
- (*) FSCACHE_OBJECT_EV_RELEASE
- (*) FSCACHE_OBJECT_EV_RETIRE
-
- These are signalled when the netfs relinquishes a cookie it was using.
- The event selected depends on whether the netfs asks for the backing
- object to be retired (deleted) or retained.
-
- (*) FSCACHE_OBJECT_EV_WITHDRAW
-
- This is signalled when the cache backend wants to withdraw an object.
- This means that the object will have to be detached from the netfs's
- cookie.
-
-Because the withdrawing releasing/retiring events are all handled by the object
-state machine, it doesn't matter if there's a collision with both ends trying
-to sever the connection at the same time. The state machine can just pick
-which one it wants to honour, and that effects the other.
diff --git a/Documentation/filesystems/caching/operations.txt b/Documentation/filesystems/caching/operations.txt
deleted file mode 100644
index d8976c434718..000000000000
--- a/Documentation/filesystems/caching/operations.txt
+++ /dev/null
@@ -1,213 +0,0 @@
- ================================
- ASYNCHRONOUS OPERATIONS HANDLING
- ================================
-
-By: David Howells <dhowells@redhat.com>
-
-Contents:
-
- (*) Overview.
-
- (*) Operation record initialisation.
-
- (*) Parameters.
-
- (*) Procedure.
-
- (*) Asynchronous callback.
-
-
-========
-OVERVIEW
-========
-
-FS-Cache has an asynchronous operations handling facility that it uses for its
-data storage and retrieval routines. Its operations are represented by
-fscache_operation structs, though these are usually embedded into some other
-structure.
-
-This facility is available to and expected to be be used by the cache backends,
-and FS-Cache will create operations and pass them off to the appropriate cache
-backend for completion.
-
-To make use of this facility, <linux/fscache-cache.h> should be #included.
-
-
-===============================
-OPERATION RECORD INITIALISATION
-===============================
-
-An operation is recorded in an fscache_operation struct:
-
- struct fscache_operation {
- union {
- struct work_struct fast_work;
- struct slow_work slow_work;
- };
- unsigned long flags;
- fscache_operation_processor_t processor;
- ...
- };
-
-Someone wanting to issue an operation should allocate something with this
-struct embedded in it. They should initialise it by calling:
-
- void fscache_operation_init(struct fscache_operation *op,
- fscache_operation_release_t release);
-
-with the operation to be initialised and the release function to use.
-
-The op->flags parameter should be set to indicate the CPU time provision and
-the exclusivity (see the Parameters section).
-
-The op->fast_work, op->slow_work and op->processor flags should be set as
-appropriate for the CPU time provision (see the Parameters section).
-
-FSCACHE_OP_WAITING may be set in op->flags prior to each submission of the
-operation and waited for afterwards.
-
-
-==========
-PARAMETERS
-==========
-
-There are a number of parameters that can be set in the operation record's flag
-parameter. There are three options for the provision of CPU time in these
-operations:
-
- (1) The operation may be done synchronously (FSCACHE_OP_MYTHREAD). A thread
- may decide it wants to handle an operation itself without deferring it to
- another thread.
-
- This is, for example, used in read operations for calling readpages() on
- the backing filesystem in CacheFiles. Although readpages() does an
- asynchronous data fetch, the determination of whether pages exist is done
- synchronously - and the netfs does not proceed until this has been
- determined.
-
- If this option is to be used, FSCACHE_OP_WAITING must be set in op->flags
- before submitting the operation, and the operating thread must wait for it
- to be cleared before proceeding:
-
- wait_on_bit(&op->flags, FSCACHE_OP_WAITING,
- TASK_UNINTERRUPTIBLE);
-
-
- (2) The operation may be fast asynchronous (FSCACHE_OP_FAST), in which case it
- will be given to keventd to process. Such an operation is not permitted
- to sleep on I/O.
-
- This is, for example, used by CacheFiles to copy data from a backing fs
- page to a netfs page after the backing fs has read the page in.
-
- If this option is used, op->fast_work and op->processor must be
- initialised before submitting the operation:
-
- INIT_WORK(&op->fast_work, do_some_work);
-
-
- (3) The operation may be slow asynchronous (FSCACHE_OP_SLOW), in which case it
- will be given to the slow work facility to process. Such an operation is
- permitted to sleep on I/O.
-
- This is, for example, used by FS-Cache to handle background writes of
- pages that have just been fetched from a remote server.
-
- If this option is used, op->slow_work and op->processor must be
- initialised before submitting the operation:
-
- fscache_operation_init_slow(op, processor)
-
-
-Furthermore, operations may be one of two types:
-
- (1) Exclusive (FSCACHE_OP_EXCLUSIVE). Operations of this type may not run in
- conjunction with any other operation on the object being operated upon.
-
- An example of this is the attribute change operation, in which the file
- being written to may need truncation.
-
- (2) Shareable. Operations of this type may be running simultaneously. It's
- up to the operation implementation to prevent interference between other
- operations running at the same time.
-
-
-=========
-PROCEDURE
-=========
-
-Operations are used through the following procedure:
-
- (1) The submitting thread must allocate the operation and initialise it
- itself. Normally this would be part of a more specific structure with the
- generic op embedded within.
-
- (2) The submitting thread must then submit the operation for processing using
- one of the following two functions:
-
- int fscache_submit_op(struct fscache_object *object,
- struct fscache_operation *op);
-
- int fscache_submit_exclusive_op(struct fscache_object *object,
- struct fscache_operation *op);
-
- The first function should be used to submit non-exclusive ops and the
- second to submit exclusive ones. The caller must still set the
- FSCACHE_OP_EXCLUSIVE flag.
-
- If successful, both functions will assign the operation to the specified
- object and return 0. -ENOBUFS will be returned if the object specified is
- permanently unavailable.
-
- The operation manager will defer operations on an object that is still
- undergoing lookup or creation. The operation will also be deferred if an
- operation of conflicting exclusivity is in progress on the object.
-
- If the operation is asynchronous, the manager will retain a reference to
- it, so the caller should put their reference to it by passing it to:
-
- void fscache_put_operation(struct fscache_operation *op);
-
- (3) If the submitting thread wants to do the work itself, and has marked the
- operation with FSCACHE_OP_MYTHREAD, then it should monitor
- FSCACHE_OP_WAITING as described above and check the state of the object if
- necessary (the object might have died while the thread was waiting).
-
- When it has finished doing its processing, it should call
- fscache_op_complete() and fscache_put_operation() on it.
-
- (4) The operation holds an effective lock upon the object, preventing other
- exclusive ops conflicting until it is released. The operation can be
- enqueued for further immediate asynchronous processing by adjusting the
- CPU time provisioning option if necessary, eg:
-
- op->flags &= ~FSCACHE_OP_TYPE;
- op->flags |= ~FSCACHE_OP_FAST;
-
- and calling:
-
- void fscache_enqueue_operation(struct fscache_operation *op)
-
- This can be used to allow other things to have use of the worker thread
- pools.
-
-
-=====================
-ASYNCHRONOUS CALLBACK
-=====================
-
-When used in asynchronous mode, the worker thread pool will invoke the
-processor method with a pointer to the operation. This should then get at the
-container struct by using container_of():
-
- static void fscache_write_op(struct fscache_operation *_op)
- {
- struct fscache_storage *op =
- container_of(_op, struct fscache_storage, op);
- ...
- }
-
-The caller holds a reference on the operation, and will invoke
-fscache_put_operation() when the processor function returns. The processor
-function is at liberty to call fscache_enqueue_operation() or to take extra
-references.
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 60/61] fscache: Document the new netfs API
2020-05-04 17:07 ` David Howells
` (59 preceding siblings ...)
(?)
@ 2020-05-04 17:16 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:16 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Document the netfs parts of the rewritten caching API.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/filesystems/caching/netfs-api.txt | 488 ++++++++++++++++++-----
1 file changed, 372 insertions(+), 116 deletions(-)
diff --git a/Documentation/filesystems/caching/netfs-api.txt b/Documentation/filesystems/caching/netfs-api.txt
index a39de0a4f336..5f8f031b423f 100644
--- a/Documentation/filesystems/caching/netfs-api.txt
+++ b/Documentation/filesystems/caching/netfs-api.txt
@@ -28,11 +28,13 @@ This document contains the following sections:
- Index registration
- Data file registration
- Miscellaneous object registration
- - Index and data file consistency
- Miscellaneous cookie operations
- Cookie unregistration
- - Index invalidation
- - Data file invalidation
+ - Coherency management
+ - Preparing to access data
+ - I/O request shaping
+ - Submitting I/O requests
+ - Read helpers
- FS-Cache specific page flag
@@ -88,9 +90,9 @@ Indices are used for two purposes:
However, since it's unlikely that any two netfs's are going to want to define
their index hierarchies in quite the same way, FS-Cache tries to impose as few
-restraints as possible on how an index is structured and where it is placed in
-the tree. The netfs can even mix indices and data files at the same level, but
-it's not recommended.
+restrictions as possible on how an index is structured and where it is placed
+in the tree. The netfs can even mix indices and data files at the same level,
+but it's not recommended.
Each index entry consists of a key of indeterminate length plus some auxiliary
data, also of indeterminate length.
@@ -151,7 +153,7 @@ This takes a text string as the name and returns a representation of a tag. It
will never return an error. It may return a dummy tag, however, if it runs out
of memory; this will inhibit caching with this tag.
-Any representation so obtained must be released by passing it to this function:
+The tag must be released by passing it to this function:
void fscache_release_cache_tag(struct fscache_cache_tag *tag);
@@ -169,114 +171,93 @@ the path to the file:
struct fscache_cookie *
fscache_acquire_cookie(struct fscache_cookie *parent,
- const struct fscache_object_def *def,
+ enum fscache_cookie_type type,
+ const char *type_name,
+ u8 advice,
+ struct fscache_cache_tag *preferred_cache,
const void *index_key,
size_t index_key_len,
const void *aux_data,
size_t aux_data_len,
- void *netfs_data,
- loff_t object_size,
- bool enable);
+ loff_t object_size);
-This function creates an index entry in the index represented by parent,
-filling in the index entry by calling the operations pointed to by def.
+This function creates an index entry in the index represented by parent. The
+entry is of the given type (FSCACHE_COOKIE_TYPE_*) and should be given a small
+type name for display purposes in /proc (trimmed to 7 chars).
+
+The advice flags inform the cache as to how best to handle the object:
+
+ (*) FSCACHE_ADV_SINGLE_CHUNK - The object is all one piece and will only be
+ written or read as a whole. Without this, it is assumed that the object
+ can be treated as granular.
+
+ (*) FSCACHE_ADV_WRITE_CACHE - Write local changes to the cache.
+
+ (*) FSCACHE_ADV_WRITE_NOCACHE - Don't write local changes to the cache, but
+ rather invalidate the object and cease caching it.
+
+A tag indicating the preferred cache for this object can be given in
+preferred_cache. If not given, this will be derived from the parent cookie.
A unique key that represents the object within the parent must be pointed to by
-index_key and is of length index_key_len.
+index_key and is of length index_key_len. A warning will be logged if a cookie
+with this key already exists and NULL will be returned.
An optional blob of auxiliary data that is to be stored within the cache can be
pointed to with aux_data and should be of length aux_data_len. This would
-typically be used for storing coherency data.
+typically be used for storing coherency data and will be checked against
+whatever is in the cache when the cache object is located.
-The netfs may pass an arbitrary value in netfs_data and this will be presented
-to it in the event of any calling back. This may also be used in tracing or
-logging of messages.
+object_size indicates the expected size of the object. This is used for
+trimming the cache object on release rather than coherency checks.
-Note that this function never returns an error - all errors are handled
-internally. It may, however, return NULL to indicate no cookie. It is quite
-acceptable to pass this token back to this function as the parent to another
-acquisition (or even to the relinquish cookie, read page and write page
-functions - see below).
+That this function never returns an error - all errors are handled internally.
+It may, however, return NULL to indicate no cookie. It is quite acceptable to
+pass this token back to this function as the parent to another acquisition (or
+even to the relinquish cookie, read and write functions - see below).
-Note also that no indices are actually created in a cache until a non-index
-object needs to be created somewhere down the hierarchy. Furthermore, an index
-may be created in several different caches independently at different times.
-This is all handled transparently, and the netfs doesn't see any of it.
+Note also that no cache activity will take place until the cache is 'used' to
+access a data object. This is handled transparently, and the netfs doesn't see
+any of it.
For example, with AFS, a cell would be added to the primary index. This index
entry would have a dependent inode containing volume mappings within this cell:
cell->cache =
fscache_acquire_cookie(afs_cache_netfs.primary_index,
- &afs_cell_cache_index_def,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.cell",
+ 0, NULL,
cell->name, strlen(cell->name),
- NULL, 0,
- cell, 0, true);
+ NULL, 0, 0);
And then a particular volume could be added to that index by ID, creating
another index for vnodes (AFS inode equivalents):
volume->cache =
fscache_acquire_cookie(volume->cell->cache,
- &afs_volume_cache_index_def,
+ FSCACHE_COOKIE_TYPE_INDEX,
+ "AFS.vol",
&volume->vid, sizeof(volume->vid),
- NULL, 0,
- volume, 0, true);
+ NULL, 0, 0);
======================
DATA FILE REGISTRATION
======================
-The fourth step is to request a data file be created in the cache. This is
-identical to index cookie acquisition. The only difference is that the type in
-the object definition should be something other than index type.
+The fourth step is to register a data file with the in the cache. This is more
+or less identical to index cookie acquisition.
vnode->cache =
fscache_acquire_cookie(volume->cache,
- &afs_vnode_cache_object_def,
+ FSCACHE_COOKIE_TYPE_DATAFILE,
+ "AFS.vnode",
+ FSCACHE_ADV_SINGLE_CHUNK,
+ NULL,
&key, sizeof(key),
&aux, sizeof(aux),
- vnode, vnode->status.size, true);
-
-
-=================================
-MISCELLANEOUS OBJECT REGISTRATION
-=================================
-
-An optional step is to request an object of miscellaneous type be created in
-the cache. This is almost identical to index cookie acquisition. The only
-difference is that the type in the object definition should be something other
-than index type. While the parent object could be an index, it's more likely
-it would be some other type of object such as a data file.
-
- xattr->cache =
- fscache_acquire_cookie(vnode->cache,
- &afs_xattr_cache_object_def,
- &xattr->name, strlen(xattr->name),
- NULL, 0,
- xattr, strlen(xattr->val), true);
-
-Miscellaneous objects might be used to store extended attributes or directory
-entries for example.
-
-
-===============================
-INDEX AND DATA FILE CONSISTENCY
-===============================
-
-To request an update of the index data for an index or other object, the
-following function should be called:
-
- void fscache_update_cookie(struct fscache_cookie *cookie,
- const void *aux_data);
-
-This function will update the cookie's auxiliary data buffer from aux_data if
-that is non-NULL and then schedule this to be stored on disk. The update
-method in the parent index definition will be called to transfer the data.
-
-Note that partial updates may happen automatically at other times, such as when
-data blocks are added to a data file object.
+ vnode->status.size);
===============================
@@ -306,65 +287,342 @@ COOKIE UNREGISTRATION
To get rid of a cookie, this function should be called.
void fscache_relinquish_cookie(struct fscache_cookie *cookie,
- const void *aux_data,
bool retire);
-If retire is non-zero, then the object will be marked for recycling, and all
-copies of it will be removed from all active caches in which it is present.
-Not only that but all child objects will also be retired.
+If retire is non-zero, then the object and, if an index, all its children will
+be invalidated, and all copies of will be removed from all active caches in
+which it is present.
-If retire is zero, then the object may be available again when next the
+If retire is false, then the object may be available again when next the
acquisition function is called. Retirement here will overrule the pinning on a
cookie.
-The cookie's auxiliary data will be updated from aux_data if that is non-NULL
-so that the cache can lazily update it on disk.
-
One very important note - relinquish must NOT be called for a cookie unless all
-the cookies for "child" indices, objects and pages have been relinquished
-first.
+the cookies for "child" cookies have been relinquished first and the cookie has
+been un-used.
-==================
-INDEX INVALIDATION
-==================
+====================
+COHERENCY MANAGEMENT
+====================
-There is no direct way to invalidate an index subtree. To do this, the caller
-should relinquish and retire the cookie they have, and then acquire a new one.
+Coherency is managed at a number of points in the lifecycle of a cache object.
+Firstly, as previously mentioned, coherency data is stored in the auxiliary
+data buffer in the cookie. This is initially filled in at cookie acquisition,
+but can be updated arbitrarily by calling:
+ void fscache_update_cookie(struct fscache_cookie *cookie,
+ const void *aux_data,
+ const loff_t *object_size);
-======================
-DATA FILE INVALIDATION
-======================
+This will update the cookie's auxiliary data buffer from aux_data if that is
+non-NULL and the cached size of the object if object_size is non-NULL.
+
+An opportunity to update the coherency information is also provided in the
+parameters passed to fscache_unuse_cookie().
+
+Beyond that, a data object cookie can also be arbitrarily invalidated at any
+time by calling:
+
+ void fscache_invalidate(struct fscache_cookie *cookie,
+ unsigned int flags);
+
+or by passing true to the 'retire' argument of fscache_relinquish_cookie() when
+the cookie is discarded. Index cookies may only be retired and not arbitrarily
+invalidated as they refer to an entire subtree.
+
+Invalidation is typically done when a foreign change is detected on the server
+with which a network filesystem is communicating.
+
+With fscache_invalidate(), in-progress I/O operations to the cache object will
+be cancelled as best they can and upcoming I/O operations will be made to wait
+until the invalidation is completed.
+
+The flags parameter can be used to qualify the level of invalidation:
+
+ (*) FSCACHE_INVAL_LIGHT - Invalidation need not do anything if the object has
+ already been invalidated and a temporary object is in use.
+
+
+========================
+PREPARING TO ACCESS DATA
+========================
+
+Before a data object can be accessed, the netfs must tell the cache that it
+wants to actively use an object. This can be done with:
+
+ void fscache_use_cookie(struct fscache_cookie *cookie,
+ bool will_modify);
+
+This allows the cache to prepare any resources, open files, etc. so that it can
+deal with upcoming requests. At this time, the objects on disk will be looked
+up and/or created as necessary.
+
+The cookie parameter indicates the object to be accessed and the will_modify
+parameter indicates whether this is for the purpose of caching a local write
+to the netfs (note that *reading* in the netfs may result in writes to the
+cache - this is not what this flag is used for).
+
+When the netfs has finished with an object, it should call the counterpart
+function:
+
+ void fscache_unuse_cookie(struct fscache_cookie *cookie,
+ const void *aux_data,
+ const loff_t *object_size);
+
+This drops the activity count on the cookie and provides an opportunity to
+update the cached coherency data and object size.
+
+It is permitted to 'use' a cookie multiple times concurrently - say for every
+open of a netfs file. The number of users is counted and the object state is
+kept in memory until the users drops off to zero. The state may be kept around
+for a while after that in case further operations are performed as it may cache
+the result of slow, synchronous filesystem operations.
+
+
+===================
+I/O REQUEST SHAPING
+===================
+
+When making a read or a write on the cache, the request needs to be 'shaped'.
+This requires making the request align to the granularity of the cache (for
+content-tracking purposes) and the I/O block size of the backing filesystem
+(for direct-I/O purposes).
+
+The way to do this is to fill in an instance of the following structure:
+
+ struct fscache_extent {
+ pgoff_t start;
+ pgoff_t block_end;
+ pgoff_t limit;
+ unsigned int dio_block_size;
+ };
+
+where 'start' and 'block_end' define the pages in a contiguous read that is
+intended to be made. 'limit' indicates the maximum page that can be read and
+should be ULONG_MAX if no limit is set. Note that both 'block_end' and 'limit'
+should be set to the page index after the last page included in the proposed
+set. dio_block_size can be ignored at this point.
+
+For instance, to read the first two pages of a file:
+
+ .start = 0,
+ .block_end = 2,
+ .limit = ULONG_MAX,
+
+Then the following function can be called:
+
+ unsigned int fscache_shape_extent(struct fscache_cookie *cookie,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write);
-Sometimes it will be necessary to invalidate an object that contains data.
-Typically this will be necessary when the server tells the netfs of a foreign
-change - at which point the netfs has to throw away all the state it had for an
-inode and reload from the server.
+where 'i_size' indicates the size of the netfs file (proposed size in the case
+of a write that increases it) and 'for_write' indicates if this is for a local
+write to the netfs.
-To indicate that a cache object should be invalidated, the following function
-can be called:
+For a granular object (ie. not single-chunk) and if for_write is false, this
+will:
- void fscache_invalidate(struct fscache_cookie *cookie);
+ (1) Round start down and round block_end up or down to the nearest granule
+ boundaries, such that the extent does not cross a data-hole transition in
+ the cache (ie. it's all-read or all-write) and doesn't transgress the
+ stated limit.
-All extant storage, retrieval and attribute change ops at this point are
-cancelled and discarded. Some future operations will be rejected until the
-cache has had a chance to insert a barrier in the operations queue. After
-that, operations will be queued again behind the invalidation operation.
+ (2) Round limit down to indicate the maximum size of a single read the cache
+ can entertain.
-The invalidation operation will perform an attribute change operation and an
-auxiliary data update operation as it is very likely these will have changed.
+In the granular case, if for_write is true, then the writes only need to be
+DIO-aligned as it is assumed that the caller has pre-fetched the block from the
+cache.
+
+Note that the cache granule size may vary between cache objects and may vary
+within a cache object.
+
+For a single-chunk object, start is reduced to 0 and block_end and limit are
+set to the EOF, indicating that the cache wants the whole chunk.
+
+In all cases, on return dio_block_size will be set to the cache's I/O block
+size and the function will return 0, FSCACHE_READ_FROM_CACHE or
+FSCACHE_WRITE_TO_CACHE to indicate whether any I/O can/should be done to/from
+the cache.
+
+
+=======================
+SUBMITTING I/O REQUESTS
+=======================
+
+Data I/O on a cache object is done using a reference-counted request descriptor
+and four functions. To begin an I/O operation, the descriptor must be
+allocated by the netfs and then initialised by calling:
+
+ void fscache_init_io_request(struct fscache_io_request *req,
+ struct fscache_cookie *cookie,
+ const struct fscache_io_request_ops *ops);
+
+This sets fields in the request, allocates any resources it needs and takes a
+reference on the cookie. Once the references have run out, the cache's
+resources in the I/O request descriptor are cleaned up with:
+
+ void fscache_free_io_request(struct fscache_io_request *req)
+
+I/O is initiated with one of two functions, one to read:
+
+ int fscache_read(struct fscache_io_request *req,
+ struct iov_iter *iter);
+
+and one to write:
+
+ int fscache_write(struct fscache_io_request *req,
+ struct iov_iter *iter);
+
+Both of them use an I/O iterator, 'iter', to indicate the data/buffer to be
+written/read into. Other parameters are defined by the request descriptor,
+including the position and length of the transfer and an optional function to
+be called on completion of the I/O operation:
+
+ struct fscache_io_request {
+ loff_t pos;
+ loff_t len;
+ void (*io_done)(struct fscache_io_request *);
+
+A number of fields are then filled in upon completion of the I/O operation:
+
+ loff_t transferred;
+ short error;
+ bool data_from_server;
+ bool data_from_cache;
+ ...
+ };
+
+including the amount of data transferred, any error that occurred and the
+source of the data.
+
+The request descriptor must also be supplied with a table of operations:
+
+ struct fscache_io_request_ops {
+ bool (*is_still_valid)(struct fscache_io_request *);
+ void (*get)(struct fscache_io_request *);
+ void (*put)(struct fscache_io_request *);
+ ...
+ };
+
+including methods to find out if a cache operation is still valid after a wait
+to begin an operation and methods to get or put a reference on the operation.
+
+[!] Note that the 'put' method may be called in softirq context and be unable
+ to sleep. A 'work' member is available in the request struct to allow
+ cleanup to be offloaded to a workqueue.
+
+Note that there are other fields and operations in these structures that
+pertain to the read helpers rather than the above I/O operations and are
+discussed there.
+
+The read and write ops will return 0 on immediate success, -EIOCBQUEUED if the
+operation has been queued for asynchronous I/O, -ESTALE if the cache object
+became invalid, -ENODATA if there's no data to read or -ENOBUFS if caching is
+unavailable. Apart from -EIOCBQUEUED, the error will also be placed in the
+request descriptor.
+
+
+============
+READ HELPERS
+============
+
+Read helpers are available that do all the work of shaping I/O, reading from
+the cache, falling back to issue a read to the server and writing the result to
+the cache.
+
+The first helper is for reading granulated I/O that is read asynchronously in
+units of pages:
+
+ int fscache_read_helper(struct fscache_io_request *req,
+ struct fscache_extent *extent,
+ struct page **requested_page,
+ struct list_head *pages,
+ enum fscache_read_type type,
+ unsigned int aop_flags);
+
+This must be called with a partially prepared request. It will fill in the
+pos, len and io_done fields for itself.
+
+This function can be called in one of three modes:
+
+ (1) Multipage read (ie. readpages).
+
+ In this case, 'type' should be set to FSCACHE_READ_PAGE_LIST and 'pages'
+ should be pointed at a list of pages that is in reverse order of index.
+ 'extent' should indicate the proposed contiguous run. requested_page
+ should be NULL. 'aop_flags' is ignored.
+
+ (2) Read to locked page (ie. readpage).
+
+ In this case, 'type' should be set to FSCACHE_READ_LOCKED_PAGE and
+ '*requested_page' should point to the page to be read. 'pages' should be
+ NULL. 'aop_flags' is ignored.
+
+ (3) Prefetch for write (ie. write_begin).
+
+ In this case, 'type' should be set to FSCACHE_READ_FOR_WRITE and
+ '*requested_page' should point to either a pointer to the page to be read
+ or to a NULL pointer that will be filled in. 'pages' should be NULL.
+ 'aop_flags' indicates the AOP_FLAG_* to be used when getting the primary
+ page. On success, '*requested_page' will be filled in with the primary
+ page and the caller will be left needing to unlock it and release the ref
+ on it.
+
+Two additional request operations must also be provided:
+
+ struct fscache_io_request_ops {
+ ...
+ void (*issue_op)(struct fscache_io_request *);
+ void (*done)(struct fscache_io_request *);
+ };
+
+'issue_op' will be called to issue a request to the server and 'done' will be
+called upon completion of the read phase of the request (though an asynchronous
+write to the cache may still be outstanding).
+
+fscache_read_helper() will attempt to expand/shape the request to fit the
+cache, filling in holes and padding both sides with extra pages attached to the
+cache, though it will abandon that if it finds an uptodate page in the way.
+
+It will attempt to read from the cache first, and if this is unavailable or
+unsuccessful, it will use 'issue_op' to talk to the server. In the latter
+case, if successful, it will set the PG_fscache bits on the pages involved and
+begin a write to the cache.
+
+
+The second helper is for single-chunk I/O that is read or written synchronously
+as a complete unit:
+
+ int fscache_read_helper_single(
+ struct fscache_io_request *req,
+ int (*check)(struct fscache_io_request *req));
+
+This also requires a prepared request descriptor, but also requires the caller
+to have done the work in creating and locking those pages. It waits for
+previous writes to the cache to get out of the way and then issues reads
+against the cache and/or the server and writes the data to the cache much in
+the same way as fscache_read_helper(). Reads will, however, be synchronous.
+Upon a successful return, the data is guaranteed to have finished being read,
+though writing to the cache may still be in progress.
===========================
FS-CACHE SPECIFIC PAGE FLAG
===========================
-FS-Cache makes use of a page flag, PG_private_2, for its own purpose. This is
-given the alternative name PG_fscache.
+FS-Cache makes use of a page flag, PG_private_2, to indicate that a page is
+undergoing write to the cache. This is given the alternative name PG_fscache.
+
+The netfs should wait on PG_fscache in:
-The netfs can use this information in methods such as releasepage() to
-determine whether it needs to uncache a page or update it.
+ releasepage()
+ invalidatepage()
+ write_begin() - after calling fscache_read_helper()
+ writepage()
+ writepages()
+ page_mkwrite()
Furthermore, if this bit is set, releasepage() and invalidatepage() operations
will be called on a page to get rid of it, even if PG_private is not set.
@@ -372,13 +630,11 @@ will be called on a page to get rid of it, even if PG_private is not set.
This bit does not overlap with such as PG_private. This means that FS-Cache
can be used with a filesystem that uses the block buffering code.
-There are a number of operations defined on this flag:
+There are a number of operations defined for this flag:
int PageFsCache(struct page *page);
void SetPageFsCache(struct page *page)
void ClearPageFsCache(struct page *page)
int TestSetPageFsCache(struct page *page)
int TestClearPageFsCache(struct page *page)
-
-These functions are bit test, bit set, bit clear, bit test and set and bit
-test and clear operations on PG_fscache.
+ wait_on_page_fscache(struct page *page);
^ permalink raw reply related [flat|nested] 92+ messages in thread* [RFC PATCH 61/61] fscache: Document the rewritten cache backend API
2020-05-04 17:07 ` David Howells
` (60 preceding siblings ...)
(?)
@ 2020-05-04 17:16 ` David Howells
-1 siblings, 0 replies; 92+ messages in thread
From: David Howells @ 2020-05-04 17:16 UTC (permalink / raw)
To: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton
Cc: dhowells, Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs,
linux-cifs, ceph-devel, v9fs-developer, linux-fsdevel,
linux-kernel
Document the rewritten cache backend API.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/filesystems/caching/backend-api.txt | 172 ++++++++++++++++++---
1 file changed, 148 insertions(+), 24 deletions(-)
diff --git a/Documentation/filesystems/caching/backend-api.txt b/Documentation/filesystems/caching/backend-api.txt
index 710d10408fcb..9af6f9618576 100644
--- a/Documentation/filesystems/caching/backend-api.txt
+++ b/Documentation/filesystems/caching/backend-api.txt
@@ -42,7 +42,7 @@ previously initialised cache definition to:
struct fscache_object *fsdef,
const char *tagname);
-Two extra arguments should also be supplied:
+Two further arguments should also be supplied:
(*) "fsdef" which should point to the object representation for the FS-Cache
master index in this cache. Netfs primary index entries will be created
@@ -50,9 +50,7 @@ Two extra arguments should also be supplied:
successful and will release it upon withdrawal of the cache.
(*) "tagname" which, if given, should be a text string naming this cache. If
- this is NULL, the identifier will be used instead. For CacheFS, the
- identifier is set to name the underlying block device and the tag can be
- supplied by mount.
+ this is NULL, the identifier will be used instead.
This function may return -ENOMEM if it ran out of memory or -EEXIST if the tag
is already in use. 0 will be returned on success.
@@ -156,28 +154,45 @@ performed on the denizens of the cache. These are held in a structure of type:
(*) Allocate a new object [mandatory]:
struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
- struct fscache_cookie *cookie)
+ struct fscache_cookie *cookie,
+ struct fscache_object *parent)
This method is used to allocate a cache object representation to back a
cookie in a particular cache. fscache_object_init() should be called on
the object to initialise it prior to returning.
+ (*) Prepare lookup data [mandatory]:
- (*) Look up and create object [mandatory]:
+ void *(*prepare_lookup_data)(struct fscache_object *object);
- void (*lookup_object)(struct fscache_object *object)
+ This method is used to prepare lookup data that can be passed to the
+ lookup_object and create_object methods.
+
+
+ (*) Look up an object [mandatory]:
+
+ void (*lookup_object)(struct fscache_object *object,
+ void *lookup_data);
This method is used to look up an object, given that the object is already
- allocated and attached to the cookie. This should instantiate that object
- in the cache if it can.
+ allocated and attached to the cookie.
+
+
+ (*) Create an object [mandatory]:
+
+ void (*create_object)(struct fscache_object *object,
+ void *lookup_data);
+
+ This method is used to create an object that has previously been looked
+ up.
(*) Release lookup data [mandatory]:
- void (*lookup_complete)(struct fscache_object *object)
+ void (*free_lookup_data)(struct fscache_object *object,
+ void *lookup_data);
- This method is called to ask the cache to release any resources it was
- using to perform a lookup.
+ This method is called to clean up the lookup data.
(*) Increment object refcount [mandatory]:
@@ -202,23 +217,27 @@ performed on the denizens of the cache. These are held in a structure of type:
int (*update_object)(struct fscache_object *object)
- This is called to update the index entry for the specified object. The
- new information should be in object->cookie->aux and ->object_size.
+ This is called to update the coherency for the specified object and to
+ trim the cache object to size. The cache should use the information
+ stored in object->cookie->aux and ->object_size for this purpose.
(*) Invalidate data object [mandatory]:
- int (*invalidate_object)(struct fscache_operation *op)
+ int (*invalidate_object)(struct fscache_operation *op,
+ unsigned int flags)
This is called to invalidate a data object.
(*) Discard object [mandatory]:
- void (*drop_object)(struct fscache_object *object)
+ void (*drop_object)(struct fscache_object *object,
+ bool invalidate)
This method is called to indicate that an object has been unbound from its
- cookie.
+ cookie. If invalidate is true, the object should be removed from the
+ cache.
This method should not attempt to release any references held by the
caller. The caller will invoke the put_object() method as appropriate.
@@ -228,8 +247,16 @@ performed on the denizens of the cache. These are held in a structure of type:
void (*put_object)(struct fscache_object *object)
- This method is used to discard a reference to an object. The object may
- be freed when all the references to it are released.
+ This method is used to discard a reference to an object. The object
+ should be freeable when all the references to it are released.
+
+
+ (*) Get object reference count [mandatory]:
+
+ unsigned int (*get_object_usage)(const struct fscache_object *object);
+
+ This method is used to read the reference count on an object for display
+ purposes.
(*) Synchronise a cache [mandatory]:
@@ -258,12 +285,85 @@ performed on the denizens of the cache. These are held in a structure of type:
size if larger than that already.
+ (*) Shape request extent [mandatory]:
+
+ unsigned int (*shape_extent)(struct fscache_object *object,
+ struct fscache_extent *extent,
+ loff_t i_size, bool for_write);
+
+ This method is called to shape a request according to the granularity of
+ the cache's content tracking. The cache may expand and contract the
+ extent, but it is required to keep the start page within the shaped
+ extent.
+
+ (*) Read data from the cache [mandatory]:
+
+ int (*read)(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
+
+ This method is called to read data from the cache into the buffer
+ specified by 'iter'. The location and length of the read are specified in
+ the request descriptor. The operation must be synchronous if req->io_done
+ is not set. req->io_done() must be called if provided, though this may be
+ deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+ The netfs is expected to have rounded its request to units of
+ dio_block_size as returned by ->shape_extent().
+
+ req->error, req->transferred and req->data_from_cache should be updated as
+ appropriate and fscache_end_io_operation() and fscache_put_io_request()
+ called. If the operation didn't read all the data, -ENODATA should be
+ set.
+
+ (*) Write data to the cache [mandatory]:
+
+ int (*write)(struct fscache_object *object,
+ struct fscache_io_request *req,
+ struct iov_iter *iter);
+
+ This method is called to write data from the buffer specified by 'iter'
+ into the cache. The location and length of the write are specified in the
+ request descriptor. The operation must be synchronous if req->io_done is
+ not set. req->io_done() must be called if provided, though this may be
+ deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+ The netfs is expected to have rounded its request to units of
+ dio_block_size as returned by ->shape_extent().
+
+ req->error should be updated as appropriate and fscache_end_io_operation()
+ and fscache_put_io_request() called. If the operation didn't write all
+ the data, -ENOBUFS should be set.
+
+ (*) Prepare to write to a live cache object [mandatory]:
+
+ int (*prepare_to_write)(struct fscache_object *object);
+
+ This method is called when fscache_use_object() is called on an object
+ that already exists and will_modify is true and FSCACHE_OBJECT_LOCAL_WRITE
+ was not yet set on the object. This allows the cache to mark the object
+ as being dirty prior to writing data to it, or to take a local copy.
+
+ (*) Display object info [mandatory]:
+
+ int (*display_object)(struct seq_file *m, struct fscache_object *object);
+
+ This method is called to summarise an object in /proc/fs/fscache/objects.
+
+
==================
FS-CACHE UTILITIES
==================
FS-Cache provides some utilities that a cache backend may make use of:
+ (*) The filesystem index cookie:
+
+ struct fscache_cookie fscache_fsdef_index
+
+ This is the cookie to which the object passed to fscache_add_cache() are
+ bound.
+
(*) Note occurrence of an I/O error in a cache:
void fscache_io_error(struct fscache_cache *cache)
@@ -283,14 +383,13 @@ FS-Cache provides some utilities that a cache backend may make use of:
This initialises all the fields in an object representation.
- (*) Indicate the destruction of an object:
+ (*) Bracket the destruction of an object.
+ void fscache_object_destroy(struct fscache_object *object);
void fscache_object_destroyed(struct fscache_cache *cache);
- This must be called to inform FS-Cache that an object that belonged to a
- cache has been destroyed and deallocated. This will allow continuation
- of the cache withdrawal process when it is stopped pending destruction of
- all the objects.
+ These must be called on either side of object destruction to clean up the
+ FS-Cache parts of an object and do the accounting.
(*) Indicate that a stale object was found and discarded:
@@ -314,3 +413,28 @@ FS-Cache provides some utilities that a cache backend may make use of:
FSCACHE_OBJECT_NO_SPACE - there was insufficient cache space
FSCACHE_OBJECT_WAS_RETIRED - the object was retired when relinquished.
FSCACHE_OBJECT_WAS_CULLED - the object was culled to make space.
+
+
+ (*) Get pointers to the cookie index key and coherency data buffers:
+
+ void *fscache_get_key(struct fscache_cookie *cookie);
+ void *fscache_get_aux(struct fscache_cookie *cookie);
+
+ These handle the switching between internal buffering for small data and
+ kmalloc'd buffering for larger data.
+
+
+ (*) End an I/O operation:
+
+ void fscache_end_io_operation(struct fscache_cookie *cookie);
+
+ This is call to manage the accounting at the end of an I/O operation.
+
+
+ (*) Get/put references on I/O request descriptors.
+
+ void fscache_get_io_request(struct fscache_io_request *req);
+ void fscache_put_io_request(struct fscache_io_request *req);
+
+ Manage references on I/O request descriptors. These may be called in
+ softirq context.
^ permalink raw reply related [flat|nested] 92+ messages in thread* Re: [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter
@ 2020-05-04 17:54 ` Jeff Layton
0 siblings, 0 replies; 92+ messages in thread
From: Jeff Layton @ 2020-05-04 17:54 UTC (permalink / raw)
To: David Howells, Trond Myklebust, Anna Schumaker, Steve French
Cc: Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs, linux-cifs,
ceph-devel, v9fs-developer, linux-fsdevel, linux-kernel
On Mon, 2020-05-04 at 18:07 +0100, David Howells wrote:
>
> (3) Make NFS, CIFS, Ceph, 9P work with it. I'm hoping that Jeff Layton will
> do Ceph. As mentioned, I'm having a crack at NFS, but it's evolved a bit
> since I last looked at it and it might be easier if I can palm that off
> to someone more current in the NFS I/O code.
>
>
[...]
This looks like a really nice overhaul. I particularly love the
diffstat. Net removal of ~4000 lines!
I'll plan to draft up a patch for Ceph in the near future. The new API
seems to be quite different, so I imagine this will more or less be a
rip and replace on the old code.
--
Jeff Layton <jlayton@redhat.com>
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter
@ 2020-05-04 17:54 ` Jeff Layton
0 siblings, 0 replies; 92+ messages in thread
From: Jeff Layton @ 2020-05-04 17:54 UTC (permalink / raw)
To: David Howells, Trond Myklebust, Anna Schumaker, Steve French
Cc: Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
On Mon, 2020-05-04 at 18:07 +0100, David Howells wrote:
>
> (3) Make NFS, CIFS, Ceph, 9P work with it. I'm hoping that Jeff Layton will
> do Ceph. As mentioned, I'm having a crack at NFS, but it's evolved a bit
> since I last looked at it and it might be easier if I can palm that off
> to someone more current in the NFS I/O code.
>
>
[...]
This looks like a really nice overhaul. I particularly love the
diffstat. Net removal of ~4000 lines!
I'll plan to draft up a patch for Ceph in the near future. The new API
seems to be quite different, so I imagine this will more or less be a
rip and replace on the old code.
--
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter
@ 2020-05-05 6:05 ` Christoph Hellwig
0 siblings, 0 replies; 92+ messages in thread
From: Christoph Hellwig @ 2020-05-05 6:05 UTC (permalink / raw)
To: David Howells
Cc: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton,
Matthew Wilcox, Alexander Viro, linux-afs, linux-nfs, linux-cifs,
ceph-devel, v9fs-developer, linux-fsdevel, linux-kernel
Can you split this into a few smaller series? > 60 patches is beyond
reviewer comprehension.
^ permalink raw reply [flat|nested] 92+ messages in thread
* Re: [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter
@ 2020-05-05 6:05 ` Christoph Hellwig
0 siblings, 0 replies; 92+ messages in thread
From: Christoph Hellwig @ 2020-05-05 6:05 UTC (permalink / raw)
To: David Howells
Cc: Trond Myklebust, Anna Schumaker, Steve French, Jeff Layton,
Matthew Wilcox, Alexander Viro,
linux-afs-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-nfs-u79uwXL29TY76Z2rM5mHXA,
linux-cifs-u79uwXL29TY76Z2rM5mHXA,
ceph-devel-u79uwXL29TY76Z2rM5mHXA,
v9fs-developer-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Can you split this into a few smaller series? > 60 patches is beyond
reviewer comprehension.
^ permalink raw reply [flat|nested] 92+ messages in thread