* [PATCH] NFS using CacheFS @ 2004-10-04 20:45 ` Steve Dickson 0 siblings, 0 replies; 22+ messages in thread From: Steve Dickson @ 2004-10-04 20:45 UTC (permalink / raw) To: Trond Myklebust Cc: nfs, Linux filesystem caching discussion list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 2002 bytes --] Hey Trond, Here here is the first stab at having NFS use the CacheFS that is currently in the -mm tree. As you know CacheFS is a caching filesystem that allows network filesystems, such as AFS and now NFS, to cache data pages on the local disk. This is a big win (especially with read-only filesystems) since it drastically cuts down on the amount of data read across the network. Plus the cached data survives umounts and reboots (since CacheFS journals the data). So when a client comes back up or the filesystem is remounted, all that is needed is a few small getattrs to insure the data is still valid. If it is, NFS reads are turned into local disk reads.... A Huge gain in performance!!!! I have structured that patch so CacheFS is a mount option. I figured this was the safest way to introduce CacheFS and not completely break NFS.... And if for some reason, NFS is not able to get an initial Cachefs cookie (i.e. it can't use CacheFS) , NFS will dynamically turn off the mount option, disabling the use of the cache.. But as usual there are some issues.... 1) NFS aliasing. The fact that: mount hades:/hades /a mount hades:/hades/xxx /b creates separate super blocks causes problems for CacheFS. With the current -mm code, these type of mount actual causes CacheFS to crash (although I do have a proposed patch that David is looking at). 2) NFS4 is not supported. I simply have not had time to get this working. But I will!! 3) There is no user level support. I realize this is extremely cheesy but I noticed that the NFS posix mount option (in the 2.6 kernel) was no longer being used, so I high jacked it. Which means to make NFS to used CacheFS you need to use the posix option: mount -o posix server:/export/home /mnt/server/home Comments? SteveD. PS. The new CacheFS mailing that have been set up is at http://www.redhat.com/mailman/listinfo/linux-cachefs I encourage you and anybody else interested in this type of technology to subscribe.... [-- Attachment #2: linux-2.6.9-rc3-mm2-cachefs-nfs.patch --] [-- Type: text/x-patch, Size: 23671 bytes --] --- 2.6.9-rc3-mm2/include/linux/nfs_fs.h.orig 2004-10-04 09:32:24.883031000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_fs.h 2004-10-04 11:02:34.664243000 -0400 @@ -30,6 +30,7 @@ #include <linux/nfs_xdr.h> #include <linux/rwsem.h> #include <linux/workqueue.h> +#include <linux/cachefs.h> /* * Enable debugging support for nfs client. @@ -189,6 +190,10 @@ struct nfs_inode { struct rw_semaphore rwsem; #endif /* CONFIG_NFS_V4*/ +#ifdef CONFIG_NFS_CACHEFS + struct cachefs_cookie *cachefs; +#endif + struct inode vfs_inode; }; --- 2.6.9-rc3-mm2/include/linux/nfs_fs_sb.h.orig 2004-10-04 09:32:24.890033000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_fs_sb.h 2004-10-04 11:02:34.694245000 -0400 @@ -3,6 +3,7 @@ #include <linux/list.h> #include <linux/backing-dev.h> +#include <linux/cachefs.h> /* * NFS client parameters stored in the superblock. @@ -46,6 +47,10 @@ struct nfs_server { that are supported on this filesystem */ #endif + +#ifdef CONFIG_NFS_CACHEFS + struct cachefs_cookie *cachefs; /* cache cookie */ +#endif }; /* Server capabilities */ --- 2.6.9-rc3-mm2/include/linux/nfs_mount.h.orig 2004-08-14 06:54:47.000000000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_mount.h 2004-10-04 11:02:34.724245000 -0400 @@ -60,6 +60,7 @@ struct nfs_mount_data { #define NFS_MOUNT_BROKEN_SUID 0x0400 /* 4 */ #define NFS_MOUNT_STRICTLOCK 0x1000 /* reserved for NFSv4 */ #define NFS_MOUNT_SECFLAVOUR 0x2000 /* 5 */ +#define NFS_MOUNT_CACHEFS NFS_MOUNT_POSIX #define NFS_MOUNT_FLAGMASK 0xFFFF #endif --- 2.6.9-rc3-mm2/fs/nfs/file.c.orig 2004-10-04 09:32:20.529019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/file.c 2004-10-04 11:02:34.729243000 -0400 @@ -27,9 +27,11 @@ #include <linux/slab.h> #include <linux/pagemap.h> #include <linux/smp_lock.h> +#include <linux/buffer_head.h> #include <asm/uaccess.h> #include <asm/system.h> +#include "nfs-cachefs.h" #include "delegation.h" @@ -240,6 +242,69 @@ static int nfs_commit_write(struct file return status; } +#ifdef CONFIG_NFS_CACHEFS +static int nfs_invalidatepage(struct page *page, unsigned long offset) +{ + int ret = 1; + struct nfs_server *server = NFS_SERVER(page->mapping->host); + + BUG_ON(!PageLocked(page)); + + if (server->flags & NFS_MOUNT_CACHEFS) { + if (PagePrivate(page)) { + struct nfs_inode *nfsi = NFS_I(page->mapping->host); + + dfprintk(PAGECACHE,"NFS: cachefs invalidatepage (0x%p/0x%p/0x%p)\n", + nfsi->cachefs, page, nfsi); + + cachefs_uncache_page(nfsi->cachefs, page); + + if (offset == 0) { + BUG_ON(!PageLocked(page)); + ret = 0; + if (!PageWriteback(page)) + ret = page->mapping->a_ops->releasepage(page, 0); + } + } + } else + ret = 0; + + return ret; +} +static int nfs_releasepage(struct page *page, int gfp_flags) +{ + struct cachefs_page *pageio; + struct nfs_server *server = NFS_SERVER(page->mapping->host); + + if (server->flags & NFS_MOUNT_CACHEFS && PagePrivate(page)) { + struct nfs_inode *nfsi = NFS_I(page->mapping->host); + + dfprintk(PAGECACHE,"NFS: cachefs releasepage (0x%p/0x%p/0x%p)\n", + nfsi->cachefs, page, nfsi); + + cachefs_uncache_page(nfsi->cachefs, page); + pageio = (struct cachefs_page *) page->private; + page->private = 0; + ClearPagePrivate(page); + + if (pageio) + kfree(pageio); + } + + return 0; +} +static int nfs_mkwrite(struct page *page) +{ + wait_on_page_fs_misc(page); + return 0; +} +#endif + +/* + * since we use page->private for our own nefarious purposes when using cachefs, we have to + * override extra address space ops to prevent fs/buffer.c from getting confused, even though we + * may not have asked its opinion + */ struct address_space_operations nfs_file_aops = { .readpage = nfs_readpage, .readpages = nfs_readpages, @@ -251,6 +316,12 @@ struct address_space_operations nfs_file #ifdef CONFIG_NFS_DIRECTIO .direct_IO = nfs_direct_IO, #endif +#ifdef CONFIG_NFS_CACHEFS + .sync_page = block_sync_page, + .releasepage = nfs_releasepage, + .invalidatepage = nfs_invalidatepage, + .page_mkwrite = nfs_mkwrite, +#endif }; /* --- 2.6.9-rc3-mm2/fs/nfs/inode.c.orig 2004-10-04 09:32:20.538020000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/inode.c 2004-10-04 11:02:34.740244000 -0400 @@ -41,6 +41,8 @@ #include "delegation.h" +#include "nfs-cachefs.h" + #define NFSDBG_FACILITY NFSDBG_VFS #define NFS_PARANOIA 1 @@ -140,7 +142,7 @@ nfs_delete_inode(struct inode * inode) /* * For the moment, the only task for the NFS clear_inode method is to - * release the mmap credential + * release the mmap credential and release the inode's on-disc cache */ static void nfs_clear_inode(struct inode *inode) @@ -153,6 +155,16 @@ nfs_clear_inode(struct inode *inode) cred = nfsi->cache_access.cred; if (cred) put_rpccred(cred); + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE, "NFS: relinquish cookie (0x%p/0x%p)\n", + nfsi, nfsi->cachefs); + cachefs_relinquish_cookie(nfsi->cachefs, 0); + nfsi->cachefs = NULL; + } +#endif + BUG_ON(atomic_read(&nfsi->data_updates) != 0); } @@ -462,6 +474,21 @@ nfs_fill_super(struct super_block *sb, s server->namelen = NFS2_MAXNAMLEN; } +#ifdef CONFIG_NFS_CACHEFS + /* create a cache index for looking up filehandles */ + server->cachefs = NULL; + if (server->flags & NFS_MOUNT_CACHEFS) { + server->cachefs = cachefs_acquire_cookie(nfs_cache_netfs.primary_index, + &nfs_cache_fh_index_def, server); + if (server->cachefs == NULL) { + server->flags &= ~NFS_MOUNT_CACHEFS; + printk(KERN_WARNING "NFS: No CacheFS cookie. Turning CacheFS off!\n"); + } + dfprintk(PAGECACHE,"NFS: cookie (0x%p/0x%p/0x%p)\n", + sb, server, server->cachefs); + } +#endif + sb->s_op = &nfs_sops; return nfs_sb_init(sb, authflavor); } @@ -518,7 +545,7 @@ static int nfs_show_options(struct seq_f } nfs_info[] = { { NFS_MOUNT_SOFT, ",soft", ",hard" }, { NFS_MOUNT_INTR, ",intr", "" }, - { NFS_MOUNT_POSIX, ",posix", "" }, + { NFS_MOUNT_POSIX, ",cachefs", "" }, { NFS_MOUNT_TCP, ",tcp", ",udp" }, { NFS_MOUNT_NOCTO, ",nocto", "" }, { NFS_MOUNT_NOAC, ",noac", "" }, @@ -568,6 +595,17 @@ nfs_zap_caches(struct inode *inode) nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_DATA; else nfsi->flags |= NFS_INO_INVALID_ATTR; + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE,"NFS: zapping cookie (0x%p/0x%p/0x%p)\n", + inode, nfsi, nfsi->cachefs); + + cachefs_relinquish_cookie(nfsi->cachefs, 1); + nfsi->cachefs = NULL; + } +#endif + } /* @@ -705,6 +743,17 @@ nfs_fhget(struct super_block *sb, struct memset(nfsi->cookieverf, 0, sizeof(nfsi->cookieverf)); nfsi->cache_access.cred = NULL; +#ifdef CONFIG_NFS_CACHEFS +{ + struct nfs_server *server = NFS_SB(sb); + if (server->flags & NFS_MOUNT_CACHEFS) { + nfsi->cachefs = cachefs_acquire_cookie(server->cachefs, NULL, nfsi); + /* XXX: Add warning when NULL is returned */ + dprintk("NFS: fhget new cookie (0x%p/0x%p/0x%p)\n", + sb, nfsi, nfsi->cachefs); + } +} +#endif unlock_new_inode(inode); } else nfs_refresh_inode(inode, fattr); @@ -1009,6 +1058,19 @@ __nfs_revalidate_inode(struct nfs_server (long long)NFS_FILEID(inode)); /* This ensures we revalidate dentries */ nfsi->cache_change_attribute++; + +#ifdef CONFIG_NFS_CACHEFS + if (server->flags & NFS_MOUNT_CACHEFS) { + struct cachefs_cookie *old = nfsi->cachefs; + + /* retire the current cachefs cache and get a new one */ + cachefs_relinquish_cookie(nfsi->cachefs, 1); + nfsi->cachefs = cachefs_acquire_cookie(server->cachefs, NULL, nfsi); + dfprintk(PAGECACHE, + "NFS: revalidation new cookie (0x%p/0x%p/0x%p/0x%p)\n", + server, nfsi, old, nfsi->cachefs); + } +#endif } dfprintk(PAGECACHE, "NFS: (%s/%Ld) revalidation complete\n", inode->i_sb->s_id, @@ -1417,6 +1479,14 @@ static struct super_block *nfs_get_sb(st return ERR_PTR(-EINVAL); } +#ifndef CONFIG_NFS_CACHEFS + if (data->flags & NFS_MOUNT_CACHEFS) { + printk(KERN_WARNING "NFS: kernel not compiled with CONFIG_NFS_CACHEFS\n"); + kfree(server); + return ERR_PTR(-EINVAL); + } +#endif + s = sget(fs_type, nfs_compare_super, nfs_set_super, server); if (IS_ERR(s) || s->s_root) { @@ -1449,6 +1519,14 @@ static void nfs_kill_super(struct super_ kill_anon_super(s); +#ifdef CONFIG_NFS_CACHEFS + if (server->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE,"NFS: killing cookie (0x%p/0x%p/0x%p)\n", + NFS_SB(s), server, server->cachefs); + cachefs_relinquish_cookie(server->cachefs, 0); + } +#endif + nfs4_renewd_prepare_shutdown(server); if (server->client != NULL && !IS_ERR(server->client)) @@ -1768,6 +1846,20 @@ static struct super_block *nfs4_get_sb(s s = ERR_PTR(-EIO); goto out_free; } +#ifdef CONFIG_NFS_CACHEFS + /* create a cache index for looking up filehandles */ + server->cachefs = NULL; + if (server->flags & NFS_MOUNT_CACHEFS) { + server->cachefs = cachefs_acquire_cookie(nfs_cache_netfs.primary_index, + &nfs_cache_fh_index_def, server); + if (server->cachefs == NULL) { + server->flags &= ~NFS_MOUNT_CACHEFS; + printk(KERN_WARNING "NFS: No CacheFS cookie. Turning CacheFS off!\n"); + } + dfprintk(PAGECACHE,"NFS: nfs4 cookie (0x%p/0x%p/0x%p)\n", + s, server, server->cachefs); + } +#endif error = nfs4_fill_super(s, data, flags & MS_VERBOSE ? 1 : 0); if (error) { @@ -1888,6 +1980,14 @@ static int __init init_nfs_fs(void) { int err; +#ifdef CONFIG_NFS_CACHEFS + /* we want to be able to cache */ + err = cachefs_register_netfs(&nfs_cache_netfs, + &nfs_cache_server_index_def); + if (err < 0) + goto out5; +#endif + err = nfs_init_nfspagecache(); if (err) goto out4; @@ -1923,6 +2023,10 @@ out2: out3: nfs_destroy_nfspagecache(); out4: +#ifdef CONFIG_NFS_CACHEFS + cachefs_unregister_netfs(&nfs_cache_netfs); +out5: +#endif return err; } @@ -1932,6 +2036,9 @@ static void __exit exit_nfs_fs(void) nfs_destroy_readpagecache(); nfs_destroy_inodecache(); nfs_destroy_nfspagecache(); +#ifdef CONFIG_NFS_CACHEFS + cachefs_unregister_netfs(&nfs_cache_netfs); +#endif #ifdef CONFIG_PROC_FS rpc_proc_unregister("nfs"); #endif --- 2.6.9-rc3-mm2/fs/nfs/Makefile.orig 2004-10-04 09:32:20.491019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/Makefile 2004-10-04 11:02:34.783245000 -0400 @@ -12,4 +12,5 @@ nfs-$(CONFIG_NFS_V4) += nfs4proc.o nfs4x delegation.o idmap.o \ callback.o callback_xdr.o callback_proc.o nfs-$(CONFIG_NFS_DIRECTIO) += direct.o +nfs-$(CONFIG_NFS_CACHEFS) += nfs-cachefs.o nfs-objs := $(nfs-y) --- /dev/null 2004-02-23 16:02:56.000000000 -0500 +++ 2.6.9-rc3-mm2/fs/nfs/nfs-cachefs.c 2004-10-04 11:02:34.793243000 -0400 @@ -0,0 +1,193 @@ +/* nfs-cachefs.c: NFS CacheFS interface + * + * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved. + * Written by David Howells (dhowells@redhat.com) + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + + +#include <linux/config.h> +#include <linux/init.h> +#include <linux/kernel.h> +#include <linux/mm.h> +#include <linux/nfs_fs.h> +#include <linux/nfs_fs_sb.h> + +#include "nfs-cachefs.h" + +#define NFS_CACHE_FH_INDEX_SIZE sizeof(struct nfs_fh) + +/* + * the root index is + */ +static struct cachefs_page *nfs_cache_get_page_cookie(struct page *page); + +static struct cachefs_netfs_operations nfs_cache_ops = { + .get_page_cookie = nfs_cache_get_page_cookie, +}; + +struct cachefs_netfs nfs_cache_netfs = { + .name = "nfs", + .version = 0, + .ops = &nfs_cache_ops, +}; + +/* + * the root index for the filesystem is defined by nfsd IP address and ports + */ +static cachefs_match_val_t nfs_cache_server_match(void *target, + const void *entry); +static void nfs_cache_server_update(void *source, void *entry); + +struct cachefs_index_def nfs_cache_server_index_def = { + .name = "server", + .data_size = 18, + .keys[0] = { CACHEFS_INDEX_KEYS_IPV6ADDR, 16 }, + .keys[1] = { CACHEFS_INDEX_KEYS_BIN, 2 }, + .match = nfs_cache_server_match, + .update = nfs_cache_server_update, +}; + +/* + * the primary index for each server is simply made up of a series of NFS file + * handles + */ +static cachefs_match_val_t nfs_cache_fh_match(void *target, const void *entry); +static void nfs_cache_fh_update(void *source, void *entry); + +struct cachefs_index_def nfs_cache_fh_index_def = { + .name = "fh", + .data_size = NFS_CACHE_FH_INDEX_SIZE, + .keys[0] = { CACHEFS_INDEX_KEYS_BIN, + sizeof(struct nfs_fh) }, + .match = nfs_cache_fh_match, + .update = nfs_cache_fh_update, +}; + +/* + * get a page cookie for the specified page + * - the cookie will be attached to page->private and PG_private will be set on + * the page + */ +static struct cachefs_page *nfs_cache_get_page_cookie(struct page *page) +{ + struct cachefs_page *page_cookie; + + page_cookie = cachefs_page_get_private(page, GFP_NOIO); + + return page_cookie; +} + +static const uint8_t nfs_cache_ipv6_wrapper_for_ipv4[12] = { + [0 ... 9] = 0x00, + [10 ... 11] = 0xff +}; + +/* + * match a server record obtained from the cache + */ +static cachefs_match_val_t nfs_cache_server_match(void *target, + const void *entry) +{ + struct nfs_server *server = target; + const uint8_t *data = entry; + + switch (server->addr.sin_family) { + case AF_INET: + if (memcmp(data + 0, + &nfs_cache_ipv6_wrapper_for_ipv4, + 12) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 12, &server->addr.sin_addr, 4) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 16, &server->addr.sin_port, 2) != 0) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + + case AF_INET6: + if (memcmp(data + 0, &server->addr.sin_addr, 16) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 16, &server->addr.sin_port, 2) != 0) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + + default: + return CACHEFS_MATCH_FAILED; + } +} + +/* + * update a server record in the cache + */ +static void nfs_cache_server_update(void *source, void *entry) +{ + struct nfs_server *server = source; + uint8_t *data = entry; + + switch (server->addr.sin_family) { + case AF_INET: + memcpy(data + 0, &nfs_cache_ipv6_wrapper_for_ipv4, 12); + memcpy(data + 12, &server->addr.sin_addr, 4); + memcpy(data + 16, &server->addr.sin_port, 2); + return; + + case AF_INET6: + memcpy(data + 0, &server->addr.sin_addr, 16); + memcpy(data + 16, &server->addr.sin_port, 2); + return; + + default: + return; + } +} + +/* + * match a file handle record obtained from the cache + */ +static cachefs_match_val_t nfs_cache_fh_match(void *target, const void *entry) +{ + struct nfs_inode *nfsi = target; + const uint8_t *data = entry; + int loop; + + /* check the file handle matches */ + if (memcmp(data, &nfsi->fh, sizeof(nfsi->fh)) == 0) { + + /* check the auxilliary data matches (if any) */ + for (loop = sizeof(nfsi->fh); + loop < NFS_CACHE_FH_INDEX_SIZE; + loop++) + if (data[loop]) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + } + + return CACHEFS_MATCH_FAILED; +} + +/* + * update a fh record in the cache + */ +static void nfs_cache_fh_update(void *source, void *entry) +{ + struct nfs_inode *nfsi = source; + uint8_t *data = entry; + + /* set the file handle */ + memcpy(data, &nfsi->fh, sizeof(nfsi->fh)); + + /* just clear the auxilliary data for now */ + memset(data + sizeof(nfsi->fh), + 0, + NFS_CACHE_FH_INDEX_SIZE - sizeof(nfsi->fh)); +} --- /dev/null 2004-02-23 16:02:56.000000000 -0500 +++ 2.6.9-rc3-mm2/fs/nfs/nfs-cachefs.h 2004-10-04 11:02:34.802243000 -0400 @@ -0,0 +1,27 @@ +/* nfs-cachefs.h: NFS CacheFS interface definitions + * + * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved. + * Written by David Howells (dhowells@redhat.com) + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + +#ifndef _NFS_CACHEFS_H +#define _NFS_CACHEFS_H + +#include <linux/cachefs.h> + +#ifdef CONFIG_NFS_CACHEFS +#ifndef CONFIG_CACHEFS +#error "CONFIG_NFS_CACHEFS is defined but not CONFIG_CACHEFS" +#endif + +extern struct cachefs_netfs nfs_cache_netfs; +extern struct cachefs_index_def nfs_cache_server_index_def; +extern struct cachefs_index_def nfs_cache_fh_index_def; + +#endif +#endif /* _NFS_CACHEFS_H */ --- 2.6.9-rc3-mm2/fs/nfs/read.c.orig 2004-10-04 09:32:20.619019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/read.c 2004-10-04 11:02:34.810243000 -0400 @@ -28,6 +28,7 @@ #include <linux/sunrpc/clnt.h> #include <linux/nfs_fs.h> #include <linux/nfs_page.h> +#include <linux/nfs_mount.h> #include <linux/smp_lock.h> #include <asm/system.h> @@ -88,6 +89,47 @@ int nfs_return_empty_page(struct page *p return 0; } +#ifdef CONFIG_NFS_CACHEFS +/* + * Store a newly fetched page in cachefs + */ +static void +nfs_readpage_to_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + dprintk("NFS: readpage_to_cachefs_complete (%p/%p/%p/%d)\n", + cookie_data, page, data, error); + + end_page_fs_misc(page); +} + +static inline void +nfs_readpage_to_cachefs(struct inode *inode, struct page *page, int sync) +{ + int ret; + + dprintk("NFS: readpage_to_cachefs(0x%p/0x%p/0x%p/%d)\n", + NFS_I(inode)->cachefs, page, inode, sync); + + SetPageFsMisc(page); + ret = cachefs_write_page(NFS_I(inode)->cachefs, page, + nfs_readpage_to_cachefs_complete, NULL, GFP_KERNEL); + if (ret != 0) { + dprintk("NFS: readpage_to_cachefs: error %d\n", ret); + cachefs_uncache_page(NFS_I(inode)->cachefs, page); + ClearPageFsMisc(page); + } + + unlock_page(page); +} +#else +static inline void +nfs_readpage_to_cachefs(struct inode *inode, struct page *page, int sync) +{ + BUG(); +} +#endif + + /* * Read a page synchronously. */ @@ -164,6 +206,13 @@ static int nfs_readpage_sync(struct nfs_ ClearPageError(page); result = 0; + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) + nfs_readpage_to_cachefs(inode, page, 1); + else + unlock_page(page); + + return result; + io_error: unlock_page(page); nfs_readdata_free(rdata); @@ -196,7 +245,13 @@ static int nfs_readpage_async(struct nfs static void nfs_readpage_release(struct nfs_page *req) { - unlock_page(req->wb_page); + struct inode *d_inode = req->wb_context->dentry->d_inode; + + if ((NFS_SERVER(d_inode)->flags & NFS_MOUNT_CACHEFS) && + PageUptodate(req->wb_page)) + nfs_readpage_to_cachefs(d_inode, req->wb_page, 0); + else + unlock_page(req->wb_page); nfs_clear_request(req); nfs_release_request(req); @@ -494,6 +549,64 @@ void nfs_readpage_result(struct rpc_task data->complete(data, status); } + +/* + * Read a page through the on-disc cache if possible + */ +#ifdef CONFIG_NFS_CACHEFS +static void +nfs_readpage_from_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + dprintk("NFS: readpage_from_cachefs_complete (0x%p/0x%p/0x%p/%d)\n", + cookie_data, page, data, error); + + if (error) + SetPageError(page); + else + SetPageUptodate(page); + + unlock_page(page); +} + +static inline int +nfs_readpage_from_cachefs(struct inode *inode, struct page *page) +{ + struct cachefs_page *pageio; + int ret; + + dprintk("NFS: readpage_from_cachefs(0x%p/0x%p/0x%p)\n", + NFS_I(inode)->cachefs, page, inode); + + pageio = cachefs_page_get_private(page, GFP_NOIO); + if (IS_ERR(pageio)) { + dprintk("NFS: cachefs_page_get_private error %ld\n", PTR_ERR(pageio)); + return PTR_ERR(pageio); + } + + ret = cachefs_read_or_alloc_page(NFS_I(inode)->cachefs, + page, + nfs_readpage_from_cachefs_complete, + NULL, + GFP_KERNEL); + + switch (ret) { + case 1: /* read BIO submitted and wb-journal entry found */ + BUG(); + + case 0: /* read BIO submitted (page in cachefs) */ + return ret; + + case -ENOBUFS: /* inode not in cache */ + case -ENODATA: /* page not in cache */ + dprintk("NFS: cachefs_read_or_alloc_page error %d\n", ret); + return 1; + + default: + return ret; + } +} +#endif + /* * Read a page over NFS. * We read the page synchronously in the following case: @@ -527,6 +640,13 @@ int nfs_readpage(struct file *file, stru ctx = get_nfs_open_context((struct nfs_open_context *) file->private_data); if (!IS_SYNC(inode)) { + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + error = nfs_readpage_from_cachefs(inode, page); + if (error < 0) + goto out_error; + if (error == 0) /* found it */ + return error; + } error = nfs_readpage_async(ctx, inode, page); goto out; } @@ -555,8 +675,21 @@ readpage_async_filler(void *data, struct struct inode *inode = page->mapping->host; struct nfs_page *new; unsigned int len; + int error; nfs_wb_page(inode, page); + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + error = nfs_readpage_from_cachefs(inode, page); + if (error < 0) + return error; + if (error == 0) { + return error; + } + } +#endif + len = nfs_page_length(inode, page); if (len == 0) return nfs_return_empty_page(page); --- 2.6.9-rc3-mm2/fs/nfs/write.c.orig 2004-10-04 09:32:20.629020000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/write.c 2004-10-04 11:02:34.820244000 -0400 @@ -273,6 +273,38 @@ static int wb_priority(struct writeback_ } /* + * store an updated page in cachefs + */ +#ifdef CONFIG_NFS_CACHEFS +static void +nfs_writepage_to_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + /* really need to synchronise the end of writeback, probably using a page flag */ +} +static inline void +nfs_writepage_to_cachefs(struct inode *inode, struct page *page) +{ + int ret; + + dprintk("NFS: writepage_to_cachefs (0x%p/0x%p/0x%p)\n", + NFS_I(inode)->cachefs, page, inode); + + ret = cachefs_write_page(NFS_I(inode)->cachefs, page, + nfs_writepage_to_cachefs_complete, NULL, GFP_KERNEL); + if (ret != 0 ) { + dprintk("NFS: cachefs_write_page error %d\n", ret); + cachefs_uncache_page(NFS_I(inode)->cachefs, page); + } +} +#else +static inline void +nfs_writepage_to_cachefs(struct inode *inode, struct page *page) +{ + BUG(); +} +#endif + +/* * Write an mmapped page to the server. */ int nfs_writepage(struct page *page, struct writeback_control *wbc) @@ -317,6 +349,10 @@ do_it: err = -EBADF; goto out; } + + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) + nfs_writepage_to_cachefs(inode, page); + lock_kernel(); if (!IS_SYNC(inode) && inode_referenced) { err = nfs_writepage_async(ctx, inode, page, 0, offset); @@ -1322,6 +1358,7 @@ nfs_commit_done(struct rpc_task *task) (long long)NFS_FILEID(req->wb_context->dentry->d_inode), req->wb_bytes, (long long)req_offset(req)); + if (task->tk_status < 0) { req->wb_context->error = task->tk_status; nfs_inode_remove_request(req); --- 2.6.9-rc3-mm2/fs/Kconfig.orig 2004-10-04 09:32:57.727132000 -0400 +++ 2.6.9-rc3-mm2/fs/Kconfig 2004-10-04 11:02:34.836246000 -0400 @@ -1482,6 +1482,13 @@ config NFS_V4 If unsure, say N. +config NFS_CACHEFS + bool "Provide NFS client caching support through CacheFS" + depends on NFS_FS && CACHEFS && EXPERIMENTAL + help + Say Y here if you want NFS data to be cached locally on disc through + the CacheFS filesystem. + config NFS_DIRECTIO bool "Allow direct I/O on NFS files (EXPERIMENTAL)" depends on NFS_FS && EXPERIMENTAL ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH] NFS using CacheFS @ 2004-10-04 20:45 ` Steve Dickson 0 siblings, 0 replies; 22+ messages in thread From: Steve Dickson @ 2004-10-04 20:45 UTC (permalink / raw) To: Trond Myklebust Cc: nfs, Linux filesystem caching discussion list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 2002 bytes --] Hey Trond, Here here is the first stab at having NFS use the CacheFS that is currently in the -mm tree. As you know CacheFS is a caching filesystem that allows network filesystems, such as AFS and now NFS, to cache data pages on the local disk. This is a big win (especially with read-only filesystems) since it drastically cuts down on the amount of data read across the network. Plus the cached data survives umounts and reboots (since CacheFS journals the data). So when a client comes back up or the filesystem is remounted, all that is needed is a few small getattrs to insure the data is still valid. If it is, NFS reads are turned into local disk reads.... A Huge gain in performance!!!! I have structured that patch so CacheFS is a mount option. I figured this was the safest way to introduce CacheFS and not completely break NFS.... And if for some reason, NFS is not able to get an initial Cachefs cookie (i.e. it can't use CacheFS) , NFS will dynamically turn off the mount option, disabling the use of the cache.. But as usual there are some issues.... 1) NFS aliasing. The fact that: mount hades:/hades /a mount hades:/hades/xxx /b creates separate super blocks causes problems for CacheFS. With the current -mm code, these type of mount actual causes CacheFS to crash (although I do have a proposed patch that David is looking at). 2) NFS4 is not supported. I simply have not had time to get this working. But I will!! 3) There is no user level support. I realize this is extremely cheesy but I noticed that the NFS posix mount option (in the 2.6 kernel) was no longer being used, so I high jacked it. Which means to make NFS to used CacheFS you need to use the posix option: mount -o posix server:/export/home /mnt/server/home Comments? SteveD. PS. The new CacheFS mailing that have been set up is at http://www.redhat.com/mailman/listinfo/linux-cachefs I encourage you and anybody else interested in this type of technology to subscribe.... [-- Attachment #2: linux-2.6.9-rc3-mm2-cachefs-nfs.patch --] [-- Type: text/x-patch, Size: 23671 bytes --] --- 2.6.9-rc3-mm2/include/linux/nfs_fs.h.orig 2004-10-04 09:32:24.883031000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_fs.h 2004-10-04 11:02:34.664243000 -0400 @@ -30,6 +30,7 @@ #include <linux/nfs_xdr.h> #include <linux/rwsem.h> #include <linux/workqueue.h> +#include <linux/cachefs.h> /* * Enable debugging support for nfs client. @@ -189,6 +190,10 @@ struct nfs_inode { struct rw_semaphore rwsem; #endif /* CONFIG_NFS_V4*/ +#ifdef CONFIG_NFS_CACHEFS + struct cachefs_cookie *cachefs; +#endif + struct inode vfs_inode; }; --- 2.6.9-rc3-mm2/include/linux/nfs_fs_sb.h.orig 2004-10-04 09:32:24.890033000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_fs_sb.h 2004-10-04 11:02:34.694245000 -0400 @@ -3,6 +3,7 @@ #include <linux/list.h> #include <linux/backing-dev.h> +#include <linux/cachefs.h> /* * NFS client parameters stored in the superblock. @@ -46,6 +47,10 @@ struct nfs_server { that are supported on this filesystem */ #endif + +#ifdef CONFIG_NFS_CACHEFS + struct cachefs_cookie *cachefs; /* cache cookie */ +#endif }; /* Server capabilities */ --- 2.6.9-rc3-mm2/include/linux/nfs_mount.h.orig 2004-08-14 06:54:47.000000000 -0400 +++ 2.6.9-rc3-mm2/include/linux/nfs_mount.h 2004-10-04 11:02:34.724245000 -0400 @@ -60,6 +60,7 @@ struct nfs_mount_data { #define NFS_MOUNT_BROKEN_SUID 0x0400 /* 4 */ #define NFS_MOUNT_STRICTLOCK 0x1000 /* reserved for NFSv4 */ #define NFS_MOUNT_SECFLAVOUR 0x2000 /* 5 */ +#define NFS_MOUNT_CACHEFS NFS_MOUNT_POSIX #define NFS_MOUNT_FLAGMASK 0xFFFF #endif --- 2.6.9-rc3-mm2/fs/nfs/file.c.orig 2004-10-04 09:32:20.529019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/file.c 2004-10-04 11:02:34.729243000 -0400 @@ -27,9 +27,11 @@ #include <linux/slab.h> #include <linux/pagemap.h> #include <linux/smp_lock.h> +#include <linux/buffer_head.h> #include <asm/uaccess.h> #include <asm/system.h> +#include "nfs-cachefs.h" #include "delegation.h" @@ -240,6 +242,69 @@ static int nfs_commit_write(struct file return status; } +#ifdef CONFIG_NFS_CACHEFS +static int nfs_invalidatepage(struct page *page, unsigned long offset) +{ + int ret = 1; + struct nfs_server *server = NFS_SERVER(page->mapping->host); + + BUG_ON(!PageLocked(page)); + + if (server->flags & NFS_MOUNT_CACHEFS) { + if (PagePrivate(page)) { + struct nfs_inode *nfsi = NFS_I(page->mapping->host); + + dfprintk(PAGECACHE,"NFS: cachefs invalidatepage (0x%p/0x%p/0x%p)\n", + nfsi->cachefs, page, nfsi); + + cachefs_uncache_page(nfsi->cachefs, page); + + if (offset == 0) { + BUG_ON(!PageLocked(page)); + ret = 0; + if (!PageWriteback(page)) + ret = page->mapping->a_ops->releasepage(page, 0); + } + } + } else + ret = 0; + + return ret; +} +static int nfs_releasepage(struct page *page, int gfp_flags) +{ + struct cachefs_page *pageio; + struct nfs_server *server = NFS_SERVER(page->mapping->host); + + if (server->flags & NFS_MOUNT_CACHEFS && PagePrivate(page)) { + struct nfs_inode *nfsi = NFS_I(page->mapping->host); + + dfprintk(PAGECACHE,"NFS: cachefs releasepage (0x%p/0x%p/0x%p)\n", + nfsi->cachefs, page, nfsi); + + cachefs_uncache_page(nfsi->cachefs, page); + pageio = (struct cachefs_page *) page->private; + page->private = 0; + ClearPagePrivate(page); + + if (pageio) + kfree(pageio); + } + + return 0; +} +static int nfs_mkwrite(struct page *page) +{ + wait_on_page_fs_misc(page); + return 0; +} +#endif + +/* + * since we use page->private for our own nefarious purposes when using cachefs, we have to + * override extra address space ops to prevent fs/buffer.c from getting confused, even though we + * may not have asked its opinion + */ struct address_space_operations nfs_file_aops = { .readpage = nfs_readpage, .readpages = nfs_readpages, @@ -251,6 +316,12 @@ struct address_space_operations nfs_file #ifdef CONFIG_NFS_DIRECTIO .direct_IO = nfs_direct_IO, #endif +#ifdef CONFIG_NFS_CACHEFS + .sync_page = block_sync_page, + .releasepage = nfs_releasepage, + .invalidatepage = nfs_invalidatepage, + .page_mkwrite = nfs_mkwrite, +#endif }; /* --- 2.6.9-rc3-mm2/fs/nfs/inode.c.orig 2004-10-04 09:32:20.538020000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/inode.c 2004-10-04 11:02:34.740244000 -0400 @@ -41,6 +41,8 @@ #include "delegation.h" +#include "nfs-cachefs.h" + #define NFSDBG_FACILITY NFSDBG_VFS #define NFS_PARANOIA 1 @@ -140,7 +142,7 @@ nfs_delete_inode(struct inode * inode) /* * For the moment, the only task for the NFS clear_inode method is to - * release the mmap credential + * release the mmap credential and release the inode's on-disc cache */ static void nfs_clear_inode(struct inode *inode) @@ -153,6 +155,16 @@ nfs_clear_inode(struct inode *inode) cred = nfsi->cache_access.cred; if (cred) put_rpccred(cred); + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE, "NFS: relinquish cookie (0x%p/0x%p)\n", + nfsi, nfsi->cachefs); + cachefs_relinquish_cookie(nfsi->cachefs, 0); + nfsi->cachefs = NULL; + } +#endif + BUG_ON(atomic_read(&nfsi->data_updates) != 0); } @@ -462,6 +474,21 @@ nfs_fill_super(struct super_block *sb, s server->namelen = NFS2_MAXNAMLEN; } +#ifdef CONFIG_NFS_CACHEFS + /* create a cache index for looking up filehandles */ + server->cachefs = NULL; + if (server->flags & NFS_MOUNT_CACHEFS) { + server->cachefs = cachefs_acquire_cookie(nfs_cache_netfs.primary_index, + &nfs_cache_fh_index_def, server); + if (server->cachefs == NULL) { + server->flags &= ~NFS_MOUNT_CACHEFS; + printk(KERN_WARNING "NFS: No CacheFS cookie. Turning CacheFS off!\n"); + } + dfprintk(PAGECACHE,"NFS: cookie (0x%p/0x%p/0x%p)\n", + sb, server, server->cachefs); + } +#endif + sb->s_op = &nfs_sops; return nfs_sb_init(sb, authflavor); } @@ -518,7 +545,7 @@ static int nfs_show_options(struct seq_f } nfs_info[] = { { NFS_MOUNT_SOFT, ",soft", ",hard" }, { NFS_MOUNT_INTR, ",intr", "" }, - { NFS_MOUNT_POSIX, ",posix", "" }, + { NFS_MOUNT_POSIX, ",cachefs", "" }, { NFS_MOUNT_TCP, ",tcp", ",udp" }, { NFS_MOUNT_NOCTO, ",nocto", "" }, { NFS_MOUNT_NOAC, ",noac", "" }, @@ -568,6 +595,17 @@ nfs_zap_caches(struct inode *inode) nfsi->flags |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_DATA; else nfsi->flags |= NFS_INO_INVALID_ATTR; + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE,"NFS: zapping cookie (0x%p/0x%p/0x%p)\n", + inode, nfsi, nfsi->cachefs); + + cachefs_relinquish_cookie(nfsi->cachefs, 1); + nfsi->cachefs = NULL; + } +#endif + } /* @@ -705,6 +743,17 @@ nfs_fhget(struct super_block *sb, struct memset(nfsi->cookieverf, 0, sizeof(nfsi->cookieverf)); nfsi->cache_access.cred = NULL; +#ifdef CONFIG_NFS_CACHEFS +{ + struct nfs_server *server = NFS_SB(sb); + if (server->flags & NFS_MOUNT_CACHEFS) { + nfsi->cachefs = cachefs_acquire_cookie(server->cachefs, NULL, nfsi); + /* XXX: Add warning when NULL is returned */ + dprintk("NFS: fhget new cookie (0x%p/0x%p/0x%p)\n", + sb, nfsi, nfsi->cachefs); + } +} +#endif unlock_new_inode(inode); } else nfs_refresh_inode(inode, fattr); @@ -1009,6 +1058,19 @@ __nfs_revalidate_inode(struct nfs_server (long long)NFS_FILEID(inode)); /* This ensures we revalidate dentries */ nfsi->cache_change_attribute++; + +#ifdef CONFIG_NFS_CACHEFS + if (server->flags & NFS_MOUNT_CACHEFS) { + struct cachefs_cookie *old = nfsi->cachefs; + + /* retire the current cachefs cache and get a new one */ + cachefs_relinquish_cookie(nfsi->cachefs, 1); + nfsi->cachefs = cachefs_acquire_cookie(server->cachefs, NULL, nfsi); + dfprintk(PAGECACHE, + "NFS: revalidation new cookie (0x%p/0x%p/0x%p/0x%p)\n", + server, nfsi, old, nfsi->cachefs); + } +#endif } dfprintk(PAGECACHE, "NFS: (%s/%Ld) revalidation complete\n", inode->i_sb->s_id, @@ -1417,6 +1479,14 @@ static struct super_block *nfs_get_sb(st return ERR_PTR(-EINVAL); } +#ifndef CONFIG_NFS_CACHEFS + if (data->flags & NFS_MOUNT_CACHEFS) { + printk(KERN_WARNING "NFS: kernel not compiled with CONFIG_NFS_CACHEFS\n"); + kfree(server); + return ERR_PTR(-EINVAL); + } +#endif + s = sget(fs_type, nfs_compare_super, nfs_set_super, server); if (IS_ERR(s) || s->s_root) { @@ -1449,6 +1519,14 @@ static void nfs_kill_super(struct super_ kill_anon_super(s); +#ifdef CONFIG_NFS_CACHEFS + if (server->flags & NFS_MOUNT_CACHEFS) { + dfprintk(PAGECACHE,"NFS: killing cookie (0x%p/0x%p/0x%p)\n", + NFS_SB(s), server, server->cachefs); + cachefs_relinquish_cookie(server->cachefs, 0); + } +#endif + nfs4_renewd_prepare_shutdown(server); if (server->client != NULL && !IS_ERR(server->client)) @@ -1768,6 +1846,20 @@ static struct super_block *nfs4_get_sb(s s = ERR_PTR(-EIO); goto out_free; } +#ifdef CONFIG_NFS_CACHEFS + /* create a cache index for looking up filehandles */ + server->cachefs = NULL; + if (server->flags & NFS_MOUNT_CACHEFS) { + server->cachefs = cachefs_acquire_cookie(nfs_cache_netfs.primary_index, + &nfs_cache_fh_index_def, server); + if (server->cachefs == NULL) { + server->flags &= ~NFS_MOUNT_CACHEFS; + printk(KERN_WARNING "NFS: No CacheFS cookie. Turning CacheFS off!\n"); + } + dfprintk(PAGECACHE,"NFS: nfs4 cookie (0x%p/0x%p/0x%p)\n", + s, server, server->cachefs); + } +#endif error = nfs4_fill_super(s, data, flags & MS_VERBOSE ? 1 : 0); if (error) { @@ -1888,6 +1980,14 @@ static int __init init_nfs_fs(void) { int err; +#ifdef CONFIG_NFS_CACHEFS + /* we want to be able to cache */ + err = cachefs_register_netfs(&nfs_cache_netfs, + &nfs_cache_server_index_def); + if (err < 0) + goto out5; +#endif + err = nfs_init_nfspagecache(); if (err) goto out4; @@ -1923,6 +2023,10 @@ out2: out3: nfs_destroy_nfspagecache(); out4: +#ifdef CONFIG_NFS_CACHEFS + cachefs_unregister_netfs(&nfs_cache_netfs); +out5: +#endif return err; } @@ -1932,6 +2036,9 @@ static void __exit exit_nfs_fs(void) nfs_destroy_readpagecache(); nfs_destroy_inodecache(); nfs_destroy_nfspagecache(); +#ifdef CONFIG_NFS_CACHEFS + cachefs_unregister_netfs(&nfs_cache_netfs); +#endif #ifdef CONFIG_PROC_FS rpc_proc_unregister("nfs"); #endif --- 2.6.9-rc3-mm2/fs/nfs/Makefile.orig 2004-10-04 09:32:20.491019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/Makefile 2004-10-04 11:02:34.783245000 -0400 @@ -12,4 +12,5 @@ nfs-$(CONFIG_NFS_V4) += nfs4proc.o nfs4x delegation.o idmap.o \ callback.o callback_xdr.o callback_proc.o nfs-$(CONFIG_NFS_DIRECTIO) += direct.o +nfs-$(CONFIG_NFS_CACHEFS) += nfs-cachefs.o nfs-objs := $(nfs-y) --- /dev/null 2004-02-23 16:02:56.000000000 -0500 +++ 2.6.9-rc3-mm2/fs/nfs/nfs-cachefs.c 2004-10-04 11:02:34.793243000 -0400 @@ -0,0 +1,193 @@ +/* nfs-cachefs.c: NFS CacheFS interface + * + * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved. + * Written by David Howells (dhowells@redhat.com) + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + + +#include <linux/config.h> +#include <linux/init.h> +#include <linux/kernel.h> +#include <linux/mm.h> +#include <linux/nfs_fs.h> +#include <linux/nfs_fs_sb.h> + +#include "nfs-cachefs.h" + +#define NFS_CACHE_FH_INDEX_SIZE sizeof(struct nfs_fh) + +/* + * the root index is + */ +static struct cachefs_page *nfs_cache_get_page_cookie(struct page *page); + +static struct cachefs_netfs_operations nfs_cache_ops = { + .get_page_cookie = nfs_cache_get_page_cookie, +}; + +struct cachefs_netfs nfs_cache_netfs = { + .name = "nfs", + .version = 0, + .ops = &nfs_cache_ops, +}; + +/* + * the root index for the filesystem is defined by nfsd IP address and ports + */ +static cachefs_match_val_t nfs_cache_server_match(void *target, + const void *entry); +static void nfs_cache_server_update(void *source, void *entry); + +struct cachefs_index_def nfs_cache_server_index_def = { + .name = "server", + .data_size = 18, + .keys[0] = { CACHEFS_INDEX_KEYS_IPV6ADDR, 16 }, + .keys[1] = { CACHEFS_INDEX_KEYS_BIN, 2 }, + .match = nfs_cache_server_match, + .update = nfs_cache_server_update, +}; + +/* + * the primary index for each server is simply made up of a series of NFS file + * handles + */ +static cachefs_match_val_t nfs_cache_fh_match(void *target, const void *entry); +static void nfs_cache_fh_update(void *source, void *entry); + +struct cachefs_index_def nfs_cache_fh_index_def = { + .name = "fh", + .data_size = NFS_CACHE_FH_INDEX_SIZE, + .keys[0] = { CACHEFS_INDEX_KEYS_BIN, + sizeof(struct nfs_fh) }, + .match = nfs_cache_fh_match, + .update = nfs_cache_fh_update, +}; + +/* + * get a page cookie for the specified page + * - the cookie will be attached to page->private and PG_private will be set on + * the page + */ +static struct cachefs_page *nfs_cache_get_page_cookie(struct page *page) +{ + struct cachefs_page *page_cookie; + + page_cookie = cachefs_page_get_private(page, GFP_NOIO); + + return page_cookie; +} + +static const uint8_t nfs_cache_ipv6_wrapper_for_ipv4[12] = { + [0 ... 9] = 0x00, + [10 ... 11] = 0xff +}; + +/* + * match a server record obtained from the cache + */ +static cachefs_match_val_t nfs_cache_server_match(void *target, + const void *entry) +{ + struct nfs_server *server = target; + const uint8_t *data = entry; + + switch (server->addr.sin_family) { + case AF_INET: + if (memcmp(data + 0, + &nfs_cache_ipv6_wrapper_for_ipv4, + 12) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 12, &server->addr.sin_addr, 4) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 16, &server->addr.sin_port, 2) != 0) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + + case AF_INET6: + if (memcmp(data + 0, &server->addr.sin_addr, 16) != 0) + return CACHEFS_MATCH_FAILED; + + if (memcmp(data + 16, &server->addr.sin_port, 2) != 0) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + + default: + return CACHEFS_MATCH_FAILED; + } +} + +/* + * update a server record in the cache + */ +static void nfs_cache_server_update(void *source, void *entry) +{ + struct nfs_server *server = source; + uint8_t *data = entry; + + switch (server->addr.sin_family) { + case AF_INET: + memcpy(data + 0, &nfs_cache_ipv6_wrapper_for_ipv4, 12); + memcpy(data + 12, &server->addr.sin_addr, 4); + memcpy(data + 16, &server->addr.sin_port, 2); + return; + + case AF_INET6: + memcpy(data + 0, &server->addr.sin_addr, 16); + memcpy(data + 16, &server->addr.sin_port, 2); + return; + + default: + return; + } +} + +/* + * match a file handle record obtained from the cache + */ +static cachefs_match_val_t nfs_cache_fh_match(void *target, const void *entry) +{ + struct nfs_inode *nfsi = target; + const uint8_t *data = entry; + int loop; + + /* check the file handle matches */ + if (memcmp(data, &nfsi->fh, sizeof(nfsi->fh)) == 0) { + + /* check the auxilliary data matches (if any) */ + for (loop = sizeof(nfsi->fh); + loop < NFS_CACHE_FH_INDEX_SIZE; + loop++) + if (data[loop]) + return CACHEFS_MATCH_FAILED; + + return CACHEFS_MATCH_SUCCESS; + } + + return CACHEFS_MATCH_FAILED; +} + +/* + * update a fh record in the cache + */ +static void nfs_cache_fh_update(void *source, void *entry) +{ + struct nfs_inode *nfsi = source; + uint8_t *data = entry; + + /* set the file handle */ + memcpy(data, &nfsi->fh, sizeof(nfsi->fh)); + + /* just clear the auxilliary data for now */ + memset(data + sizeof(nfsi->fh), + 0, + NFS_CACHE_FH_INDEX_SIZE - sizeof(nfsi->fh)); +} --- /dev/null 2004-02-23 16:02:56.000000000 -0500 +++ 2.6.9-rc3-mm2/fs/nfs/nfs-cachefs.h 2004-10-04 11:02:34.802243000 -0400 @@ -0,0 +1,27 @@ +/* nfs-cachefs.h: NFS CacheFS interface definitions + * + * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved. + * Written by David Howells (dhowells@redhat.com) + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + +#ifndef _NFS_CACHEFS_H +#define _NFS_CACHEFS_H + +#include <linux/cachefs.h> + +#ifdef CONFIG_NFS_CACHEFS +#ifndef CONFIG_CACHEFS +#error "CONFIG_NFS_CACHEFS is defined but not CONFIG_CACHEFS" +#endif + +extern struct cachefs_netfs nfs_cache_netfs; +extern struct cachefs_index_def nfs_cache_server_index_def; +extern struct cachefs_index_def nfs_cache_fh_index_def; + +#endif +#endif /* _NFS_CACHEFS_H */ --- 2.6.9-rc3-mm2/fs/nfs/read.c.orig 2004-10-04 09:32:20.619019000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/read.c 2004-10-04 11:02:34.810243000 -0400 @@ -28,6 +28,7 @@ #include <linux/sunrpc/clnt.h> #include <linux/nfs_fs.h> #include <linux/nfs_page.h> +#include <linux/nfs_mount.h> #include <linux/smp_lock.h> #include <asm/system.h> @@ -88,6 +89,47 @@ int nfs_return_empty_page(struct page *p return 0; } +#ifdef CONFIG_NFS_CACHEFS +/* + * Store a newly fetched page in cachefs + */ +static void +nfs_readpage_to_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + dprintk("NFS: readpage_to_cachefs_complete (%p/%p/%p/%d)\n", + cookie_data, page, data, error); + + end_page_fs_misc(page); +} + +static inline void +nfs_readpage_to_cachefs(struct inode *inode, struct page *page, int sync) +{ + int ret; + + dprintk("NFS: readpage_to_cachefs(0x%p/0x%p/0x%p/%d)\n", + NFS_I(inode)->cachefs, page, inode, sync); + + SetPageFsMisc(page); + ret = cachefs_write_page(NFS_I(inode)->cachefs, page, + nfs_readpage_to_cachefs_complete, NULL, GFP_KERNEL); + if (ret != 0) { + dprintk("NFS: readpage_to_cachefs: error %d\n", ret); + cachefs_uncache_page(NFS_I(inode)->cachefs, page); + ClearPageFsMisc(page); + } + + unlock_page(page); +} +#else +static inline void +nfs_readpage_to_cachefs(struct inode *inode, struct page *page, int sync) +{ + BUG(); +} +#endif + + /* * Read a page synchronously. */ @@ -164,6 +206,13 @@ static int nfs_readpage_sync(struct nfs_ ClearPageError(page); result = 0; + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) + nfs_readpage_to_cachefs(inode, page, 1); + else + unlock_page(page); + + return result; + io_error: unlock_page(page); nfs_readdata_free(rdata); @@ -196,7 +245,13 @@ static int nfs_readpage_async(struct nfs static void nfs_readpage_release(struct nfs_page *req) { - unlock_page(req->wb_page); + struct inode *d_inode = req->wb_context->dentry->d_inode; + + if ((NFS_SERVER(d_inode)->flags & NFS_MOUNT_CACHEFS) && + PageUptodate(req->wb_page)) + nfs_readpage_to_cachefs(d_inode, req->wb_page, 0); + else + unlock_page(req->wb_page); nfs_clear_request(req); nfs_release_request(req); @@ -494,6 +549,64 @@ void nfs_readpage_result(struct rpc_task data->complete(data, status); } + +/* + * Read a page through the on-disc cache if possible + */ +#ifdef CONFIG_NFS_CACHEFS +static void +nfs_readpage_from_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + dprintk("NFS: readpage_from_cachefs_complete (0x%p/0x%p/0x%p/%d)\n", + cookie_data, page, data, error); + + if (error) + SetPageError(page); + else + SetPageUptodate(page); + + unlock_page(page); +} + +static inline int +nfs_readpage_from_cachefs(struct inode *inode, struct page *page) +{ + struct cachefs_page *pageio; + int ret; + + dprintk("NFS: readpage_from_cachefs(0x%p/0x%p/0x%p)\n", + NFS_I(inode)->cachefs, page, inode); + + pageio = cachefs_page_get_private(page, GFP_NOIO); + if (IS_ERR(pageio)) { + dprintk("NFS: cachefs_page_get_private error %ld\n", PTR_ERR(pageio)); + return PTR_ERR(pageio); + } + + ret = cachefs_read_or_alloc_page(NFS_I(inode)->cachefs, + page, + nfs_readpage_from_cachefs_complete, + NULL, + GFP_KERNEL); + + switch (ret) { + case 1: /* read BIO submitted and wb-journal entry found */ + BUG(); + + case 0: /* read BIO submitted (page in cachefs) */ + return ret; + + case -ENOBUFS: /* inode not in cache */ + case -ENODATA: /* page not in cache */ + dprintk("NFS: cachefs_read_or_alloc_page error %d\n", ret); + return 1; + + default: + return ret; + } +} +#endif + /* * Read a page over NFS. * We read the page synchronously in the following case: @@ -527,6 +640,13 @@ int nfs_readpage(struct file *file, stru ctx = get_nfs_open_context((struct nfs_open_context *) file->private_data); if (!IS_SYNC(inode)) { + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + error = nfs_readpage_from_cachefs(inode, page); + if (error < 0) + goto out_error; + if (error == 0) /* found it */ + return error; + } error = nfs_readpage_async(ctx, inode, page); goto out; } @@ -555,8 +675,21 @@ readpage_async_filler(void *data, struct struct inode *inode = page->mapping->host; struct nfs_page *new; unsigned int len; + int error; nfs_wb_page(inode, page); + +#ifdef CONFIG_NFS_CACHEFS + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) { + error = nfs_readpage_from_cachefs(inode, page); + if (error < 0) + return error; + if (error == 0) { + return error; + } + } +#endif + len = nfs_page_length(inode, page); if (len == 0) return nfs_return_empty_page(page); --- 2.6.9-rc3-mm2/fs/nfs/write.c.orig 2004-10-04 09:32:20.629020000 -0400 +++ 2.6.9-rc3-mm2/fs/nfs/write.c 2004-10-04 11:02:34.820244000 -0400 @@ -273,6 +273,38 @@ static int wb_priority(struct writeback_ } /* + * store an updated page in cachefs + */ +#ifdef CONFIG_NFS_CACHEFS +static void +nfs_writepage_to_cachefs_complete(void *cookie_data, struct page *page, void *data, int error) +{ + /* really need to synchronise the end of writeback, probably using a page flag */ +} +static inline void +nfs_writepage_to_cachefs(struct inode *inode, struct page *page) +{ + int ret; + + dprintk("NFS: writepage_to_cachefs (0x%p/0x%p/0x%p)\n", + NFS_I(inode)->cachefs, page, inode); + + ret = cachefs_write_page(NFS_I(inode)->cachefs, page, + nfs_writepage_to_cachefs_complete, NULL, GFP_KERNEL); + if (ret != 0 ) { + dprintk("NFS: cachefs_write_page error %d\n", ret); + cachefs_uncache_page(NFS_I(inode)->cachefs, page); + } +} +#else +static inline void +nfs_writepage_to_cachefs(struct inode *inode, struct page *page) +{ + BUG(); +} +#endif + +/* * Write an mmapped page to the server. */ int nfs_writepage(struct page *page, struct writeback_control *wbc) @@ -317,6 +349,10 @@ do_it: err = -EBADF; goto out; } + + if (NFS_SERVER(inode)->flags & NFS_MOUNT_CACHEFS) + nfs_writepage_to_cachefs(inode, page); + lock_kernel(); if (!IS_SYNC(inode) && inode_referenced) { err = nfs_writepage_async(ctx, inode, page, 0, offset); @@ -1322,6 +1358,7 @@ nfs_commit_done(struct rpc_task *task) (long long)NFS_FILEID(req->wb_context->dentry->d_inode), req->wb_bytes, (long long)req_offset(req)); + if (task->tk_status < 0) { req->wb_context->error = task->tk_status; nfs_inode_remove_request(req); --- 2.6.9-rc3-mm2/fs/Kconfig.orig 2004-10-04 09:32:57.727132000 -0400 +++ 2.6.9-rc3-mm2/fs/Kconfig 2004-10-04 11:02:34.836246000 -0400 @@ -1482,6 +1482,13 @@ config NFS_V4 If unsure, say N. +config NFS_CACHEFS + bool "Provide NFS client caching support through CacheFS" + depends on NFS_FS && CACHEFS && EXPERIMENTAL + help + Say Y here if you want NFS data to be cached locally on disc through + the CacheFS filesystem. + config NFS_DIRECTIO bool "Allow direct I/O on NFS files (EXPERIMENTAL)" depends on NFS_FS && EXPERIMENTAL ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-04 20:45 ` Steve Dickson @ 2004-10-04 21:53 ` Trond Myklebust -1 siblings, 0 replies; 22+ messages in thread From: Trond Myklebust @ 2004-10-04 21:53 UTC (permalink / raw) To: Steve Dickson; +Cc: nfs, Linux filesystem caching discussion list, linux-kernel På må , 04/10/2004 klokka 22:45, skreiv Steve Dickson: > 3) There is no user level support. I realize this is extremely cheesy > but I noticed that the NFS posix mount option (in the 2.6 kernel) > was no longer being used, so I high jacked it. Which means > to make NFS to used CacheFS you need to use the posix option: > > mount -o posix server:/export/home /mnt/server/home This is my one and only real gripe about it. The posix mount option is clearly documented, so we really cannot play around with it. Why can't you just add a separate cachefs flag? Otherwise, I'm a bit dubious about the wisdom of putting nfs_invalidatepage() and nfs_releasepage() into fs/nfs/file.c. These are not file operations, but rather pure page cache operations. I would have thought that either read.c or possibly nfs-cachefs.c would be more appropriate. Please note too that Chuck has made generic functions for copying and comparing NFS filehandles. They should be used in nfs_cache_fh_match() & co. I'm a bit worried about the use of the raw IP address in nfs_cache_server_match(). It seems to me that when we add the NFSv4.1 support for trunking over several different transport mechanisms (RDMA, IPv4/v6 etc) on the same mountpoint, then we may end up with a problem. We can probably leave it in for now, but later we may want to consider switching to using server->hostname or something equivalent. Otherwise, it looks good. Looking forward to try it out... Cheers, Trond ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-04 21:53 ` Trond Myklebust 0 siblings, 0 replies; 22+ messages in thread From: Trond Myklebust @ 2004-10-04 21:53 UTC (permalink / raw) To: Steve Dickson; +Cc: nfs, Linux filesystem caching discussion list, linux-kernel P=E5 m=E5 , 04/10/2004 klokka 22:45, skreiv Steve Dickson: > 3) There is no user level support. I realize this is extremely cheesy > but I noticed that the NFS posix mount option (in the 2.6 kerne= l) > was no longer being used, so I high jacked it. Which means > to make NFS to used CacheFS you need to use the posix option: >=20 > mount -o posix server:/export/home /mnt/server/home This is my one and only real gripe about it. The posix mount option is clearly documented, so we really cannot play around with it. Why can't you just add a separate cachefs flag? Otherwise, I'm a bit dubious about the wisdom of putting nfs_invalidatepage() and nfs_releasepage() into fs/nfs/file.c. These ar= e not file operations, but rather pure page cache operations. I would hav= e thought that either read.c or possibly nfs-cachefs.c would be more appropriate. Please note too that Chuck has made generic functions for copying and comparing NFS filehandles. They should be used in nfs_cache_fh_match() = & co. I'm a bit worried about the use of the raw IP address in nfs_cache_server_match(). It seems to me that when we add the NFSv4.1 support for trunking over several different transport mechanisms (RDMA, IPv4/v6 etc) on the same mountpoint, then we may end up with a problem. We can probably leave it in for now, but later we may want to consider switching to using server->hostname or something equivalent. Otherwise, it looks good. Looking forward to try it out... Cheers, Trond ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-04 20:45 ` Steve Dickson @ 2004-10-05 10:01 ` David Howells -1 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-05 10:01 UTC (permalink / raw) To: Trond Myklebust Cc: Steve Dickson, nfs, Linux filesystem caching discussion list, linux-kernel > This is my one and only real gripe about it. The posix mount option is > clearly documented, so we really cannot play around with it. Why can't > you just add a separate cachefs flag? That's just for the prototype. It will be changed later. It's also a stand-in for the superblock un-aliasing patch. > I'm a bit worried about the use of the raw IP address in > nfs_cache_server_match(). It seems to me that when we add the NFSv4.1 > support for trunking over several different transport mechanisms (RDMA, > IPv4/v6 etc) on the same mountpoint, then we may end up with a problem. We > can probably leave it in for now, but later we may want to consider > switching to using server->hostname or something equivalent. There needs to be some way of matching NFS file handle sets in the cache if you want persistant caches. The only way I could see was to use IP addr and port number for NFS. However, changing this later is trivial. If you bump the version number in the netfs definition, then cachefs will nuke existing data in the cache that isn't of the expected format. David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-05 10:01 ` David Howells 0 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-05 10:01 UTC (permalink / raw) To: Trond Myklebust Cc: Steve Dickson, nfs, Linux filesystem caching discussion list, linux-kernel > This is my one and only real gripe about it. The posix mount option is > clearly documented, so we really cannot play around with it. Why can't > you just add a separate cachefs flag? That's just for the prototype. It will be changed later. It's also a stand-in for the superblock un-aliasing patch. > I'm a bit worried about the use of the raw IP address in > nfs_cache_server_match(). It seems to me that when we add the NFSv4.1 > support for trunking over several different transport mechanisms (RDMA, > IPv4/v6 etc) on the same mountpoint, then we may end up with a problem. We > can probably leave it in for now, but later we may want to consider > switching to using server->hostname or something equivalent. There needs to be some way of matching NFS file handle sets in the cache if you want persistant caches. The only way I could see was to use IP addr and port number for NFS. However, changing this later is trivial. If you bump the version number in the netfs definition, then cachefs will nuke existing data in the cache that isn't of the expected format. David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-04 20:45 ` Steve Dickson @ 2004-10-08 4:36 ` Clemens Schwaighofer -1 siblings, 0 replies; 22+ messages in thread From: Clemens Schwaighofer @ 2004-10-08 4:36 UTC (permalink / raw) To: Steve Dickson Cc: Trond Myklebust, nfs, Linux filesystem caching discussion list, linux-kernel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/05/2004 05:45 AM, Steve Dickson wrote: | 3) There is no user level support. I realize this is extremely cheesy | but I noticed that the NFS posix mount option (in the 2.6 kernel) | was no longer being used, so I high jacked it. Which means | to make NFS to used CacheFS you need to use the posix option: | | mount -o posix server:/export/home /mnt/server/home brr :) why is it posix, this is so out of the context for me (as a user). Is it possible to have a cachefs flag. Would make it more logical. - -- [ Clemens Schwaighofer -----=====:::::~ ] [ TBWA\ && TEQUILA\ Japan IT Group ] [ 6-17-2 Ginza Chuo-ku, Tokyo 104-0061, JAPAN ] [ Tel: +81-(0)3-3545-7703 Fax: +81-(0)3-3545-7343 ] [ http://www.tequila.co.jp http://www.tbwajapan.co.jp ] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBZhlPjBz/yQjBxz8RAtKbAKCLWJBqWQUcroSaLzlrH1r2nuqI8gCfUzyL kpANJQ4pYt1026VBRm8UlpU= =NGFC -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-08 4:36 ` Clemens Schwaighofer 0 siblings, 0 replies; 22+ messages in thread From: Clemens Schwaighofer @ 2004-10-08 4:36 UTC (permalink / raw) To: Steve Dickson Cc: Trond Myklebust, nfs, Linux filesystem caching discussion list, linux-kernel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/05/2004 05:45 AM, Steve Dickson wrote: | 3) There is no user level support. I realize this is extremely cheesy | but I noticed that the NFS posix mount option (in the 2.6 kernel) | was no longer being used, so I high jacked it. Which means | to make NFS to used CacheFS you need to use the posix option: | | mount -o posix server:/export/home /mnt/server/home brr :) why is it posix, this is so out of the context for me (as a user). Is it possible to have a cachefs flag. Would make it more logical. - -- [ Clemens Schwaighofer -----=====:::::~ ] [ TBWA\ && TEQUILA\ Japan IT Group ] [ 6-17-2 Ginza Chuo-ku, Tokyo 104-0061, JAPAN ] [ Tel: +81-(0)3-3545-7703 Fax: +81-(0)3-3545-7343 ] [ http://www.tequila.co.jp http://www.tbwajapan.co.jp ] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBZhlPjBz/yQjBxz8RAtKbAKCLWJBqWQUcroSaLzlrH1r2nuqI8gCfUzyL kpANJQ4pYt1026VBRm8UlpU= =NGFC -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-08 4:36 ` Clemens Schwaighofer @ 2004-10-08 11:22 ` Steve Dickson -1 siblings, 0 replies; 22+ messages in thread From: Steve Dickson @ 2004-10-08 11:22 UTC (permalink / raw) To: Clemens Schwaighofer Cc: nfs, Linux filesystem caching discussion list, linux-kernel Clemens Schwaighofer wrote: > brr :) why is it posix, this is so out of the context for me (as a > user). Is it possible to have a cachefs flag. Would make it more logical. Because it was the easiest way to get things started (i.e. no userlevel changes needed at all).... The 'fscache' flag will be coming along with the nfs4 support, since nfs4 mounting code does not have an open (unused) mounting flag.... SteveD. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-08 11:22 ` Steve Dickson 0 siblings, 0 replies; 22+ messages in thread From: Steve Dickson @ 2004-10-08 11:22 UTC (permalink / raw) To: Clemens Schwaighofer Cc: nfs, Linux filesystem caching discussion list, linux-kernel Clemens Schwaighofer wrote: > brr :) why is it posix, this is so out of the context for me (as a > user). Is it possible to have a cachefs flag. Would make it more logical. Because it was the easiest way to get things started (i.e. no userlevel changes needed at all).... The 'fscache' flag will be coming along with the nfs4 support, since nfs4 mounting code does not have an open (unused) mounting flag.... SteveD. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-08 11:22 ` Steve Dickson (?) @ 2004-10-08 15:39 ` Clemens Schwaighofer -1 siblings, 0 replies; 22+ messages in thread From: Clemens Schwaighofer @ 2004-10-08 15:39 UTC (permalink / raw) To: Steve Dickson; +Cc: Linux filesystem caching discussion list, linux-kernel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/08/2004 08:22 PM, Steve Dickson wrote: > Clemens Schwaighofer wrote: > >> brr :) why is it posix, this is so out of the context for me (as a >> user). Is it possible to have a cachefs flag. Would make it more logical. > > > Because it was the easiest way to get things started (i.e. no userlevel > changes needed at all).... The 'fscache' flag will be coming along with > the nfs4 support, since nfs4 mounting code does not have an open > (unused) mounting flag.... okay, thank you for the information. Anyway inbetween I will try that out with NFS3 ... lg, clemens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBZrTAjBz/yQjBxz8RAoddAJwIVyxuMFy0tD9H8z8D/UKjXmvAtgCg4mcY 2NZyYRCsdR6CzbTKKcJiybw= =sKPN -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-08 11:22 ` Steve Dickson @ 2004-10-11 14:23 ` Martin Waitz -1 siblings, 0 replies; 22+ messages in thread From: Martin Waitz @ 2004-10-11 14:23 UTC (permalink / raw) To: Steve Dickson Cc: Clemens Schwaighofer, nfs, Linux filesystem caching discussion list, linux-kernel [-- Attachment #1: Type: text/plain, Size: 383 bytes --] hi :) On Fri, Oct 08, 2004 at 07:22:13AM -0400, Steve Dickson wrote: > The 'fscache' flag will be coming along with the nfs4 support, since > nfs4 mounting code does not have an open (unused) mounting flag.... is such a flag even neccessary? The way I see fscache is that its operations will be no-ops anyway if you haven't mounted any backing cache. -- Martin Waitz [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-11 14:23 ` Martin Waitz 0 siblings, 0 replies; 22+ messages in thread From: Martin Waitz @ 2004-10-11 14:23 UTC (permalink / raw) To: Steve Dickson Cc: Linux filesystem caching discussion list, nfs, linux-kernel, Clemens Schwaighofer [-- Attachment #1.1: Type: text/plain, Size: 383 bytes --] hi :) On Fri, Oct 08, 2004 at 07:22:13AM -0400, Steve Dickson wrote: > The 'fscache' flag will be coming along with the nfs4 support, since > nfs4 mounting code does not have an open (unused) mounting flag.... is such a flag even neccessary? The way I see fscache is that its operations will be no-ops anyway if you haven't mounted any backing cache. -- Martin Waitz [-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --] [-- Attachment #2: Type: text/plain, Size: 108 bytes --] -- Linux-cachefs mailing list Linux-cachefs@redhat.com http://www.redhat.com/mailman/listinfo/linux-cachefs ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS 2004-10-11 14:23 ` Martin Waitz @ 2004-10-11 15:30 ` Trond Myklebust -1 siblings, 0 replies; 22+ messages in thread From: Trond Myklebust @ 2004-10-11 15:30 UTC (permalink / raw) To: Martin Waitz Cc: Steve Dickson, Clemens Schwaighofer, nfs, Linux filesystem caching discussion list, linux-kernel På må , 11/10/2004 klokka 16:23, skreiv Martin Waitz: > hi :) > > On Fri, Oct 08, 2004 at 07:22:13AM -0400, Steve Dickson wrote: > > The 'fscache' flag will be coming along with the nfs4 support, since > > nfs4 mounting code does not have an open (unused) mounting flag.... > > is such a flag even neccessary? > The way I see fscache is that its operations will be no-ops anyway if you > haven't mounted any backing cache. You may not want to run cachefs on *all* your NFS partitions. It will slow you down on those partitions that have lots of cache contention. That said, David & co.: why did you choose not to use something similar to the Solaris syntax for cachefs? The "layered filesystem" syntax has the advantage that it would avoid entirely the need to change the NFS mount syntax, and would make it easier to port cachefs to cifs etc. Cheers, Trond ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] NFS using CacheFS @ 2004-10-11 15:30 ` Trond Myklebust 0 siblings, 0 replies; 22+ messages in thread From: Trond Myklebust @ 2004-10-11 15:30 UTC (permalink / raw) To: Martin Waitz Cc: linux-kernel, Linux filesystem caching discussion list, nfs, Steve Dickson, Clemens Schwaighofer P=E5 m=E5 , 11/10/2004 klokka 16:23, skreiv Martin Waitz: > hi :) >=20 > On Fri, Oct 08, 2004 at 07:22:13AM -0400, Steve Dickson wrote: > > The 'fscache' flag will be coming along with the nfs4 support, since > > nfs4 mounting code does not have an open (unused) mounting flag.... >=20 > is such a flag even neccessary? > The way I see fscache is that its operations will be no-ops anyway if y= ou > haven't mounted any backing cache. You may not want to run cachefs on *all* your NFS partitions. It will slow you down on those partitions that have lots of cache contention. That said, David & co.: why did you choose not to use something similar to the Solaris syntax for cachefs? The "layered filesystem" syntax has the advantage that it would avoid entirely the need to change the NFS mount syntax, and would make it easier to port cachefs to cifs etc. Cheers, Trond -- Linux-cachefs mailing list Linux-cachefs@redhat.com http://www.redhat.com/mailman/listinfo/linux-cachefs ^ permalink raw reply [flat|nested] 22+ messages in thread
* accessing/modifiying the nfs share files as different users... 2004-10-11 15:30 ` Trond Myklebust (?) @ 2004-10-11 20:31 ` bruce -1 siblings, 0 replies; 22+ messages in thread From: bruce @ 2004-10-11 20:31 UTC (permalink / raw) To: nfs hi... hopefully this is the nfs list. i've got an nfs share (linux rh8.0) that i can access/copy files to/from... however, i can't seem to figure out how to access it as a user other than root. basically, i'd like to set it up, to allow the 'apache' user to be able to automatically be able to add/modify files that reside on the share... in searching through google, i haven't come up with a way of being able to accomplish this... thanks... -bruce ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Linux-cachefs] Re: [PATCH] NFS using CacheFS 2004-10-11 14:23 ` Martin Waitz @ 2004-10-13 10:40 ` David Howells -1 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-13 10:40 UTC (permalink / raw) To: Trond Myklebust Cc: Linux filesystem caching discussion list, Martin Waitz, linux-kernel, nfs, Steve Dickson, Clemens Schwaighofer Trond Myklebust <trond.myklebust@fys.uio.no> wrote: > That said, David & co.: why did you choose not to use something similar > to the Solaris syntax for cachefs? Well... (1) when I wrote cachefs I didn't know that Solaris had such a thing (2) I don't know what Solaris syntax is - I've seen one example, and that didn't make a lot of sense (3) my way doesn't involve any changes to userspace programs such as mount > The "layered filesystem" syntax has the advantage that it would avoid > entirely the need to change the NFS mount syntax, And the disadvantage that we'd have to change the VFS to support it, I think. I don't know how this "layered filesystem" thing operates or is used, so I'm guessing. > and would make it easier to port cachefs to cifs etc. I doubt it. The netfs still has to interact with fscache internally to decide how match netfs files to cache files, which is what most of the interface is about; and to push/pull pages to/from the cache (this bit could possibly be made transparent, but I'm not sure how you'd do it on Linux with the present VM & VFS interfaces). David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Re: [PATCH] NFS using CacheFS @ 2004-10-13 10:40 ` David Howells 0 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-13 10:40 UTC (permalink / raw) To: Trond Myklebust Cc: Linux filesystem caching discussion list, nfs, Martin Waitz, linux-kernel, Steve Dickson, Clemens Schwaighofer Trond Myklebust <trond.myklebust@fys.uio.no> wrote: > That said, David & co.: why did you choose not to use something similar > to the Solaris syntax for cachefs? Well... (1) when I wrote cachefs I didn't know that Solaris had such a thing (2) I don't know what Solaris syntax is - I've seen one example, and that didn't make a lot of sense (3) my way doesn't involve any changes to userspace programs such as mount > The "layered filesystem" syntax has the advantage that it would avoid > entirely the need to change the NFS mount syntax, And the disadvantage that we'd have to change the VFS to support it, I think. I don't know how this "layered filesystem" thing operates or is used, so I'm guessing. > and would make it easier to port cachefs to cifs etc. I doubt it. The netfs still has to interact with fscache internally to decide how match netfs files to cache files, which is what most of the interface is about; and to push/pull pages to/from the cache (this bit could possibly be made transparent, but I'm not sure how you'd do it on Linux with the present VM & VFS interfaces). David -- Linux-cachefs mailing list Linux-cachefs@redhat.com http://www.redhat.com/mailman/listinfo/linux-cachefs ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Linux-cachefs] Re: [PATCH] NFS using CacheFS 2004-10-08 11:22 ` Steve Dickson @ 2004-10-11 15:18 ` David Howells -1 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-11 15:18 UTC (permalink / raw) To: Martin Waitz Cc: Linux filesystem caching discussion list, Steve Dickson, nfs, linux-kernel, Clemens Schwaighofer, Alexander Viro > > The 'fscache' flag will be coming along with the nfs4 support, since > > nfs4 mounting code does not have an open (unused) mounting flag.... > > is such a flag even neccessary? > The way I see fscache is that its operations will be no-ops anyway if you > haven't mounted any backing cache. There are two reasons: (1) The fscache middle bit builds up a cookie tree even if there are no backing caches. This means if the cache is added later (NFS root, for example), the netfs then starts caching immediately without requiring a remount. (2) NFS is not currently safe with respect to fscache in the following situation: mount warthog:/warthog/x /x mount warthog:/warthog /y Because NFS ends up with two superblocks for what is one set of filehandles. This means it tries to get the same cookies out of fscache twice, which fscache is ill-equipped to handle. This also introduces unnecessary (IMNSHO) aliasing in each client of inodes and pages both. I have a patch to fix NFS to cope with this by modifying the get_sb() method to allow the filesystem to override the automatic selection of super->s_root as the root dentry. However, Al Viro is leery of the patch for some reason. David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Re: [PATCH] NFS using CacheFS @ 2004-10-11 15:18 ` David Howells 0 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-11 15:18 UTC (permalink / raw) To: Martin Waitz Cc: Linux filesystem caching discussion list, nfs, Alexander Viro, linux-kernel, Steve Dickson, Clemens Schwaighofer > > The 'fscache' flag will be coming along with the nfs4 support, since > > nfs4 mounting code does not have an open (unused) mounting flag.... > > is such a flag even neccessary? > The way I see fscache is that its operations will be no-ops anyway if you > haven't mounted any backing cache. There are two reasons: (1) The fscache middle bit builds up a cookie tree even if there are no backing caches. This means if the cache is added later (NFS root, for example), the netfs then starts caching immediately without requiring a remount. (2) NFS is not currently safe with respect to fscache in the following situation: mount warthog:/warthog/x /x mount warthog:/warthog /y Because NFS ends up with two superblocks for what is one set of filehandles. This means it tries to get the same cookies out of fscache twice, which fscache is ill-equipped to handle. This also introduces unnecessary (IMNSHO) aliasing in each client of inodes and pages both. I have a patch to fix NFS to cope with this by modifying the get_sb() method to allow the filesystem to override the automatic selection of super->s_root as the root dentry. However, Al Viro is leery of the patch for some reason. David -- Linux-cachefs mailing list Linux-cachefs@redhat.com http://www.redhat.com/mailman/listinfo/linux-cachefs ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Linux-cachefs] Re: [PATCH] NFS using CacheFS 2004-10-04 20:45 ` Steve Dickson @ 2004-10-08 9:02 ` David Howells -1 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-08 9:02 UTC (permalink / raw) To: Linux filesystem caching discussion list Cc: Steve Dickson, nfs, linux-kernel, Trond Myklebust > brr :) why is it posix, this is so out of the context for me (as a > user). Is it possible to have a cachefs flag. Would make it more logical. It's a prototype, so Steve's made use of an existing flag for convenience sake. David ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Linux-cachefs] Re: [PATCH] NFS using CacheFS @ 2004-10-08 9:02 ` David Howells 0 siblings, 0 replies; 22+ messages in thread From: David Howells @ 2004-10-08 9:02 UTC (permalink / raw) To: Linux filesystem caching discussion list Cc: Steve Dickson, nfs, linux-kernel, Trond Myklebust > brr :) why is it posix, this is so out of the context for me (as a > user). Is it possible to have a cachefs flag. Would make it more logical. It's a prototype, so Steve's made use of an existing flag for convenience sake. David ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2004-10-13 10:40 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2004-10-04 20:45 [PATCH] NFS using CacheFS Steve Dickson 2004-10-04 20:45 ` Steve Dickson 2004-10-04 21:53 ` Trond Myklebust 2004-10-04 21:53 ` Trond Myklebust 2004-10-05 10:01 ` David Howells 2004-10-05 10:01 ` David Howells 2004-10-08 4:36 ` Clemens Schwaighofer 2004-10-08 4:36 ` Clemens Schwaighofer 2004-10-08 11:22 ` Steve Dickson 2004-10-08 11:22 ` Steve Dickson 2004-10-08 15:39 ` Clemens Schwaighofer 2004-10-11 14:23 ` Martin Waitz 2004-10-11 14:23 ` Martin Waitz 2004-10-11 15:30 ` Trond Myklebust 2004-10-11 15:30 ` Trond Myklebust 2004-10-11 20:31 ` accessing/modifiying the nfs share files as different users bruce 2004-10-13 10:40 ` [Linux-cachefs] Re: [PATCH] NFS using CacheFS David Howells 2004-10-13 10:40 ` David Howells 2004-10-11 15:18 ` [Linux-cachefs] " David Howells 2004-10-11 15:18 ` David Howells 2004-10-08 9:02 ` [Linux-cachefs] " David Howells 2004-10-08 9:02 ` David Howells
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.