[patch 00/11] fuse updates

[patch 00/11] fuse updates

[Miklos Szeredi]

Various fuse bits.  Some of these are in preparation for the writable
mmap support, but do stand on their own.

All this is 2.6.24 material.

Thanks,
Miklos
--

[patch 01/11] fuse: update backing_dev_info congestion state

[Miklos Szeredi]

[-- Attachment #1: fuse_congestion.patch --]
[-- Type: text/plain, Size: 1834 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Set the read and write congestion state if the request queue is close
to blocking, and clear it when it's not.

This prevents unnecessary blocking in readahead and (when writable
mmaps are allowed) writeback.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 17:54:50.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 17:54:53.000000000 +0200
@@ -224,6 +224,10 @@ static void request_end(struct fuse_conn
 			fc->blocked = 0;
 			wake_up_all(&fc->blocked_waitq);
 		}
+		if (fc->num_background == FUSE_CONGESTION_THRESHOLD) {
+			clear_bdi_congested(&fc->bdi, READ);
+			clear_bdi_congested(&fc->bdi, WRITE);
+		}
 		fc->num_background--;
 	}
 	spin_unlock(&fc->lock);
@@ -378,6 +382,10 @@ static void request_send_nowait(struct f
 		fc->num_background++;
 		if (fc->num_background == FUSE_MAX_BACKGROUND)
 			fc->blocked = 1;
+		if (fc->num_background == FUSE_CONGESTION_THRESHOLD) {
+			set_bdi_congested(&fc->bdi, READ);
+			set_bdi_congested(&fc->bdi, WRITE);
+		}
 
 		queue_request(fc, req);
 		spin_unlock(&fc->lock);
Index: linux/fs/fuse/fuse_i.h
===================================================================
--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 17:54:50.000000000 +0200
+++ linux/fs/fuse/fuse_i.h	2007-08-03 17:54:53.000000000 +0200
@@ -20,7 +20,10 @@
 #define FUSE_MAX_PAGES_PER_REQ 32
 
 /** Maximum number of outstanding background requests */
-#define FUSE_MAX_BACKGROUND 10
+#define FUSE_MAX_BACKGROUND 12
+
+/** Congestion starts at 75% of maximum */
+#define FUSE_CONGESTION_THRESHOLD (FUSE_MAX_BACKGROUND * 75 / 100)
 
 /** It could be as large as PATH_MAX, but would that have any uses? */
 #define FUSE_NAME_MAX 1024

--

[patch 02/11] fuse: fix reserved request wake up

[Miklos Szeredi]

[-- Attachment #1: fuse_wake_up_fix.patch --]
[-- Type: text/plain, Size: 2425 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Use wake_up_all instead of wake_up in put_reserved_req(), otherwise it
is possible that the right task is not woken up.

Also create a separate reserved_req_waitq in addition to the
blocked_waitq, since they fulfill totally separate functions.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 17:54:53.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 17:54:54.000000000 +0200
@@ -129,7 +129,7 @@ static struct fuse_req *get_reserved_req
 	struct fuse_file *ff = file->private_data;
 
 	do {
-		wait_event(fc->blocked_waitq, ff->reserved_req);
+		wait_event(fc->reserved_req_waitq, ff->reserved_req);
 		spin_lock(&fc->lock);
 		if (ff->reserved_req) {
 			req = ff->reserved_req;
@@ -155,7 +155,7 @@ static void put_reserved_req(struct fuse
 	fuse_request_init(req);
 	BUG_ON(ff->reserved_req);
 	ff->reserved_req = req;
-	wake_up(&fc->blocked_waitq);
+	wake_up_all(&fc->reserved_req_waitq);
 	spin_unlock(&fc->lock);
 	fput(file);
 }
Index: linux/fs/fuse/fuse_i.h
===================================================================
--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 17:54:53.000000000 +0200
+++ linux/fs/fuse/fuse_i.h	2007-08-03 17:54:54.000000000 +0200
@@ -289,6 +289,9 @@ struct fuse_conn {
 	/** waitq for blocked connection */
 	wait_queue_head_t blocked_waitq;
 
+	/** waitq for reserved requests */
+	wait_queue_head_t reserved_req_waitq;
+
 	/** The next unique request id */
 	u64 reqctr;
 
Index: linux/fs/fuse/inode.c
===================================================================
--- linux.orig/fs/fuse/inode.c	2007-08-03 17:54:50.000000000 +0200
+++ linux/fs/fuse/inode.c	2007-08-03 17:54:54.000000000 +0200
@@ -232,6 +232,7 @@ static void fuse_put_super(struct super_
 	kill_fasync(&fc->fasync, SIGIO, POLL_IN);
 	wake_up_all(&fc->waitq);
 	wake_up_all(&fc->blocked_waitq);
+	wake_up_all(&fc->reserved_req_waitq);
 	mutex_lock(&fuse_mutex);
 	list_del(&fc->entry);
 	fuse_ctl_remove_conn(fc);
@@ -426,6 +427,7 @@ static struct fuse_conn *new_conn(void)
 		atomic_set(&fc->count, 1);
 		init_waitqueue_head(&fc->waitq);
 		init_waitqueue_head(&fc->blocked_waitq);
+		init_waitqueue_head(&fc->reserved_req_waitq);
 		INIT_LIST_HEAD(&fc->pending);
 		INIT_LIST_HEAD(&fc->processing);
 		INIT_LIST_HEAD(&fc->io);

--

[patch 03/11] fuse: add reference counting to fuse_file

[Miklos Szeredi]

[-- Attachment #1: fuse_refcounted_fuse_file.patch --]
[-- Type: text/plain, Size: 9823 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Make lifetime of 'struct fuse_file' independent from 'struct file' by
adding a reference counter and destructor.

This will enable asynchronous page writeback, where it cannot be
guaranteed, that the file is not released while a request with this
file handle is being served.

The actual RELEASE request is only sent when there are no more
references to the fuse_file.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/file.c
===================================================================
--- linux.orig/fs/fuse/file.c	2007-08-03 17:54:49.000000000 +0200
+++ linux/fs/fuse/file.c	2007-08-03 17:54:55.000000000 +0200
@@ -54,6 +54,7 @@ struct fuse_file *fuse_file_alloc(void)
 			kfree(ff);
 			ff = NULL;
 		}
+		atomic_set(&ff->count, 0);
 	}
 	return ff;
 }
@@ -64,6 +65,22 @@ void fuse_file_free(struct fuse_file *ff
 	kfree(ff);
 }
 
+static struct fuse_file *fuse_file_get(struct fuse_file *ff)
+{
+	atomic_inc(&ff->count);
+	return ff;
+}
+
+static void fuse_file_put(struct fuse_file *ff)
+{
+	if (atomic_dec_and_test(&ff->count)) {
+		struct fuse_req *req = ff->reserved_req;
+		struct fuse_conn *fc = get_fuse_conn(req->dentry->d_inode);
+		request_send_background(fc, req);
+		kfree(ff);
+	}
+}
+
 void fuse_finish_open(struct inode *inode, struct file *file,
 		      struct fuse_file *ff, struct fuse_open_out *outarg)
 {
@@ -72,7 +89,7 @@ void fuse_finish_open(struct inode *inod
 	if (!(outarg->open_flags & FOPEN_KEEP_CACHE))
 		invalidate_mapping_pages(inode->i_mapping, 0, -1);
 	ff->fh = outarg->fh;
-	file->private_data = ff;
+	file->private_data = fuse_file_get(ff);
 }
 
 int fuse_open_common(struct inode *inode, struct file *file, int isdir)
@@ -113,8 +130,7 @@ int fuse_open_common(struct inode *inode
 	return err;
 }
 
-struct fuse_req *fuse_release_fill(struct fuse_file *ff, u64 nodeid, int flags,
-				   int opcode)
+void fuse_release_fill(struct fuse_file *ff, u64 nodeid, int flags, int opcode)
 {
 	struct fuse_req *req = ff->reserved_req;
 	struct fuse_release_in *inarg = &req->misc.release_in;
@@ -126,25 +142,24 @@ struct fuse_req *fuse_release_fill(struc
 	req->in.numargs = 1;
 	req->in.args[0].size = sizeof(struct fuse_release_in);
 	req->in.args[0].value = inarg;
-	kfree(ff);
-
-	return req;
 }
 
 int fuse_release_common(struct inode *inode, struct file *file, int isdir)
 {
 	struct fuse_file *ff = file->private_data;
 	if (ff) {
-		struct fuse_conn *fc = get_fuse_conn(inode);
-		struct fuse_req *req;
-
-		req = fuse_release_fill(ff, get_node_id(inode), file->f_flags,
-					isdir ? FUSE_RELEASEDIR : FUSE_RELEASE);
+		fuse_release_fill(ff, get_node_id(inode), file->f_flags,
+				  isdir ? FUSE_RELEASEDIR : FUSE_RELEASE);
 
 		/* Hold vfsmount and dentry until release is finished */
-		req->vfsmount = mntget(file->f_path.mnt);
-		req->dentry = dget(file->f_path.dentry);
-		request_send_background(fc, req);
+		ff->reserved_req->vfsmount = mntget(file->f_path.mnt);
+		ff->reserved_req->dentry = dget(file->f_path.dentry);
+		/*
+		 * Normally this will send the RELEASE request,
+		 * however if some asynchronous READ or WRITE requests
+		 * are outstanding, the sending will be delayed
+		 */
+		fuse_file_put(ff);
 	}
 
 	/* Return value is ignored by VFS */
@@ -264,10 +279,9 @@ static int fuse_fsync(struct file *file,
 	return fuse_fsync_common(file, de, datasync, 0);
 }
 
-void fuse_read_fill(struct fuse_req *req, struct file *file,
+void fuse_read_fill(struct fuse_req *req, struct fuse_file *ff,
 		    struct inode *inode, loff_t pos, size_t count, int opcode)
 {
-	struct fuse_file *ff = file->private_data;
 	struct fuse_read_in *inarg = &req->misc.read_in;
 
 	inarg->fh = ff->fh;
@@ -288,7 +302,8 @@ static size_t fuse_send_read(struct fuse
 			     struct inode *inode, loff_t pos, size_t count)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
-	fuse_read_fill(req, file, inode, pos, count, FUSE_READ);
+	struct fuse_file *ff = file->private_data;
+	fuse_read_fill(req, ff, inode, pos, count, FUSE_READ);
 	request_send(fc, req);
 	return req->out.args[0].size;
 }
@@ -337,20 +352,21 @@ static void fuse_readpages_end(struct fu
 			SetPageError(page);
 		unlock_page(page);
 	}
+	if (req->ff)
+		fuse_file_put(req->ff);
 	fuse_put_request(fc, req);
 }
 
-static void fuse_send_readpages(struct fuse_req *req, struct file *file,
+static void fuse_send_readpages(struct fuse_req *req, struct fuse_file *ff,
 				struct inode *inode)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
 	loff_t pos = page_offset(req->pages[0]);
 	size_t count = req->num_pages << PAGE_CACHE_SHIFT;
 	req->out.page_zeroing = 1;
-	fuse_read_fill(req, file, inode, pos, count, FUSE_READ);
+	fuse_read_fill(req, ff, inode, pos, count, FUSE_READ);
 	if (fc->async_read) {
-		get_file(file);
-		req->file = file;
+		req->ff = fuse_file_get(ff);
 		req->end = fuse_readpages_end;
 		request_send_background(fc, req);
 	} else {
@@ -359,15 +375,15 @@ static void fuse_send_readpages(struct f
 	}
 }
 
-struct fuse_readpages_data {
+struct fuse_fill_data {
 	struct fuse_req *req;
-	struct file *file;
+	struct fuse_file *ff;
 	struct inode *inode;
 };
 
 static int fuse_readpages_fill(void *_data, struct page *page)
 {
-	struct fuse_readpages_data *data = _data;
+	struct fuse_fill_data *data = _data;
 	struct fuse_req *req = data->req;
 	struct inode *inode = data->inode;
 	struct fuse_conn *fc = get_fuse_conn(inode);
@@ -376,7 +392,7 @@ static int fuse_readpages_fill(void *_da
 	    (req->num_pages == FUSE_MAX_PAGES_PER_REQ ||
 	     (req->num_pages + 1) * PAGE_CACHE_SIZE > fc->max_read ||
 	     req->pages[req->num_pages - 1]->index + 1 != page->index)) {
-		fuse_send_readpages(req, data->file, inode);
+		fuse_send_readpages(req, data->ff, inode);
 		data->req = req = fuse_get_req(fc);
 		if (IS_ERR(req)) {
 			unlock_page(page);
@@ -393,14 +409,14 @@ static int fuse_readpages(struct file *f
 {
 	struct inode *inode = mapping->host;
 	struct fuse_conn *fc = get_fuse_conn(inode);
-	struct fuse_readpages_data data;
+	struct fuse_fill_data data;
 	int err;
 
 	err = -EIO;
 	if (is_bad_inode(inode))
 		goto out;
 
-	data.file = file;
+	data.ff = file->private_data;
 	data.inode = inode;
 	data.req = fuse_get_req(fc);
 	err = PTR_ERR(data.req);
@@ -410,7 +426,7 @@ static int fuse_readpages(struct file *f
 	err = read_cache_pages(mapping, pages, fuse_readpages_fill, &data);
 	if (!err) {
 		if (data.req->num_pages)
-			fuse_send_readpages(data.req, file, inode);
+			fuse_send_readpages(data.req, data.ff, inode);
 		else
 			fuse_put_request(fc, data.req);
 	}
Index: linux/fs/fuse/fuse_i.h
===================================================================
--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 17:54:54.000000000 +0200
+++ linux/fs/fuse/fuse_i.h	2007-08-03 17:54:55.000000000 +0200
@@ -72,6 +72,9 @@ struct fuse_file {
 
 	/** File handle used by userspace */
 	u64 fh;
+
+	/** Refcount */
+	atomic_t count;
 };
 
 /** One input argument of a request */
@@ -216,7 +219,7 @@ struct fuse_req {
 	unsigned page_offset;
 
 	/** File used in the request (or NULL) */
-	struct file *file;
+	struct fuse_file *ff;
 
 	/** vfsmount used in release */
 	struct vfsmount *vfsmount;
@@ -420,7 +423,7 @@ void fuse_send_forget(struct fuse_conn *
 /**
  * Initialize READ or READDIR request
  */
-void fuse_read_fill(struct fuse_req *req, struct file *file,
+void fuse_read_fill(struct fuse_req *req, struct fuse_file *ff,
 		    struct inode *inode, loff_t pos, size_t count, int opcode);
 
 /**
@@ -433,9 +436,9 @@ void fuse_file_free(struct fuse_file *ff
 void fuse_finish_open(struct inode *inode, struct file *file,
 		      struct fuse_file *ff, struct fuse_open_out *outarg);
 
-/** */
-struct fuse_req *fuse_release_fill(struct fuse_file *ff, u64 nodeid, int flags,
-				   int opcode);
+/** Fill in ff->reserved_req with a RELEASE request */
+void fuse_release_fill(struct fuse_file *ff, u64 nodeid, int flags, int opcode);
+
 /**
  * Send RELEASE or RELEASEDIR request
  */
Index: linux/fs/fuse/dir.c
===================================================================
--- linux.orig/fs/fuse/dir.c	2007-08-03 17:54:49.000000000 +0200
+++ linux/fs/fuse/dir.c	2007-08-03 17:54:55.000000000 +0200
@@ -288,12 +288,11 @@ static struct dentry *fuse_lookup(struct
 static void fuse_sync_release(struct fuse_conn *fc, struct fuse_file *ff,
 			      u64 nodeid, int flags)
 {
-	struct fuse_req *req;
-
-	req = fuse_release_fill(ff, nodeid, flags, FUSE_RELEASE);
-	req->force = 1;
-	request_send(fc, req);
-	fuse_put_request(fc, req);
+	fuse_release_fill(ff, nodeid, flags, FUSE_RELEASE);
+	ff->reserved_req->force = 1;
+	request_send(fc, ff->reserved_req);
+	fuse_put_request(fc, ff->reserved_req);
+	kfree(ff);
 }
 
 /*
@@ -859,6 +858,7 @@ static int fuse_readdir(struct file *fil
 	struct page *page;
 	struct inode *inode = file->f_path.dentry->d_inode;
 	struct fuse_conn *fc = get_fuse_conn(inode);
+	struct fuse_file *ff = file->private_data;
 	struct fuse_req *req;
 
 	if (is_bad_inode(inode))
@@ -875,7 +875,7 @@ static int fuse_readdir(struct file *fil
 	}
 	req->num_pages = 1;
 	req->pages[0] = page;
-	fuse_read_fill(req, file, inode, file->f_pos, PAGE_SIZE, FUSE_READDIR);
+	fuse_read_fill(req, ff, inode, file->f_pos, PAGE_SIZE, FUSE_READDIR);
 	request_send(fc, req);
 	nbytes = req->out.args[0].size;
 	err = req->out.h.error;
Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 17:54:54.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 17:54:55.000000000 +0200
@@ -233,8 +233,6 @@ static void request_end(struct fuse_conn
 	spin_unlock(&fc->lock);
 	dput(req->dentry);
 	mntput(req->vfsmount);
-	if (req->file)
-		fput(req->file);
 	wake_up(&req->waitq);
 	if (end)
 		end(fc, req);

--

[patch 04/11] fuse: truncate on spontaneous size change

[Miklos Szeredi]

[-- Attachment #1: fuse_truncate_cleanup.patch --]
[-- Type: text/plain, Size: 4479 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Memory mappings were only truncated on an explicit truncate, but not
when the file size was changed externally.

Fix this by moving the truncation code from fuse_setattr to
fuse_change_attributes.

Yes, there are races between write and and external truncation, but we
can't really do anything about them.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dir.c
===================================================================
--- linux.orig/fs/fuse/dir.c	2007-08-03 17:54:55.000000000 +0200
+++ linux/fs/fuse/dir.c	2007-08-03 17:54:55.000000000 +0200
@@ -980,23 +980,6 @@ static void iattr_to_fattr(struct iattr 
 	}
 }
 
-static void fuse_vmtruncate(struct inode *inode, loff_t offset)
-{
-	struct fuse_conn *fc = get_fuse_conn(inode);
-	int need_trunc;
-
-	spin_lock(&fc->lock);
-	need_trunc = inode->i_size > offset;
-	i_size_write(inode, offset);
-	spin_unlock(&fc->lock);
-
-	if (need_trunc) {
-		struct address_space *mapping = inode->i_mapping;
-		unmap_mapping_range(mapping, offset + PAGE_SIZE - 1, 0, 1);
-		truncate_inode_pages(mapping, offset);
-	}
-}
-
 /*
  * Set attributes, and at the same time refresh them.
  *
@@ -1014,7 +997,6 @@ static int fuse_setattr(struct dentry *e
 	struct fuse_setattr_in inarg;
 	struct fuse_attr_out outarg;
 	int err;
-	int is_truncate = 0;
 
 	if (fc->flags & FUSE_DEFAULT_PERMISSIONS) {
 		err = inode_change_ok(inode, attr);
@@ -1024,7 +1006,6 @@ static int fuse_setattr(struct dentry *e
 
 	if (attr->ia_valid & ATTR_SIZE) {
 		unsigned long limit;
-		is_truncate = 1;
 		if (IS_SWAPFILE(inode))
 			return -ETXTBSY;
 		limit = current->signal->rlim[RLIMIT_FSIZE].rlim_cur;
@@ -1051,21 +1032,20 @@ static int fuse_setattr(struct dentry *e
 	request_send(fc, req);
 	err = req->out.h.error;
 	fuse_put_request(fc, req);
-	if (!err) {
-		if ((inode->i_mode ^ outarg.attr.mode) & S_IFMT) {
-			make_bad_inode(inode);
-			err = -EIO;
-		} else {
-			if (is_truncate)
-				fuse_vmtruncate(inode, outarg.attr.size);
-			fuse_change_attributes(inode, &outarg.attr);
-			fi->i_time = time_to_jiffies(outarg.attr_valid,
-						     outarg.attr_valid_nsec);
-		}
-	} else if (err == -EINTR)
-		fuse_invalidate_attr(inode);
+	if (err) {
+		if (err == -EINTR)
+			fuse_invalidate_attr(inode);
+		return err;
+	}
 
-	return err;
+	if ((inode->i_mode ^ outarg.attr.mode) & S_IFMT) {
+		make_bad_inode(inode);
+		return -EIO;
+	}
+
+	fuse_change_attributes(inode, &outarg.attr);
+	fi->i_time = time_to_jiffies(outarg.attr_valid, outarg.attr_valid_nsec);
+	return 0;
 }
 
 static int fuse_getattr(struct vfsmount *mnt, struct dentry *entry,
Index: linux/fs/fuse/inode.c
===================================================================
--- linux.orig/fs/fuse/inode.c	2007-08-03 17:54:54.000000000 +0200
+++ linux/fs/fuse/inode.c	2007-08-03 17:54:55.000000000 +0200
@@ -109,20 +109,24 @@ static int fuse_remount_fs(struct super_
 	return 0;
 }
 
+static void fuse_truncate(struct address_space *mapping, loff_t offset)
+{
+	/* See vmtruncate() */
+	unmap_mapping_range(mapping, offset + PAGE_SIZE - 1, 0, 1);
+	truncate_inode_pages(mapping, offset);
+	unmap_mapping_range(mapping, offset + PAGE_SIZE - 1, 0, 1);
+}
+
 void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
-	if (S_ISREG(inode->i_mode) && i_size_read(inode) != attr->size)
-		invalidate_mapping_pages(inode->i_mapping, 0, -1);
+	loff_t oldsize;
 
 	inode->i_ino     = attr->ino;
 	inode->i_mode    = (inode->i_mode & S_IFMT) + (attr->mode & 07777);
 	inode->i_nlink   = attr->nlink;
 	inode->i_uid     = attr->uid;
 	inode->i_gid     = attr->gid;
-	spin_lock(&fc->lock);
-	i_size_write(inode, attr->size);
-	spin_unlock(&fc->lock);
 	inode->i_blocks  = attr->blocks;
 	inode->i_atime.tv_sec   = attr->atime;
 	inode->i_atime.tv_nsec  = attr->atimensec;
@@ -130,6 +134,17 @@ void fuse_change_attributes(struct inode
 	inode->i_mtime.tv_nsec  = attr->mtimensec;
 	inode->i_ctime.tv_sec   = attr->ctime;
 	inode->i_ctime.tv_nsec  = attr->ctimensec;
+
+	spin_lock(&fc->lock);
+	oldsize = inode->i_size;
+	i_size_write(inode, attr->size);
+	spin_unlock(&fc->lock);
+
+	if (S_ISREG(inode->i_mode) && oldsize != attr->size) {
+		if (attr->size < oldsize)
+			fuse_truncate(inode->i_mapping, attr->size);
+		invalidate_mapping_pages(inode->i_mapping, 0, -1);
+	}
 }
 
 static void fuse_init_inode(struct inode *inode, struct fuse_attr *attr)

--

[patch 05/11] fuse: fix page invalidation

[Miklos Szeredi]

[-- Attachment #1: fuse_invalidate_fix.patch --]
[-- Type: text/plain, Size: 1634 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Other than truncate, there are two cases, when fuse tries to get rid
of cached pages:

 a) in open, if KEEP_CACHE flag is not set
 b) in getattr, if file size changed spontaneously

Until now invalidate_mapping_pages() were used, which didn't get rid
of mapped pages.  This is wrong, and becomes more wrong as dirty pages
are introduced.  So instead properly invalidate all pages with
invalidate_inode_pages2().

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/file.c
===================================================================
--- linux.orig/fs/fuse/file.c	2007-08-03 18:36:05.000000000 +0200
+++ linux/fs/fuse/file.c	2007-08-03 18:36:13.000000000 +0200
@@ -87,7 +87,7 @@ void fuse_finish_open(struct inode *inod
 	if (outarg->open_flags & FOPEN_DIRECT_IO)
 		file->f_op = &fuse_direct_io_file_operations;
 	if (!(outarg->open_flags & FOPEN_KEEP_CACHE))
-		invalidate_mapping_pages(inode->i_mapping, 0, -1);
+		invalidate_inode_pages2(inode->i_mapping);
 	ff->fh = outarg->fh;
 	file->private_data = fuse_file_get(ff);
 }
Index: linux/fs/fuse/inode.c
===================================================================
--- linux.orig/fs/fuse/inode.c	2007-08-03 18:36:05.000000000 +0200
+++ linux/fs/fuse/inode.c	2007-08-03 18:36:13.000000000 +0200
@@ -143,7 +143,7 @@ void fuse_change_attributes(struct inode
 	if (S_ISREG(inode->i_mode) && oldsize != attr->size) {
 		if (attr->size < oldsize)
 			fuse_truncate(inode->i_mapping, attr->size);
-		invalidate_mapping_pages(inode->i_mapping, 0, -1);
+		invalidate_inode_pages2(inode->i_mapping);
 	}
 }
 

--

[patch 06/11] fuse: set i_nlink to sane value after mount

[Miklos Szeredi]

[-- Attachment #1: fuse_i_nlink_fix.patch --]
[-- Type: text/plain, Size: 701 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Aufs seems to depend on a positive i_nlink value.  So fill in a dummy
but sane value for the root inode at mount time.

The inode attributes are refreshed with the correct values at the
first opportunity.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/inode.c
===================================================================
--- linux.orig/fs/fuse/inode.c	2007-08-03 18:36:13.000000000 +0200
+++ linux/fs/fuse/inode.c	2007-08-03 18:36:14.000000000 +0200
@@ -480,6 +480,7 @@ static struct inode *get_root_inode(stru
 
 	attr.mode = mode;
 	attr.ino = FUSE_ROOT_ID;
+	attr.nlink = 1;
 	return fuse_iget(sb, 1, 0, &attr);
 }
 

--

[patch 07/11] fuse: refresh stale attributes in fuse_permission()

[Miklos Szeredi]

[-- Attachment #1: fuse_permission_fix.patch --]
[-- Type: text/plain, Size: 4851 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

fuse_permission() didn't refresh inode attributes before using them,
even if the validity has already expired.

Thanks to Junjiro Okajima for spotting this.

Also remove some old code to unconditionally refresh the attributes on
the root inode.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dir.c
===================================================================
--- linux.orig/fs/fuse/dir.c	2007-08-03 18:36:01.000000000 +0200
+++ linux/fs/fuse/dir.c	2007-08-03 18:36:15.000000000 +0200
@@ -663,7 +663,7 @@ static int fuse_link(struct dentry *entr
 	return err;
 }
 
-int fuse_do_getattr(struct inode *inode)
+static int fuse_do_getattr(struct inode *inode)
 {
 	int err;
 	struct fuse_attr_out arg;
@@ -723,30 +723,6 @@ static int fuse_allow_task(struct fuse_c
 	return 0;
 }
 
-/*
- * Check whether the inode attributes are still valid
- *
- * If the attribute validity timeout has expired, then fetch the fresh
- * attributes with a 'getattr' request
- *
- * I'm not sure why cached attributes are never returned for the root
- * inode, this is probably being too cautious.
- */
-static int fuse_revalidate(struct dentry *entry)
-{
-	struct inode *inode = entry->d_inode;
-	struct fuse_inode *fi = get_fuse_inode(inode);
-	struct fuse_conn *fc = get_fuse_conn(inode);
-
-	if (!fuse_allow_task(fc, current))
-		return -EACCES;
-	if (get_node_id(inode) != FUSE_ROOT_ID &&
-	    fi->i_time >= get_jiffies_64())
-		return 0;
-
-	return fuse_do_getattr(inode);
-}
-
 static int fuse_access(struct inode *inode, int mask)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
@@ -794,16 +770,33 @@ static int fuse_access(struct inode *ino
 static int fuse_permission(struct inode *inode, int mask, struct nameidata *nd)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
+	struct fuse_inode *fi = get_fuse_inode(inode);
+	bool refreshed = false;
+	int err = 0;
 
 	if (!fuse_allow_task(fc, current))
 		return -EACCES;
-	else if (fc->flags & FUSE_DEFAULT_PERMISSIONS) {
+
+	/*
+	 * If attributes are needed, but are stale, refresh them
+	 * before proceeding
+	 */
+	if (((fc->flags & FUSE_DEFAULT_PERMISSIONS) || (mask & MAY_EXEC)) &&
+	    fi->i_time < get_jiffies_64()) {
+		err = fuse_do_getattr(inode);
+		if (err)
+			return err;
+
+		refreshed = true;
+	}
+
+	if (fc->flags & FUSE_DEFAULT_PERMISSIONS) {
 		int err = generic_permission(inode, mask, NULL);
 
 		/* If permission is denied, try to refresh file
 		   attributes.  This is also needed, because the root
 		   node will at first have no permissions */
-		if (err == -EACCES) {
+		if (err == -EACCES && !refreshed) {
 		 	err = fuse_do_getattr(inode);
 			if (!err)
 				err = generic_permission(inode, mask, NULL);
@@ -814,7 +807,6 @@ static int fuse_permission(struct inode 
 		   noticed immediately, only after the attribute
 		   timeout has expired */
 
-		return err;
 	} else {
 		int mode = inode->i_mode;
 		if ((mask & MAY_EXEC) && !S_ISDIR(mode) && !(mode & S_IXUGO))
@@ -822,8 +814,8 @@ static int fuse_permission(struct inode 
 
 		if (nd && (nd->flags & (LOOKUP_ACCESS | LOOKUP_CHDIR)))
 			return fuse_access(inode, mask);
-		return 0;
 	}
+	return err;
 }
 
 static int parse_dirfile(char *buf, size_t nbytes, struct file *file,
@@ -1052,7 +1044,16 @@ static int fuse_getattr(struct vfsmount 
 			struct kstat *stat)
 {
 	struct inode *inode = entry->d_inode;
-	int err = fuse_revalidate(entry);
+	struct fuse_inode *fi = get_fuse_inode(inode);
+	struct fuse_conn *fc = get_fuse_conn(inode);
+	int err = 0;
+
+	if (!fuse_allow_task(fc, current))
+		return -EACCES;
+
+	if (fi->i_time < get_jiffies_64())
+		err = fuse_do_getattr(inode);
+
 	if (!err)
 		generic_fillattr(inode, stat);
 
Index: linux/fs/fuse/file.c
===================================================================
--- linux.orig/fs/fuse/file.c	2007-08-03 18:36:13.000000000 +0200
+++ linux/fs/fuse/file.c	2007-08-03 18:36:15.000000000 +0200
@@ -106,14 +106,6 @@ int fuse_open_common(struct inode *inode
 	if (err)
 		return err;
 
-	/* If opening the root node, no lookup has been performed on
-	   it, so the attributes must be refreshed */
-	if (get_node_id(inode) == FUSE_ROOT_ID) {
-		err = fuse_do_getattr(inode);
-		if (err)
-		 	return err;
-	}
-
 	ff = fuse_file_alloc();
 	if (!ff)
 		return -ENOMEM;
Index: linux/fs/fuse/fuse_i.h
===================================================================
--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 18:36:05.000000000 +0200
+++ linux/fs/fuse/fuse_i.h	2007-08-03 18:36:15.000000000 +0200
@@ -533,11 +533,6 @@ void request_send_background(struct fuse
 void fuse_abort_conn(struct fuse_conn *fc);
 
 /**
- * Get the attributes of a file
- */
-int fuse_do_getattr(struct inode *inode);
-
-/**
  * Invalidate inode attributes
  */
 void fuse_invalidate_attr(struct inode *inode);

--

[patch 08/11] fuse: fix permission checking on sticky directories

[Miklos Szeredi]

[-- Attachment #1: fuse_sticky_fix.patch --]
[-- Type: text/plain, Size: 2659 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

The VFS checks sticky bits on the parent directory even if the
filesystem defines it's own ->permission().  In some situations
(sshfs, mountlo, etc) the user does have permission to delete a file
even if the attribute based checking would not allow it.

So work around this by storing the permission bits separately and
returning them in stat(), but cutting the permission bits off from
inode->i_mode.

This is slightly hackish, but it's probably not worth it to add new
infrastructure in VFS and a slight performance penalty for all
filesystems, just for the sake of fuse.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dir.c
===================================================================
--- linux.orig/fs/fuse/dir.c	2007-08-03 18:36:15.000000000 +0200
+++ linux/fs/fuse/dir.c	2007-08-03 18:36:17.000000000 +0200
@@ -1054,8 +1054,10 @@ static int fuse_getattr(struct vfsmount 
 	if (fi->i_time < get_jiffies_64())
 		err = fuse_do_getattr(inode);
 
-	if (!err)
+	if (!err) {
 		generic_fillattr(inode, stat);
+		stat->mode = fi->orig_i_mode;
+	}
 
 	return err;
 }
Index: linux/fs/fuse/fuse_i.h
===================================================================
--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 18:36:15.000000000 +0200
+++ linux/fs/fuse/fuse_i.h	2007-08-03 18:36:17.000000000 +0200
@@ -63,6 +63,10 @@ struct fuse_inode {
 
 	/** Time in jiffies until the file attributes are valid */
 	u64 i_time;
+
+	/** The sticky bit in inode->i_mode may have been remved, so
+	    preserve the original mode */
+	mode_t orig_i_mode;
 };
 
 /** FUSE specific file data */
Index: linux/fs/fuse/inode.c
===================================================================
--- linux.orig/fs/fuse/inode.c	2007-08-03 18:36:14.000000000 +0200
+++ linux/fs/fuse/inode.c	2007-08-03 18:36:37.000000000 +0200
@@ -120,10 +120,18 @@ static void fuse_truncate(struct address
 void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr)
 {
 	struct fuse_conn *fc = get_fuse_conn(inode);
+	struct fuse_inode *fi = get_fuse_inode(inode);
 	loff_t oldsize;
 
 	inode->i_ino     = attr->ino;
-	inode->i_mode    = (inode->i_mode & S_IFMT) + (attr->mode & 07777);
+	fi->orig_i_mode  = (inode->i_mode & S_IFMT) + (attr->mode & 07777);
+	/*
+	 * Don't set the mode bits in i_mode, unless we want the VFS
+	 * to check permissions.  This prevents failures due to the
+	 * sticky bit check in may_delete().
+	 */
+	if (fc->flags & FUSE_DEFAULT_PERMISSIONS)
+		inode->i_mode = fi->orig_i_mode;
 	inode->i_nlink   = attr->nlink;
 	inode->i_uid     = attr->uid;
 	inode->i_gid     = attr->gid;

--

[patch 09/11] fuse: cleanup in release

[Miklos Szeredi]

[-- Attachment #1: fuse_release_end.patch --]
[-- Type: text/plain, Size: 1390 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Move dput/mntput pair from request_end() to fuse_release_end(),
because there's no other place they are used.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 18:36:01.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 18:37:24.000000000 +0200
@@ -231,8 +231,6 @@ static void request_end(struct fuse_conn
 		fc->num_background--;
 	}
 	spin_unlock(&fc->lock);
-	dput(req->dentry);
-	mntput(req->vfsmount);
 	wake_up(&req->waitq);
 	if (end)
 		end(fc, req);
Index: linux/fs/fuse/file.c
===================================================================
--- linux.orig/fs/fuse/file.c	2007-08-03 18:36:15.000000000 +0200
+++ linux/fs/fuse/file.c	2007-08-03 18:37:24.000000000 +0200
@@ -71,11 +71,19 @@ static struct fuse_file *fuse_file_get(s
 	return ff;
 }
 
+static void fuse_release_end(struct fuse_conn *fc, struct fuse_req *req)
+{
+	dput(req->dentry);
+	mntput(req->vfsmount);
+	fuse_put_request(fc, req);
+}
+
 static void fuse_file_put(struct fuse_file *ff)
 {
 	if (atomic_dec_and_test(&ff->count)) {
 		struct fuse_req *req = ff->reserved_req;
 		struct fuse_conn *fc = get_fuse_conn(req->dentry->d_inode);
+		req->end = fuse_release_end;
 		request_send_background(fc, req);
 		kfree(ff);
 	}

--

[patch 10/11] fuse: no abort on interrupt

[Miklos Szeredi]

[-- Attachment #1: fuse_kill_no_abort.patch --]
[-- Type: text/plain, Size: 2514 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Don't set 'aborted' flag on a request if it's interrupted.  We have to
wait for the answer anyway, and this would only a very little time
while copying the reply.

This means, that write() on the fuse device will not return -ENOENT
during normal operation, only if the filesystem is aborted by a forced
umount or through the fusectl interface.

This could simplify userspace code somewhat when backward
compatibility with earlier kernel versions is not required.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 18:37:24.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 18:37:25.000000000 +0200
@@ -273,28 +273,41 @@ static void request_wait_answer(struct f
 			queue_interrupt(fc, req);
 	}
 
-	if (req->force) {
-		spin_unlock(&fc->lock);
-		wait_event(req->waitq, req->state == FUSE_REQ_FINISHED);
-		spin_lock(&fc->lock);
-	} else {
+	if (!req->force) {
 		sigset_t oldset;
 
 		/* Only fatal signals may interrupt this */
 		block_sigs(&oldset);
 		wait_answer_interruptible(fc, req);
 		restore_sigs(&oldset);
+
+		if (req->aborted)
+			goto aborted;
+		if (req->state == FUSE_REQ_FINISHED)
+			return;
+
+		/* Request is not yet in userspace, bail out */
+		if (req->state == FUSE_REQ_PENDING) {
+			list_del(&req->list);
+			__fuse_put_request(req);
+			req->out.h.error = -EINTR;
+			return;
+		}
 	}
 
-	if (req->aborted)
-		goto aborted;
-	if (req->state == FUSE_REQ_FINISHED)
- 		return;
+	/*
+	 * Either request is already in userspace, or it was forced.
+	 * Wait it out.
+	 */
+	spin_unlock(&fc->lock);
+	wait_event(req->waitq, req->state == FUSE_REQ_FINISHED);
+	spin_lock(&fc->lock);
 
-	req->out.h.error = -EINTR;
-	req->aborted = 1;
+	if (!req->aborted)
+		return;
 
  aborted:
+	BUG_ON(req->state != FUSE_REQ_FINISHED);
 	if (req->locked) {
 		/* This is uninterruptible sleep, because data is
 		   being copied to/from the buffers of req.  During
@@ -305,14 +318,6 @@ static void request_wait_answer(struct f
 		wait_event(req->waitq, !req->locked);
 		spin_lock(&fc->lock);
 	}
-	if (req->state == FUSE_REQ_PENDING) {
-		list_del(&req->list);
-		__fuse_put_request(req);
-	} else if (req->state == FUSE_REQ_SENT) {
-		spin_unlock(&fc->lock);
-		wait_event(req->waitq, req->state == FUSE_REQ_FINISHED);
-		spin_lock(&fc->lock);
-	}
 }
 
 static unsigned len_args(unsigned numargs, struct fuse_arg *args)

--

[patch 11/11] fuse: no ENOENT from fuse device read

[Miklos Szeredi]

[-- Attachment #1: fuse_read_no_enoent.patch --]
[-- Type: text/plain, Size: 990 bytes --]

From: Miklos Szeredi <mszeredi@suse.cz>

Don't return -ENOENT for a read() on the fuse device when the request
was aborted.  Instead return -ENODEV, meaning the filesystem has been
force-umounted or aborted.

Previously ENOENT meant that the request was interrupted, but now the
'aborted' flag is not set in case of interrupts.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---

Index: linux/fs/fuse/dev.c
===================================================================
--- linux.orig/fs/fuse/dev.c	2007-08-03 18:40:17.000000000 +0200
+++ linux/fs/fuse/dev.c	2007-08-03 18:40:17.000000000 +0200
@@ -747,11 +747,12 @@ static ssize_t fuse_dev_read(struct kioc
 	fuse_copy_finish(&cs);
 	spin_lock(&fc->lock);
 	req->locked = 0;
-	if (!err && req->aborted)
-		err = -ENOENT;
+	if (req->aborted) {
+		request_end(fc, req);
+		return -ENODEV;
+	}
 	if (err) {
-		if (!req->aborted)
-			req->out.h.error = -EIO;
+		req->out.h.error = -EIO;
 		request_end(fc, req);
 		return err;
 	}

--

Re: [patch 08/11] fuse: fix permission checking on sticky directories

[Jan Engelhardt]

On Aug 3 2007 19:44, Miklos Szeredi wrote:
>--- linux.orig/fs/fuse/fuse_i.h	2007-08-03 18:36:15.000000000 +0200
>+++ linux/fs/fuse/fuse_i.h	2007-08-03 18:36:17.000000000 +0200
>@@ -63,6 +63,10 @@ struct fuse_inode {
> 
> 	/** Time in jiffies until the file attributes are valid */
> 	u64 i_time;
>+
>+	/** The sticky bit in inode->i_mode may have been remved, so
                                                             ^
                                            may have been removed.

>+	    preserve the original mode */
>+	mode_t orig_i_mode;
> };
> 
> /** FUSE specific file data */
>Index: linux/fs/fuse/inode.c
>===================================================================
>--- linux.orig/fs/fuse/inode.c	2007-08-03 18:36:14.000000000 +0200
>+++ linux/fs/fuse/inode.c	2007-08-03 18:36:37.000000000 +0200
>@@ -120,10 +120,18 @@ static void fuse_truncate(struct address
> void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr)
> {
> 	struct fuse_conn *fc = get_fuse_conn(inode);
>+	struct fuse_inode *fi = get_fuse_inode(inode);
> 	loff_t oldsize;
> 
> 	inode->i_ino     = attr->ino;
>-	inode->i_mode    = (inode->i_mode & S_IFMT) + (attr->mode & 07777);
>+	fi->orig_i_mode  = (inode->i_mode & S_IFMT) + (attr->mode & 07777);

Although it does not matter in this case, | is safer (and perhaps faster?).



	Jan
-- 

Re: [patch 03/11] fuse: add reference counting to fuse_file

[Ingo Oeser]

Hi Miklos,

On Friday 03 August 2007, Miklos Szeredi wrote:
> From: Miklos Szeredi <mszeredi@suse.cz>
> 
> Make lifetime of 'struct fuse_file' independent from 'struct file' by
> adding a reference counter and destructor.

What about using krefs to implement that?

see include/linux/kref.h
and lib/kref.c

Just embed that "struct kref" inside your struct fuse_file

struct fuse_file {
...
	struct kref ref;
...
}

init in struct fuse_file *fuse_file_alloc(void) where you added the counter.
...
kref_init(&ff->ref);
...

and implement the release function like:

static void fuse_file_release(struct kref *ff_ref)
{
	struct fuse_file *ff = container_of(ff_ref, struct fuse_file, ref);
        struct fuse_req *req = ff->reserved_req;
        struct fuse_conn *fc = get_fuse_conn(req->dentry->d_inode);
        request_send_background(fc, req);
        kfree(ff);
}

This will also fix the missing smp_barriers, is very simple, saves code,
makes your life easier and is a well known known kernel infrastructure :-)

BTW: FUSE rocks! :-)

You can add my "Signed-off-by: Ingo Oeser <ioe-lkml@rameria.de>",
if you want to use that suggestion.

Best Regards

Ingo Oeser

Re: [patch 03/11] fuse: add reference counting to fuse_file

[Miklos Szeredi]

> On Friday 03 August 2007, Miklos Szeredi wrote:
> > From: Miklos Szeredi <mszeredi@suse.cz>
> > 
> > Make lifetime of 'struct fuse_file' independent from 'struct file' by
> > adding a reference counter and destructor.
> 
> What about using krefs to implement that?

Thanks for the suggestion.  I'm aware of kref, but I don't see the
huge advantage over using the raw atomic interface.

> This will also fix the missing smp_barriers,

Why are the barriers needed?  The documentation doesn't say anything
about that.

> is very simple, saves code, makes your life easier and is a well
> known known kernel infrastructure :-)

I think it saves one line, and adds another, for a total of zero lines
saved ;)

Thanks,
Miklos