All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH -v2 0/6] fuse: make maximum read/write request size tunable
@ 2012-07-19 12:48 Mitsuo Hayasaka
  2012-07-19 12:49 ` [PATCH -v2 1/6] pipe: make the maximum pipe size referable from kernel module Mitsuo Hayasaka
                   ` (5 more replies)
  0 siblings, 6 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:48 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc, yrl.pp-manager.tt

Hi,

This patch series make maximum read/write request size tunable in FUSE.
Currently, it is limited to FUSE_MAX_PAGES_PER_REQ which is equal
to 32 pages. It is required to change it in order to improve the
throughput since optimized value depends on various factors such
as type and version of local filesystems used and HW specs, etc.

In addition, recently FUSE is widely used as a gateway to connect
cloud storage services and distributed filesystems. Larger data
might be stored in them over networking via FUSE and the overhead
might affect the throughput.

It seems there were many requests to increase FUSE_MAX_PAGES_PER_REQ
to improve the throughput, as follows.

http://sourceforge.net/mailarchive/forum.php?thread_name=4FC2F7A1.4010609%40gmail.com&forum_name=fuse-devel

http://old.nabble.com/-Fuse-2.8--big_write-option---%3E-128kb-write-syscall-...-howto-set-higher-value-td22292589.html

http://old.nabble.com/Block-size-%3E128k-td18675772.html

These discussions mention how to change both FUSE kernel and libfuse
sources such as FUSE_MAX_PAGES_PER_REQ and MIN_BUFSIZE, but the
changed and increased values have not been default yet. We guess this
is because it will be applied to the FUSE filesystems that do not need
the increased value.

One of the ways to solve this is to make them tunable.
In this series, the new sysfs parameter max_pages_per_req is introduced.
It limits the maximum read/write size in fuse request and it can be
changed to arbitrary number between 32 pages and nr_pages equivalent to
the maximum pipe size. When the max_read/max_write mount option is
specified, FUSE request size is set per mount. (The size is rounded-up
to page size and limited up to max_pages_per_req.)

We think the sysfs parameter control is required, as follows.

- The libfuse should change the current MIN_BUFSIZE limitation according
  to this value. If not, The libfuse must always set it to the maximum
  request limit (= [nr_pages (equivalent to pipe_max_size) * 4KB + 0x1000]),
  which leads to waste of memory.

- It is easy to find and set it to the optimized value in order to
  improve the throughput.

The 32 pages are set by default and the minimum value. The upper limit
is the number of pages equivalent to the maximum pipe size that can
be changed by only privileged user. So, we can flexibly set it to the
optimized value considering the system configuration. 

Also, the patch set for libfuse to change current MIN_BUFSIZE limitation
according to the sysfs parameter will be sent soon.


* Performance example

We evaluated the performance improvement due to this patch series.
FUSE filesystems are mounted on tmpfs and we measured the read/write
throughput using 512MB random data.

The results of average read/write throughtput are shown as follows.
 - we measured 10 times throughput for read and write operations,
   and calculated their average.
 - the results in 512 and 1024 pages are measured after changing and
   increasing both the maximum pipe size via /proc/sys/fs/pipe-max-size
   and the max_pages_per_req.

** write

For without direct_io option,
# of pages   |original(32)|tuning(32)|(64) |(128)|(256)|(512)|(1024)
--------------------------------------------------------------------
thruput(MB/s)|305.4       |303.9     |364.6|414.4|441.5|442.4|437.4


For with direct_io option,
# of pages   |original(32)|tuning(32)|(64) |(128)|(256)|(512)|(1024)
---------------------------------------------------------------------
thruput(MB/s)|391.6       |387.7     |502.6|595.4|675.7|762.4|743.9


** read

For without direct_io option, there is no deference between
original 32 pages and tuning patches since the read request size
are not changed even if changing the sysfs parameter.


For with direct_io option,
# of pages   |original(32)|tuning(32)|(64) |(128)|(256)|(512)|(1024)
---------------------------------------------------------------------
thruput(MB/s)|484.6       |485.1     |567.7|611.9|653.5|794.5|788.2


 From these evaluations, this patch series can improve the
performance with an increase of the sysfs parameter. In
particular, the read/write throughput with direct_io achieves
a high improvement. However, it is clear that the results for
1024 pages do not always lead to the highest improvement.
These are just an exmaple and the results may be changed in
different systems. Therefore, we think a tunable functionality
of read/write request size is useful.

Changed in v2:
 - add a functionality to get the maximum pipe size from kernel
   module.
 - change the upper limit of fuse request size from 256 to
   nr_pages equivalent to the maximum pipe size.
 - revise the documentation in /Documentation/filesystems/
   fuse.txt

Thanks,

---

Mitsuo Hayasaka (6):
      fuse: add documentation of sysfs parameter to limit maximum fuse request size
      fuse: set default global limit considering tunable request size
      fuse: add a sysfs parameter to control the maximum request size
      fuse: remove cache for fuse request allocation
      fuse: make the maximum read/write request size tunable
      pipe: make the maximum pipe size referable from kernel module


 Documentation/filesystems/fuse.txt |   15 +++-
 fs/fuse/dev.c                      |   48 ++++-------
 fs/fuse/file.c                     |   32 ++++---
 fs/fuse/fuse_i.h                   |   31 +++++--
 fs/fuse/inode.c                    |  154 ++++++++++++++++++++++++++++++++----
 fs/pipe.c                          |    7 ++
 include/linux/pipe_fs_i.h          |    3 +
 7 files changed, 213 insertions(+), 77 deletions(-)

-- 
Mitsuo Hayasaka (mitsuo.hayasaka.hu@hitachi.com)

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH -v2 1/6] pipe: make the maximum pipe size referable from kernel module
  2012-07-19 12:48 [PATCH -v2 0/6] fuse: make maximum read/write request size tunable Mitsuo Hayasaka
@ 2012-07-19 12:49 ` Mitsuo Hayasaka
  2012-07-19 12:49 ` [PATCH -v2 2/6] fuse: make the maximum read/write request size tunable Mitsuo Hayasaka
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Alexander Viro,
	Andrew Morton, Muthukumar R, Miklos Szeredi

Make the maximum pipe size referable from a kernel module.

The /proc/sys/fs/pipe-max-size defines an upper limit for the
capacity of a pipe. It is also used as an upper limit of a
fuse read/write request size in this patch series. So, it
is necessary to make it referable from a kernel module.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Muthukumar R <muthur@gmail.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
---

 fs/pipe.c                 |    7 +++++++
 include/linux/pipe_fs_i.h |    3 +++
 2 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/fs/pipe.c b/fs/pipe.c
index 49c1065..f0f3768 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -36,6 +36,13 @@ unsigned int pipe_max_size = 1048576;
  */
 unsigned int pipe_min_size = PAGE_SIZE;
 
+/* get pipe_max_size */
+unsigned int pipe_get_max_size(void)
+{
+	return pipe_max_size;
+}
+EXPORT_SYMBOL(pipe_get_max_size);
+
 /*
  * We use a start+len construction, which provides full use of the 
  * allocated memory.
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index e1ac1ce..50a16dd 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -134,6 +134,9 @@ struct pipe_buf_operations {
    memory allocation, whereas PIPE_BUF makes atomicity guarantees.  */
 #define PIPE_SIZE		PAGE_SIZE
 
+/* get pipe_max_size */
+unsigned int pipe_get_max_size(void);
+
 /* Pipe lock and unlock operations */
 void pipe_lock(struct pipe_inode_info *);
 void pipe_unlock(struct pipe_inode_info *);


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 2/6] fuse: make the maximum read/write request size tunable
  2012-07-19 12:48 [PATCH -v2 0/6] fuse: make maximum read/write request size tunable Mitsuo Hayasaka
  2012-07-19 12:49 ` [PATCH -v2 1/6] pipe: make the maximum pipe size referable from kernel module Mitsuo Hayasaka
@ 2012-07-19 12:49 ` Mitsuo Hayasaka
  2012-08-08 14:04   ` Miklos Szeredi
  2012-07-19 12:49   ` Mitsuo Hayasaka
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Miklos Szeredi,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Make the maximum read/write request size tunable between
32 pages and the number of pages equivalent to pipe_max_size.
The max_read/max_write mount options affect the size. The
32 pages are used by default without these options.

Currently, the maximum read/write request size is limited to
FUSE_MAX_PAGES_PER_REQ which is equal to 32 pages. It is
required to change it in order to maximize the throughput
since the optimized value depends on various factors such as
type and version of local filesystems used and hardware specs,
etc.

In addition, recently FUSE is widely used as a gateway to
connect cloud storage services and distributed filesystems.
Larger data might be stored in them over networking via FUSE
and the overhead might affect the read/write throughput.

So, a tunable functionality of read/write request size is
useful.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Nikolaus Rath <Nikolaus@rath.org>
Cc: Liu Yuan <namei.unix@gmail.com>
Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
---

 fs/fuse/dev.c    |   27 ++++++++++++++-------------
 fs/fuse/file.c   |   32 +++++++++++++++++---------------
 fs/fuse/fuse_i.h |   27 +++++++++++++++++----------
 fs/fuse/inode.c  |   42 +++++++++++++++++++++++++++++++++++-------
 4 files changed, 83 insertions(+), 45 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 7df2b5e..511560b 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -34,35 +34,36 @@ static struct fuse_conn *fuse_get_conn(struct file *file)
 	return file->private_data;
 }
 
-static void fuse_request_init(struct fuse_req *req)
+static void fuse_request_init(struct fuse_conn *fc, struct fuse_req *req)
 {
-	memset(req, 0, sizeof(*req));
+	memset(req, 0, fc->fuse_req_size);
 	INIT_LIST_HEAD(&req->list);
 	INIT_LIST_HEAD(&req->intr_entry);
 	init_waitqueue_head(&req->waitq);
 	atomic_set(&req->count, 1);
 }
 
-struct fuse_req *fuse_request_alloc(void)
+struct fuse_req *fuse_request_alloc(struct fuse_conn *fc)
 {
-	struct fuse_req *req = kmem_cache_alloc(fuse_req_cachep, GFP_KERNEL);
+	struct fuse_req *req = kmalloc(fc->fuse_req_size, GFP_KERNEL);
+
 	if (req)
-		fuse_request_init(req);
+		fuse_request_init(fc, req);
 	return req;
 }
 EXPORT_SYMBOL_GPL(fuse_request_alloc);
 
-struct fuse_req *fuse_request_alloc_nofs(void)
+struct fuse_req *fuse_request_alloc_nofs(struct fuse_conn *fc)
 {
-	struct fuse_req *req = kmem_cache_alloc(fuse_req_cachep, GFP_NOFS);
+	struct fuse_req *req = kmalloc(fc->fuse_req_size, GFP_NOFS);
 	if (req)
-		fuse_request_init(req);
+		fuse_request_init(fc, req);
 	return req;
 }
 
 void fuse_request_free(struct fuse_req *req)
 {
-	kmem_cache_free(fuse_req_cachep, req);
+	kfree(req);
 }
 
 static void block_sigs(sigset_t *oldset)
@@ -116,7 +117,7 @@ struct fuse_req *fuse_get_req(struct fuse_conn *fc)
 	if (!fc->connected)
 		goto out;
 
-	req = fuse_request_alloc();
+	req = fuse_request_alloc(fc);
 	err = -ENOMEM;
 	if (!req)
 		goto out;
@@ -166,7 +167,7 @@ static void put_reserved_req(struct fuse_conn *fc, struct fuse_req *req)
 	struct fuse_file *ff = file->private_data;
 
 	spin_lock(&fc->lock);
-	fuse_request_init(req);
+	fuse_request_init(fc, req);
 	BUG_ON(ff->reserved_req);
 	ff->reserved_req = req;
 	wake_up_all(&fc->reserved_req_waitq);
@@ -193,7 +194,7 @@ struct fuse_req *fuse_get_req_nofail(struct fuse_conn *fc, struct file *file)
 
 	atomic_inc(&fc->num_waiting);
 	wait_event(fc->blocked_waitq, !fc->blocked);
-	req = fuse_request_alloc();
+	req = fuse_request_alloc(fc);
 	if (!req)
 		req = get_reserved_req(fc, file);
 
@@ -1564,7 +1565,7 @@ static int fuse_retrieve(struct fuse_conn *fc, struct inode *inode,
 	else if (outarg->offset + num > file_size)
 		num = file_size - outarg->offset;
 
-	while (num && req->num_pages < FUSE_MAX_PAGES_PER_REQ) {
+	while (num && req->num_pages < fc->max_pages) {
 		struct page *page;
 		unsigned int this_num;
 
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index b321a68..7b96b00 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -57,7 +57,7 @@ struct fuse_file *fuse_file_alloc(struct fuse_conn *fc)
 		return NULL;
 
 	ff->fc = fc;
-	ff->reserved_req = fuse_request_alloc();
+	ff->reserved_req = fuse_request_alloc(fc);
 	if (unlikely(!ff->reserved_req)) {
 		kfree(ff);
 		return NULL;
@@ -653,7 +653,7 @@ static int fuse_readpages_fill(void *_data, struct page *page)
 	fuse_wait_on_page_writeback(inode, page->index);
 
 	if (req->num_pages &&
-	    (req->num_pages == FUSE_MAX_PAGES_PER_REQ ||
+	    (req->num_pages == fc->max_pages ||
 	     (req->num_pages + 1) * PAGE_CACHE_SIZE > fc->max_read ||
 	     req->pages[req->num_pages - 1]->index + 1 != page->index)) {
 		fuse_send_readpages(req, data->file);
@@ -866,7 +866,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_req *req,
 		if (!fc->big_writes)
 			break;
 	} while (iov_iter_count(ii) && count < fc->max_write &&
-		 req->num_pages < FUSE_MAX_PAGES_PER_REQ && offset == 0);
+		 req->num_pages < fc->max_pages && offset == 0);
 
 	return count > 0 ? count : err;
 }
@@ -1020,8 +1020,9 @@ static void fuse_release_user_pages(struct fuse_req *req, int write)
 	}
 }
 
-static int fuse_get_user_pages(struct fuse_req *req, const char __user *buf,
-			       size_t *nbytesp, int write)
+static int fuse_get_user_pages(struct fuse_conn *fc, struct fuse_req *req,
+			       const char __user *buf, size_t *nbytesp,
+			       int write)
 {
 	size_t nbytes = *nbytesp;
 	unsigned long user_addr = (unsigned long) buf;
@@ -1038,9 +1039,9 @@ static int fuse_get_user_pages(struct fuse_req *req, const char __user *buf,
 		return 0;
 	}
 
-	nbytes = min_t(size_t, nbytes, FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT);
+	nbytes = min_t(size_t, nbytes, fc->max_pages << PAGE_SHIFT);
 	npages = (nbytes + offset + PAGE_SIZE - 1) >> PAGE_SHIFT;
-	npages = clamp(npages, 1, FUSE_MAX_PAGES_PER_REQ);
+	npages = clamp(npages, 1, (int)fc->max_pages);
 	npages = get_user_pages_fast(user_addr, npages, !write, req->pages);
 	if (npages < 0)
 		return npages;
@@ -1077,7 +1078,7 @@ ssize_t fuse_direct_io(struct file *file, const char __user *buf,
 		size_t nres;
 		fl_owner_t owner = current->files;
 		size_t nbytes = min(count, nmax);
-		int err = fuse_get_user_pages(req, buf, &nbytes, write);
+		int err = fuse_get_user_pages(fc, req, buf, &nbytes, write);
 		if (err) {
 			res = err;
 			break;
@@ -1269,7 +1270,7 @@ static int fuse_writepage_locked(struct page *page)
 
 	set_page_writeback(page);
 
-	req = fuse_request_alloc_nofs();
+	req = fuse_request_alloc_nofs(fc);
 	if (!req)
 		goto err;
 
@@ -1695,10 +1696,11 @@ static int fuse_copy_ioctl_iovec_old(struct iovec *dst, void *src,
 }
 
 /* Make sure iov_length() won't overflow */
-static int fuse_verify_ioctl_iov(struct iovec *iov, size_t count)
+static int fuse_verify_ioctl_iov(struct fuse_conn *fc, struct iovec *iov,
+				 size_t count)
 {
 	size_t n;
-	u32 max = FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT;
+	u32 max = fc->max_pages << PAGE_SHIFT;
 
 	for (n = 0; n < count; n++) {
 		if (iov->iov_len > (size_t) max)
@@ -1821,7 +1823,7 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
 	BUILD_BUG_ON(sizeof(struct fuse_ioctl_iovec) * FUSE_IOCTL_MAX_IOV > PAGE_SIZE);
 
 	err = -ENOMEM;
-	pages = kcalloc(FUSE_MAX_PAGES_PER_REQ, sizeof(pages[0]), GFP_KERNEL);
+	pages = kcalloc(fc->max_pages, sizeof(pages[0]), GFP_KERNEL);
 	iov_page = (struct iovec *) __get_free_page(GFP_KERNEL);
 	if (!pages || !iov_page)
 		goto out;
@@ -1860,7 +1862,7 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
 
 	/* make sure there are enough buffer pages and init request with them */
 	err = -ENOMEM;
-	if (max_pages > FUSE_MAX_PAGES_PER_REQ)
+	if (max_pages > fc->max_pages)
 		goto out;
 	while (num_pages < max_pages) {
 		pages[num_pages] = alloc_page(GFP_KERNEL | __GFP_HIGHMEM);
@@ -1943,11 +1945,11 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
 		in_iov = iov_page;
 		out_iov = in_iov + in_iovs;
 
-		err = fuse_verify_ioctl_iov(in_iov, in_iovs);
+		err = fuse_verify_ioctl_iov(fc, in_iov, in_iovs);
 		if (err)
 			goto out;
 
-		err = fuse_verify_ioctl_iov(out_iov, out_iovs);
+		err = fuse_verify_ioctl_iov(fc, out_iov, out_iovs);
 		if (err)
 			goto out;
 
diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
index 771fb63..46df615 100644
--- a/fs/fuse/fuse_i.h
+++ b/fs/fuse/fuse_i.h
@@ -22,9 +22,10 @@
 #include <linux/rbtree.h>
 #include <linux/poll.h>
 #include <linux/workqueue.h>
+#include <linux/pipe_fs_i.h>
 
-/** Max number of pages that can be used in a single read request */
-#define FUSE_MAX_PAGES_PER_REQ 32
+/** Default number of pages that can be used in a single read/write request */
+#define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32
 
 /** Bias for fi->writectr, meaning new writepages must not be sent */
 #define FUSE_NOWRITE INT_MIN
@@ -290,12 +291,6 @@ struct fuse_req {
 		struct fuse_lk_in lk_in;
 	} misc;
 
-	/** page vector */
-	struct page *pages[FUSE_MAX_PAGES_PER_REQ];
-
-	/** number of pages in vector */
-	unsigned num_pages;
-
 	/** offset of data on first page */
 	unsigned page_offset;
 
@@ -313,6 +308,12 @@ struct fuse_req {
 
 	/** Request is stolen from fuse_file->reserved_req */
 	struct file *stolen_file;
+
+	/** number of pages in vector */
+	unsigned num_pages;
+
+	/** page vector */
+	struct page *pages[0];
 };
 
 /**
@@ -347,6 +348,12 @@ struct fuse_conn {
 	/** Maximum write size */
 	unsigned max_write;
 
+	/** Maximum number of pages per req */
+	unsigned max_pages;
+
+	/** fuse_req size per connection */
+	unsigned fuse_req_size;
+
 	/** Readers of the connection are waiting on this */
 	wait_queue_head_t waitq;
 
@@ -655,9 +662,9 @@ void fuse_ctl_cleanup(void);
 /**
  * Allocate a request
  */
-struct fuse_req *fuse_request_alloc(void);
+struct fuse_req *fuse_request_alloc(struct fuse_conn *fc);
 
-struct fuse_req *fuse_request_alloc_nofs(void);
+struct fuse_req *fuse_request_alloc_nofs(struct fuse_conn *fc);
 
 /**
  * Free a request
diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index 1cd6165..f7f3c5d 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -550,6 +550,9 @@ void fuse_conn_init(struct fuse_conn *fc)
 	atomic_set(&fc->num_waiting, 0);
 	fc->max_background = FUSE_DEFAULT_MAX_BACKGROUND;
 	fc->congestion_threshold = FUSE_DEFAULT_CONGESTION_THRESHOLD;
+	fc->max_pages = FUSE_DEFAULT_MAX_PAGES_PER_REQ;
+	fc->fuse_req_size = sizeof(struct fuse_req) +
+			    fc->max_pages * sizeof(struct page *);
 	fc->khctr = 0;
 	fc->polled_files = RB_ROOT;
 	fc->reqctr = 0;
@@ -774,6 +777,18 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
 	return 0;
 }
 
+static void set_conn_max_pages(struct fuse_conn *fc, unsigned max_pages)
+{
+	unsigned pipe_max_size = pipe_get_max_size();
+	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
+
+	if (max_pages > fc->max_pages) {
+		fc->max_pages = min_t(unsigned, pipe_max_pages, max_pages);
+		fc->fuse_req_size = sizeof(struct fuse_req) +
+				    fc->max_pages * sizeof(struct page *);
+	}
+}
+
 static void process_init_limits(struct fuse_conn *fc, struct fuse_init_out *arg)
 {
 	int cap_sys_admin = capable(CAP_SYS_ADMIN);
@@ -807,6 +822,7 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_req *req)
 		fc->conn_error = 1;
 	else {
 		unsigned long ra_pages;
+		unsigned max_pages;
 
 		process_init_limits(fc, arg);
 
@@ -844,6 +860,8 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_req *req)
 		fc->minor = arg->minor;
 		fc->max_write = arg->minor < 5 ? 4096 : arg->max_write;
 		fc->max_write = max_t(unsigned, 4096, fc->max_write);
+		max_pages = DIV_ROUND_UP(fc->max_write, PAGE_SIZE);
+		set_conn_max_pages(fc, max_pages);
 		fc->conn_init = 1;
 	}
 	fc->blocked = 0;
@@ -880,6 +898,20 @@ static void fuse_free_conn(struct fuse_conn *fc)
 	kfree(fc);
 }
 
+static void fuse_conn_setup(struct fuse_conn *fc,
+			    struct fuse_mount_data *d)
+{
+	unsigned max_pages;
+
+	fc->release = fuse_free_conn;
+	fc->flags = d->flags;
+	fc->user_id = d->user_id;
+	fc->group_id = d->group_id;
+	fc->max_read = max_t(unsigned, 4096, d->max_read);
+	max_pages = DIV_ROUND_UP(fc->max_read, PAGE_SIZE);
+	set_conn_max_pages(fc, max_pages);
+}
+
 static int fuse_bdi_init(struct fuse_conn *fc, struct super_block *sb)
 {
 	int err;
@@ -986,11 +1018,7 @@ static int fuse_fill_super(struct super_block *sb, void *data, int silent)
 		fc->dont_mask = 1;
 	sb->s_flags |= MS_POSIXACL;
 
-	fc->release = fuse_free_conn;
-	fc->flags = d.flags;
-	fc->user_id = d.user_id;
-	fc->group_id = d.group_id;
-	fc->max_read = max_t(unsigned, 4096, d.max_read);
+	fuse_conn_setup(fc, &d);
 
 	/* Used by get_root_inode() */
 	sb->s_fs_info = fc;
@@ -1003,12 +1031,12 @@ static int fuse_fill_super(struct super_block *sb, void *data, int silent)
 	/* only now - we want root dentry with NULL ->d_op */
 	sb->s_d_op = &fuse_dentry_operations;
 
-	init_req = fuse_request_alloc();
+	init_req = fuse_request_alloc(fc);
 	if (!init_req)
 		goto err_put_root;
 
 	if (is_bdev) {
-		fc->destroy_req = fuse_request_alloc();
+		fc->destroy_req = fuse_request_alloc(fc);
 		if (!fc->destroy_req)
 			goto err_free_init_req;
 	}


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 3/6] fuse: remove cache for fuse request allocation
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Miklos Szeredi,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Remove fuse_req_cachep, which was used for fuse request buffer.
It is no longer used since the buffer is allocated dynamically
due to the tunable maximum read/write request size.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Nikolaus Rath <Nikolaus@rath.org>
Cc: Liu Yuan <namei.unix@gmail.com>
Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
---

 fs/fuse/dev.c |   21 +--------------------
 1 files changed, 1 insertions(+), 20 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 511560b..4087ff4 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -23,8 +23,6 @@
 MODULE_ALIAS_MISCDEV(FUSE_MINOR);
 MODULE_ALIAS("devname:fuse");
 
-static struct kmem_cache *fuse_req_cachep;
-
 static struct fuse_conn *fuse_get_conn(struct file *file)
 {
 	/*
@@ -2075,27 +2073,10 @@ static struct miscdevice fuse_miscdevice = {
 
 int __init fuse_dev_init(void)
 {
-	int err = -ENOMEM;
-	fuse_req_cachep = kmem_cache_create("fuse_request",
-					    sizeof(struct fuse_req),
-					    0, 0, NULL);
-	if (!fuse_req_cachep)
-		goto out;
-
-	err = misc_register(&fuse_miscdevice);
-	if (err)
-		goto out_cache_clean;
-
-	return 0;
-
- out_cache_clean:
-	kmem_cache_destroy(fuse_req_cachep);
- out:
-	return err;
+	return misc_register(&fuse_miscdevice);
 }
 
 void fuse_dev_cleanup(void)
 {
 	misc_deregister(&fuse_miscdevice);
-	kmem_cache_destroy(fuse_req_cachep);
 }


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 3/6] fuse: remove cache for fuse request allocation
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: linux-doc-u79uwXL29TY76Z2rM5mHXA,
	fuse-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f, Miklos Szeredi,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA, Nikolaus Rath,
	yrl.pp-manager.tt-FCd8Q96Dh0JBDgjK7y7TUQ,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA

Remove fuse_req_cachep, which was used for fuse request buffer.
It is no longer used since the buffer is allocated dynamically
due to the tunable maximum read/write request size.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu-FCd8Q96Dh0JBDgjK7y7TUQ@public.gmane.org>
Cc: Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org>
Cc: Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org>
Cc: Liu Yuan <namei.unix-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Has-Wen Nienhuys <hanwen-qWit8jRvyhVmR6Xm/wNWPw@public.gmane.org>
---

 fs/fuse/dev.c |   21 +--------------------
 1 files changed, 1 insertions(+), 20 deletions(-)

diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 511560b..4087ff4 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -23,8 +23,6 @@
 MODULE_ALIAS_MISCDEV(FUSE_MINOR);
 MODULE_ALIAS("devname:fuse");
 
-static struct kmem_cache *fuse_req_cachep;
-
 static struct fuse_conn *fuse_get_conn(struct file *file)
 {
 	/*
@@ -2075,27 +2073,10 @@ static struct miscdevice fuse_miscdevice = {
 
 int __init fuse_dev_init(void)
 {
-	int err = -ENOMEM;
-	fuse_req_cachep = kmem_cache_create("fuse_request",
-					    sizeof(struct fuse_req),
-					    0, 0, NULL);
-	if (!fuse_req_cachep)
-		goto out;
-
-	err = misc_register(&fuse_miscdevice);
-	if (err)
-		goto out_cache_clean;
-
-	return 0;
-
- out_cache_clean:
-	kmem_cache_destroy(fuse_req_cachep);
- out:
-	return err;
+	return misc_register(&fuse_miscdevice);
 }
 
 void fuse_dev_cleanup(void)
 {
 	misc_deregister(&fuse_miscdevice);
-	kmem_cache_destroy(fuse_req_cachep);
 }


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 4/6] fuse: add a sysfs parameter to control the maximum request size
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Miklos Szeredi,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Add a max_pages_per_req sysfs paramater to limit the maximum
read/write request size. It can be changed to arbitrary number
between 32 and the nr_pages equivalent to pipe_max_size, and the
32 pages are set by default.

The sysfs parameter control is required, as follows.

* The libfuse should change the current MIN_BUFSIZE limitation
  according to the current maximum request size in FUSE. If not,
  the libfuse must always set MIN_BUFSIZE to the maximum request
  limit (= [nr_pages (equivalent to pipe_max_size) * 4KB + 0x1000]),
  which leads to waste of memory.

* It is easy to find and set the paramter to the optimized value
  in order to improve the read/write throughput, since the
  maximum request limit does not always provides the highest
  throughput.

So, it is necessary to get and set the maximum size from userspace.

Existing FUSE mounts must be remounted for this change to take
effect.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Nikolaus Rath <Nikolaus@rath.org>
Cc: Liu Yuan <namei.unix@gmail.com>
Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
---

 fs/fuse/inode.c |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 54 insertions(+), 4 deletions(-)

diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index f7f3c5d..5f84a40 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -47,6 +47,14 @@ MODULE_PARM_DESC(max_user_congthresh,
  "Global limit for the maximum congestion threshold an "
  "unprivileged user can set");
 
+/**
+ * Maximum number of pages allocated for struct fuse_req.
+ * It can be changed via sysfs to arbitrary number between
+ * FUSE_DEFAULT_MAX_PAGES_PER_REQ and nr_pages equivalent
+ * to pipe_max_size.
+ */
+static unsigned sysfs_max_req_pages = FUSE_DEFAULT_MAX_PAGES_PER_REQ;
+
 #define FUSE_SUPER_MAGIC 0x65735546
 
 #define FUSE_DEFAULT_BLKSIZE 512
@@ -779,11 +787,8 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
 
 static void set_conn_max_pages(struct fuse_conn *fc, unsigned max_pages)
 {
-	unsigned pipe_max_size = pipe_get_max_size();
-	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
-
 	if (max_pages > fc->max_pages) {
-		fc->max_pages = min_t(unsigned, pipe_max_pages, max_pages);
+		fc->max_pages = min_t(unsigned, sysfs_max_req_pages, max_pages);
 		fc->fuse_req_size = sizeof(struct fuse_req) +
 				    fc->max_pages * sizeof(struct page *);
 	}
@@ -1205,6 +1210,45 @@ static void fuse_fs_cleanup(void)
 static struct kobject *fuse_kobj;
 static struct kobject *connections_kobj;
 
+static ssize_t max_req_pages_show(struct kobject *kobj,
+				  struct kobj_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%u\n", sysfs_max_req_pages);
+}
+
+static ssize_t max_req_pages_store(struct kobject *kobj,
+				   struct kobj_attribute *attr,
+				   const char *buf, size_t count)
+{
+	int err;
+	unsigned long t;
+	unsigned pipe_max_size = pipe_get_max_size();
+	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
+
+	err = kstrtoul(skip_spaces(buf), 0, &t);
+	if (err)
+		return err;
+
+	t = max_t(unsigned long, t, FUSE_DEFAULT_MAX_PAGES_PER_REQ);
+	t = min_t(unsigned long, t, pipe_max_pages);
+
+	sysfs_max_req_pages = t;
+	return count;
+}
+
+static struct kobj_attribute max_req_pages_attr =
+	__ATTR(max_pages_per_req, 0644, max_req_pages_show,
+	       max_req_pages_store);
+
+static struct attribute *fuse_attrs[] = {
+	&max_req_pages_attr.attr,
+	NULL,
+};
+
+static struct attribute_group fuse_attr_grp = {
+	.attrs = fuse_attrs,
+};
+
 static int fuse_sysfs_init(void)
 {
 	int err;
@@ -1221,8 +1265,14 @@ static int fuse_sysfs_init(void)
 		goto out_fuse_unregister;
 	}
 
+	err = sysfs_create_group(fuse_kobj, &fuse_attr_grp);
+	if (err)
+		goto out_conn_unregister;
+
 	return 0;
 
+ out_conn_unregister:
+	kobject_put(connections_kobj);
  out_fuse_unregister:
 	kobject_put(fuse_kobj);
  out_err:


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 4/6] fuse: add a sysfs parameter to control the maximum request size
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: linux-doc-u79uwXL29TY76Z2rM5mHXA,
	fuse-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f, Miklos Szeredi,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA, Nikolaus Rath,
	yrl.pp-manager.tt-FCd8Q96Dh0JBDgjK7y7TUQ,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA

Add a max_pages_per_req sysfs paramater to limit the maximum
read/write request size. It can be changed to arbitrary number
between 32 and the nr_pages equivalent to pipe_max_size, and the
32 pages are set by default.

The sysfs parameter control is required, as follows.

* The libfuse should change the current MIN_BUFSIZE limitation
  according to the current maximum request size in FUSE. If not,
  the libfuse must always set MIN_BUFSIZE to the maximum request
  limit (= [nr_pages (equivalent to pipe_max_size) * 4KB + 0x1000]),
  which leads to waste of memory.

* It is easy to find and set the paramter to the optimized value
  in order to improve the read/write throughput, since the
  maximum request limit does not always provides the highest
  throughput.

So, it is necessary to get and set the maximum size from userspace.

Existing FUSE mounts must be remounted for this change to take
effect.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu-FCd8Q96Dh0JBDgjK7y7TUQ@public.gmane.org>
Cc: Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org>
Cc: Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org>
Cc: Liu Yuan <namei.unix-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Has-Wen Nienhuys <hanwen-qWit8jRvyhVmR6Xm/wNWPw@public.gmane.org>
---

 fs/fuse/inode.c |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 files changed, 54 insertions(+), 4 deletions(-)

diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index f7f3c5d..5f84a40 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -47,6 +47,14 @@ MODULE_PARM_DESC(max_user_congthresh,
  "Global limit for the maximum congestion threshold an "
  "unprivileged user can set");
 
+/**
+ * Maximum number of pages allocated for struct fuse_req.
+ * It can be changed via sysfs to arbitrary number between
+ * FUSE_DEFAULT_MAX_PAGES_PER_REQ and nr_pages equivalent
+ * to pipe_max_size.
+ */
+static unsigned sysfs_max_req_pages = FUSE_DEFAULT_MAX_PAGES_PER_REQ;
+
 #define FUSE_SUPER_MAGIC 0x65735546
 
 #define FUSE_DEFAULT_BLKSIZE 512
@@ -779,11 +787,8 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
 
 static void set_conn_max_pages(struct fuse_conn *fc, unsigned max_pages)
 {
-	unsigned pipe_max_size = pipe_get_max_size();
-	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
-
 	if (max_pages > fc->max_pages) {
-		fc->max_pages = min_t(unsigned, pipe_max_pages, max_pages);
+		fc->max_pages = min_t(unsigned, sysfs_max_req_pages, max_pages);
 		fc->fuse_req_size = sizeof(struct fuse_req) +
 				    fc->max_pages * sizeof(struct page *);
 	}
@@ -1205,6 +1210,45 @@ static void fuse_fs_cleanup(void)
 static struct kobject *fuse_kobj;
 static struct kobject *connections_kobj;
 
+static ssize_t max_req_pages_show(struct kobject *kobj,
+				  struct kobj_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%u\n", sysfs_max_req_pages);
+}
+
+static ssize_t max_req_pages_store(struct kobject *kobj,
+				   struct kobj_attribute *attr,
+				   const char *buf, size_t count)
+{
+	int err;
+	unsigned long t;
+	unsigned pipe_max_size = pipe_get_max_size();
+	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
+
+	err = kstrtoul(skip_spaces(buf), 0, &t);
+	if (err)
+		return err;
+
+	t = max_t(unsigned long, t, FUSE_DEFAULT_MAX_PAGES_PER_REQ);
+	t = min_t(unsigned long, t, pipe_max_pages);
+
+	sysfs_max_req_pages = t;
+	return count;
+}
+
+static struct kobj_attribute max_req_pages_attr =
+	__ATTR(max_pages_per_req, 0644, max_req_pages_show,
+	       max_req_pages_store);
+
+static struct attribute *fuse_attrs[] = {
+	&max_req_pages_attr.attr,
+	NULL,
+};
+
+static struct attribute_group fuse_attr_grp = {
+	.attrs = fuse_attrs,
+};
+
 static int fuse_sysfs_init(void)
 {
 	int err;
@@ -1221,8 +1265,14 @@ static int fuse_sysfs_init(void)
 		goto out_fuse_unregister;
 	}
 
+	err = sysfs_create_group(fuse_kobj, &fuse_attr_grp);
+	if (err)
+		goto out_conn_unregister;
+
 	return 0;
 
+ out_conn_unregister:
+	kobject_put(connections_kobj);
  out_fuse_unregister:
 	kobject_put(fuse_kobj);
  out_err:


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 5/6] fuse: set default global limit considering tunable request size
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Miklos Szeredi,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Set default global limits for backgrounded requests and congestion
threshold considering the tunable maximum request size.

They are calculated using size of fuse_req structure, which is
variable due to it. This patch sets them according to the current
request size unless they are set via mod_param by the system
administrator.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Nikolaus Rath <Nikolaus@rath.org>
Cc: Liu Yuan <namei.unix@gmail.com>
Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
---

 fs/fuse/fuse_i.h |    4 +++
 fs/fuse/inode.c  |   62 ++++++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 55 insertions(+), 11 deletions(-)

diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
index 46df615..2dda6eb 100644
--- a/fs/fuse/fuse_i.h
+++ b/fs/fuse/fuse_i.h
@@ -27,6 +27,10 @@
 /** Default number of pages that can be used in a single read/write request */
 #define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32
 
+/** Maximum size of struct fuse_req */
+#define FUSE_CURRENT_REQ_SIZE (sizeof(struct fuse_req) +\
+			       sysfs_max_req_pages * sizeof(struct page *))
+
 /** Bias for fi->writectr, meaning new writepages must not be sent */
 #define FUSE_NOWRITE INT_MIN
 
diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index 5f84a40..dc0302f 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -29,25 +29,36 @@ static struct kmem_cache *fuse_inode_cachep;
 struct list_head fuse_conn_list;
 DEFINE_MUTEX(fuse_mutex);
 
-static int set_global_limit(const char *val, struct kernel_param *kp);
+static int set_global_limit_bgreq(const char *val, struct kernel_param *kp);
+static int set_global_limit_thresh(const char *val, struct kernel_param *kp);
 
 unsigned max_user_bgreq;
-module_param_call(max_user_bgreq, set_global_limit, param_get_uint,
-		  &max_user_bgreq, 0644);
+module_param_call(max_user_bgreq, set_global_limit_bgreq,
+		  param_get_uint, &max_user_bgreq, 0644);
 __MODULE_PARM_TYPE(max_user_bgreq, "uint");
 MODULE_PARM_DESC(max_user_bgreq,
  "Global limit for the maximum number of backgrounded requests an "
  "unprivileged user can set");
 
 unsigned max_user_congthresh;
-module_param_call(max_user_congthresh, set_global_limit, param_get_uint,
-		  &max_user_congthresh, 0644);
+module_param_call(max_user_congthresh, set_global_limit_thresh,
+		  param_get_uint, &max_user_congthresh, 0644);
 __MODULE_PARM_TYPE(max_user_congthresh, "uint");
 MODULE_PARM_DESC(max_user_congthresh,
  "Global limit for the maximum congestion threshold an "
  "unprivileged user can set");
 
 /**
+ * The flags below are used in order to distinguish how to set
+ * max_user_bgreq and max_user_congthresh, respectively. They
+ * should be used if they are set via mod_param. If not, we should
+ * check their current limitation using check_global_limit() any
+ * time due to the tunable read/write request size.
+ */
+static bool mod_param_set_flg_bgreq;
+static bool mod_param_set_flg_thresh;
+
+/**
  * Maximum number of pages allocated for struct fuse_req.
  * It can be changed via sysfs to arbitrary number between
  * FUSE_DEFAULT_MAX_PAGES_PER_REQ and nr_pages equivalent
@@ -766,13 +777,39 @@ static void sanitize_global_limit(unsigned *limit)
 {
 	if (*limit == 0)
 		*limit = ((num_physpages << PAGE_SHIFT) >> 13) /
-			 sizeof(struct fuse_req);
+			 FUSE_CURRENT_REQ_SIZE;
 
 	if (*limit >= 1 << 16)
 		*limit = (1 << 16) - 1;
 }
 
-static int set_global_limit(const char *val, struct kernel_param *kp)
+static void check_global_limit(unsigned *limit, bool mod_param_flg)
+{
+	if (!mod_param_flg) {
+		unsigned cur_global_limit = 0;
+
+		sanitize_global_limit(&cur_global_limit);
+		*limit = cur_global_limit;
+	}
+}
+
+static int set_global_limit_bgreq(const char *val, struct kernel_param *kp)
+{
+	int rv;
+
+	rv = param_set_uint(val, kp);
+	if (rv)
+		return rv;
+
+	sanitize_global_limit((unsigned *)kp->arg);
+
+	/* max_user_bgreq is set via mod_param */
+	mod_param_set_flg_bgreq = true;
+
+	return 0;
+}
+
+static int set_global_limit_thresh(const char *val, struct kernel_param *kp)
 {
 	int rv;
 
@@ -782,6 +819,9 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
 
 	sanitize_global_limit((unsigned *)kp->arg);
 
+	/* max_user_congthresh is set via mod_param */
+	mod_param_set_flg_thresh = true;
+
 	return 0;
 }
 
@@ -801,8 +841,8 @@ static void process_init_limits(struct fuse_conn *fc, struct fuse_init_out *arg)
 	if (arg->minor < 13)
 		return;
 
-	sanitize_global_limit(&max_user_bgreq);
-	sanitize_global_limit(&max_user_congthresh);
+	check_global_limit(&max_user_bgreq, mod_param_set_flg_bgreq);
+	check_global_limit(&max_user_congthresh, mod_param_set_flg_thresh);
 
 	if (arg->max_background) {
 		fc->max_background = arg->max_background;
@@ -1309,8 +1349,8 @@ static int __init fuse_init(void)
 	if (res)
 		goto err_sysfs_cleanup;
 
-	sanitize_global_limit(&max_user_bgreq);
-	sanitize_global_limit(&max_user_congthresh);
+	check_global_limit(&max_user_bgreq, mod_param_set_flg_bgreq);
+	check_global_limit(&max_user_congthresh, mod_param_set_flg_thresh);
 
 	return 0;
 


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 5/6] fuse: set default global limit considering tunable request size
@ 2012-07-19 12:49   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:49 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: linux-doc-u79uwXL29TY76Z2rM5mHXA,
	fuse-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f, Miklos Szeredi,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA, Nikolaus Rath,
	yrl.pp-manager.tt-FCd8Q96Dh0JBDgjK7y7TUQ,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA

Set default global limits for backgrounded requests and congestion
threshold considering the tunable maximum request size.

They are calculated using size of fuse_req structure, which is
variable due to it. This patch sets them according to the current
request size unless they are set via mod_param by the system
administrator.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu-FCd8Q96Dh0JBDgjK7y7TUQ@public.gmane.org>
Cc: Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org>
Cc: Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org>
Cc: Liu Yuan <namei.unix-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Has-Wen Nienhuys <hanwen-qWit8jRvyhVmR6Xm/wNWPw@public.gmane.org>
---

 fs/fuse/fuse_i.h |    4 +++
 fs/fuse/inode.c  |   62 ++++++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 55 insertions(+), 11 deletions(-)

diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
index 46df615..2dda6eb 100644
--- a/fs/fuse/fuse_i.h
+++ b/fs/fuse/fuse_i.h
@@ -27,6 +27,10 @@
 /** Default number of pages that can be used in a single read/write request */
 #define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32
 
+/** Maximum size of struct fuse_req */
+#define FUSE_CURRENT_REQ_SIZE (sizeof(struct fuse_req) +\
+			       sysfs_max_req_pages * sizeof(struct page *))
+
 /** Bias for fi->writectr, meaning new writepages must not be sent */
 #define FUSE_NOWRITE INT_MIN
 
diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index 5f84a40..dc0302f 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -29,25 +29,36 @@ static struct kmem_cache *fuse_inode_cachep;
 struct list_head fuse_conn_list;
 DEFINE_MUTEX(fuse_mutex);
 
-static int set_global_limit(const char *val, struct kernel_param *kp);
+static int set_global_limit_bgreq(const char *val, struct kernel_param *kp);
+static int set_global_limit_thresh(const char *val, struct kernel_param *kp);
 
 unsigned max_user_bgreq;
-module_param_call(max_user_bgreq, set_global_limit, param_get_uint,
-		  &max_user_bgreq, 0644);
+module_param_call(max_user_bgreq, set_global_limit_bgreq,
+		  param_get_uint, &max_user_bgreq, 0644);
 __MODULE_PARM_TYPE(max_user_bgreq, "uint");
 MODULE_PARM_DESC(max_user_bgreq,
  "Global limit for the maximum number of backgrounded requests an "
  "unprivileged user can set");
 
 unsigned max_user_congthresh;
-module_param_call(max_user_congthresh, set_global_limit, param_get_uint,
-		  &max_user_congthresh, 0644);
+module_param_call(max_user_congthresh, set_global_limit_thresh,
+		  param_get_uint, &max_user_congthresh, 0644);
 __MODULE_PARM_TYPE(max_user_congthresh, "uint");
 MODULE_PARM_DESC(max_user_congthresh,
  "Global limit for the maximum congestion threshold an "
  "unprivileged user can set");
 
 /**
+ * The flags below are used in order to distinguish how to set
+ * max_user_bgreq and max_user_congthresh, respectively. They
+ * should be used if they are set via mod_param. If not, we should
+ * check their current limitation using check_global_limit() any
+ * time due to the tunable read/write request size.
+ */
+static bool mod_param_set_flg_bgreq;
+static bool mod_param_set_flg_thresh;
+
+/**
  * Maximum number of pages allocated for struct fuse_req.
  * It can be changed via sysfs to arbitrary number between
  * FUSE_DEFAULT_MAX_PAGES_PER_REQ and nr_pages equivalent
@@ -766,13 +777,39 @@ static void sanitize_global_limit(unsigned *limit)
 {
 	if (*limit == 0)
 		*limit = ((num_physpages << PAGE_SHIFT) >> 13) /
-			 sizeof(struct fuse_req);
+			 FUSE_CURRENT_REQ_SIZE;
 
 	if (*limit >= 1 << 16)
 		*limit = (1 << 16) - 1;
 }
 
-static int set_global_limit(const char *val, struct kernel_param *kp)
+static void check_global_limit(unsigned *limit, bool mod_param_flg)
+{
+	if (!mod_param_flg) {
+		unsigned cur_global_limit = 0;
+
+		sanitize_global_limit(&cur_global_limit);
+		*limit = cur_global_limit;
+	}
+}
+
+static int set_global_limit_bgreq(const char *val, struct kernel_param *kp)
+{
+	int rv;
+
+	rv = param_set_uint(val, kp);
+	if (rv)
+		return rv;
+
+	sanitize_global_limit((unsigned *)kp->arg);
+
+	/* max_user_bgreq is set via mod_param */
+	mod_param_set_flg_bgreq = true;
+
+	return 0;
+}
+
+static int set_global_limit_thresh(const char *val, struct kernel_param *kp)
 {
 	int rv;
 
@@ -782,6 +819,9 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
 
 	sanitize_global_limit((unsigned *)kp->arg);
 
+	/* max_user_congthresh is set via mod_param */
+	mod_param_set_flg_thresh = true;
+
 	return 0;
 }
 
@@ -801,8 +841,8 @@ static void process_init_limits(struct fuse_conn *fc, struct fuse_init_out *arg)
 	if (arg->minor < 13)
 		return;
 
-	sanitize_global_limit(&max_user_bgreq);
-	sanitize_global_limit(&max_user_congthresh);
+	check_global_limit(&max_user_bgreq, mod_param_set_flg_bgreq);
+	check_global_limit(&max_user_congthresh, mod_param_set_flg_thresh);
 
 	if (arg->max_background) {
 		fc->max_background = arg->max_background;
@@ -1309,8 +1349,8 @@ static int __init fuse_init(void)
 	if (res)
 		goto err_sysfs_cleanup;
 
-	sanitize_global_limit(&max_user_bgreq);
-	sanitize_global_limit(&max_user_congthresh);
+	check_global_limit(&max_user_bgreq, mod_param_set_flg_bgreq);
+	check_global_limit(&max_user_congthresh, mod_param_set_flg_thresh);
 
 	return 0;
 


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 6/6] fuse: add documentation of sysfs parameter to limit maximum fuse request size
@ 2012-07-19 12:50   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:50 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Mitsuo Hayasaka, Rob Landley, Miklos Szeredi,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Add an explanation about the sysfs parameter to limit the
maximum read/write request size.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
Cc: Rob Landley <rob@landley.net>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Nikolaus Rath <Nikolaus@rath.org>
Cc: Liu Yuan <namei.unix@gmail.com>
Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
---

 Documentation/filesystems/fuse.txt |   15 ++++++++++++++-
 1 files changed, 14 insertions(+), 1 deletions(-)

diff --git a/Documentation/filesystems/fuse.txt b/Documentation/filesystems/fuse.txt
index 13af4a4..4e706ec 100644
--- a/Documentation/filesystems/fuse.txt
+++ b/Documentation/filesystems/fuse.txt
@@ -108,13 +108,26 @@ Mount options
 
   With this option the maximum size of read operations can be set.
   The default is infinite.  Note that the size of read requests is
-  limited anyway to 32 pages (which is 128kbyte on i386).
+  limited by max_pages_per_req sysfs parameter (See below for details.)
 
 'blksize=N'
 
   Set the block size for the filesystem.  The default is 512.  This
   option is only valid for 'fuseblk' type mounts.
 
+Sysfs parameter
+~~~~~~~~~~~~~~~
+
+  '/sys/fs/fuse/max_pages_per_req'
+
+Specify max request size in pages, which limits max_read/max_write
+mount option. The default is 32 pages (which is 128kbyte on i386).
+It can be changed to arbitrary number between 32 and the number of
+pages equivalent to pipe_max_size.
+
+Changing it may improve read/write throughput on systems. Existing
+FUSE mount must be remounted for this change to take effect.
+
 Control filesystem
 ~~~~~~~~~~~~~~~~~~
 


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH -v2 6/6] fuse: add documentation of sysfs parameter to limit maximum fuse request size
@ 2012-07-19 12:50   ` Mitsuo Hayasaka
  0 siblings, 0 replies; 14+ messages in thread
From: Mitsuo Hayasaka @ 2012-07-19 12:50 UTC (permalink / raw)
  To: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R
  Cc: linux-doc-u79uwXL29TY76Z2rM5mHXA,
	fuse-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f, Miklos Szeredi,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA, Nikolaus Rath, Rob Landley,
	yrl.pp-manager.tt-FCd8Q96Dh0JBDgjK7y7TUQ,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA

Add an explanation about the sysfs parameter to limit the
maximum read/write request size.

Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu-FCd8Q96Dh0JBDgjK7y7TUQ@public.gmane.org>
Cc: Rob Landley <rob-VoJi6FS/r0vR7s880joybQ@public.gmane.org>
Cc: Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org>
Cc: Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org>
Cc: Liu Yuan <namei.unix-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Has-Wen Nienhuys <hanwen-qWit8jRvyhVmR6Xm/wNWPw@public.gmane.org>
---

 Documentation/filesystems/fuse.txt |   15 ++++++++++++++-
 1 files changed, 14 insertions(+), 1 deletions(-)

diff --git a/Documentation/filesystems/fuse.txt b/Documentation/filesystems/fuse.txt
index 13af4a4..4e706ec 100644
--- a/Documentation/filesystems/fuse.txt
+++ b/Documentation/filesystems/fuse.txt
@@ -108,13 +108,26 @@ Mount options
 
   With this option the maximum size of read operations can be set.
   The default is infinite.  Note that the size of read requests is
-  limited anyway to 32 pages (which is 128kbyte on i386).
+  limited by max_pages_per_req sysfs parameter (See below for details.)
 
 'blksize=N'
 
   Set the block size for the filesystem.  The default is 512.  This
   option is only valid for 'fuseblk' type mounts.
 
+Sysfs parameter
+~~~~~~~~~~~~~~~
+
+  '/sys/fs/fuse/max_pages_per_req'
+
+Specify max request size in pages, which limits max_read/max_write
+mount option. The default is 32 pages (which is 128kbyte on i386).
+It can be changed to arbitrary number between 32 and the number of
+pages equivalent to pipe_max_size.
+
+Changing it may improve read/write throughput on systems. Existing
+FUSE mount must be remounted for this change to take effect.
+
 Control filesystem
 ~~~~~~~~~~~~~~~~~~
 


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* Re: [PATCH -v2 6/6] fuse: add documentation of sysfs parameter to limit maximum fuse request size
  2012-07-19 12:50   ` Mitsuo Hayasaka
  (?)
@ 2012-07-19 17:51   ` Rob Landley
  -1 siblings, 0 replies; 14+ messages in thread
From: Rob Landley @ 2012-07-19 17:51 UTC (permalink / raw)
  To: Mitsuo Hayasaka
  Cc: Miklos Szeredi, Alexander Viro, Andrew Morton, Muthukumar R,
	fuse-devel, linux-kernel, linux-fsdevel, linux-doc,
	yrl.pp-manager.tt, Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

On 07/19/2012 07:50 AM, Mitsuo Hayasaka wrote:
> Add an explanation about the sysfs parameter to limit the
> maximum read/write request size.
> 
> Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
> Cc: Rob Landley <rob@landley.net>
> Cc: Miklos Szeredi <miklos@szeredi.hu>
> Cc: Nikolaus Rath <Nikolaus@rath.org>
> Cc: Liu Yuan <namei.unix@gmail.com>
> Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
> ---

Acked-by: Rob Landley <rob@landley.net>

I'll forward it along next merge window if nobody else grabs it first.

Thanks,

Rob
-- 
GNU/Linux isn't: Linux=GPLv2, GNU=GPLv3+, they can't share code.
Either it's "mere aggregation", or a license violation.  Pick one.

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH -v2 2/6] fuse: make the maximum read/write request size tunable
  2012-07-19 12:49 ` [PATCH -v2 2/6] fuse: make the maximum read/write request size tunable Mitsuo Hayasaka
@ 2012-08-08 14:04   ` Miklos Szeredi
  0 siblings, 0 replies; 14+ messages in thread
From: Miklos Szeredi @ 2012-08-08 14:04 UTC (permalink / raw)
  To: Mitsuo Hayasaka
  Cc: Alexander Viro, Andrew Morton, Muthukumar R, fuse-devel,
	linux-kernel, linux-fsdevel, linux-doc, yrl.pp-manager.tt,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com> writes:

> Make the maximum read/write request size tunable between
> 32 pages and the number of pages equivalent to pipe_max_size.
> The max_read/max_write mount options affect the size. The
> 32 pages are used by default without these options.
>
> Currently, the maximum read/write request size is limited to
> FUSE_MAX_PAGES_PER_REQ which is equal to 32 pages. It is
> required to change it in order to maximize the throughput
> since the optimized value depends on various factors such as
> type and version of local filesystems used and hardware specs,
> etc.
>
> In addition, recently FUSE is widely used as a gateway to
> connect cloud storage services and distributed filesystems.
> Larger data might be stored in them over networking via FUSE
> and the overhead might affect the read/write throughput.
>
> So, a tunable functionality of read/write request size is
> useful.

One problem with this patch is that one request will use more memory
even by default (to be more precise 50% more on 64bit since
sizeof(struct fuse_request) is currently 608 bytes, six of which fit in
a single page slab, so one request uses 4096/6 bytes.  After this patch
requests will use 1024 bytes).

One idea that I already mentioned is to allocate the page array
separately (possibly adding a small array inline, but I suspect that
doesn't make much sense).  And allocate the page array only for requests
that actually need it (i.e. read, write and friends).  That way the
memory use of requests would actually decrease significantly, not
increase.

Hmm?

Thanks,
Miklos

>
> Signed-off-by: Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com>
> Cc: Miklos Szeredi <miklos@szeredi.hu>
> Cc: Nikolaus Rath <Nikolaus@rath.org>
> Cc: Liu Yuan <namei.unix@gmail.com>
> Cc: Has-Wen Nienhuys <hanwen@xs4all.nl>
> ---
>
>  fs/fuse/dev.c    |   27 ++++++++++++++-------------
>  fs/fuse/file.c   |   32 +++++++++++++++++---------------
>  fs/fuse/fuse_i.h |   27 +++++++++++++++++----------
>  fs/fuse/inode.c  |   42 +++++++++++++++++++++++++++++++++++-------
>  4 files changed, 83 insertions(+), 45 deletions(-)
>
> diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
> index 7df2b5e..511560b 100644
> --- a/fs/fuse/dev.c
> +++ b/fs/fuse/dev.c
> @@ -34,35 +34,36 @@ static struct fuse_conn *fuse_get_conn(struct file *file)
>  	return file->private_data;
>  }
>  
> -static void fuse_request_init(struct fuse_req *req)
> +static void fuse_request_init(struct fuse_conn *fc, struct fuse_req *req)
>  {
> -	memset(req, 0, sizeof(*req));
> +	memset(req, 0, fc->fuse_req_size);
>  	INIT_LIST_HEAD(&req->list);
>  	INIT_LIST_HEAD(&req->intr_entry);
>  	init_waitqueue_head(&req->waitq);
>  	atomic_set(&req->count, 1);
>  }
>  
> -struct fuse_req *fuse_request_alloc(void)
> +struct fuse_req *fuse_request_alloc(struct fuse_conn *fc)
>  {
> -	struct fuse_req *req = kmem_cache_alloc(fuse_req_cachep, GFP_KERNEL);
> +	struct fuse_req *req = kmalloc(fc->fuse_req_size, GFP_KERNEL);
> +
>  	if (req)
> -		fuse_request_init(req);
> +		fuse_request_init(fc, req);
>  	return req;
>  }
>  EXPORT_SYMBOL_GPL(fuse_request_alloc);
>  
> -struct fuse_req *fuse_request_alloc_nofs(void)
> +struct fuse_req *fuse_request_alloc_nofs(struct fuse_conn *fc)
>  {
> -	struct fuse_req *req = kmem_cache_alloc(fuse_req_cachep, GFP_NOFS);
> +	struct fuse_req *req = kmalloc(fc->fuse_req_size, GFP_NOFS);
>  	if (req)
> -		fuse_request_init(req);
> +		fuse_request_init(fc, req);
>  	return req;
>  }
>  
>  void fuse_request_free(struct fuse_req *req)
>  {
> -	kmem_cache_free(fuse_req_cachep, req);
> +	kfree(req);
>  }
>  
>  static void block_sigs(sigset_t *oldset)
> @@ -116,7 +117,7 @@ struct fuse_req *fuse_get_req(struct fuse_conn *fc)
>  	if (!fc->connected)
>  		goto out;
>  
> -	req = fuse_request_alloc();
> +	req = fuse_request_alloc(fc);
>  	err = -ENOMEM;
>  	if (!req)
>  		goto out;
> @@ -166,7 +167,7 @@ static void put_reserved_req(struct fuse_conn *fc, struct fuse_req *req)
>  	struct fuse_file *ff = file->private_data;
>  
>  	spin_lock(&fc->lock);
> -	fuse_request_init(req);
> +	fuse_request_init(fc, req);
>  	BUG_ON(ff->reserved_req);
>  	ff->reserved_req = req;
>  	wake_up_all(&fc->reserved_req_waitq);
> @@ -193,7 +194,7 @@ struct fuse_req *fuse_get_req_nofail(struct fuse_conn *fc, struct file *file)
>  
>  	atomic_inc(&fc->num_waiting);
>  	wait_event(fc->blocked_waitq, !fc->blocked);
> -	req = fuse_request_alloc();
> +	req = fuse_request_alloc(fc);
>  	if (!req)
>  		req = get_reserved_req(fc, file);
>  
> @@ -1564,7 +1565,7 @@ static int fuse_retrieve(struct fuse_conn *fc, struct inode *inode,
>  	else if (outarg->offset + num > file_size)
>  		num = file_size - outarg->offset;
>  
> -	while (num && req->num_pages < FUSE_MAX_PAGES_PER_REQ) {
> +	while (num && req->num_pages < fc->max_pages) {
>  		struct page *page;
>  		unsigned int this_num;
>  
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index b321a68..7b96b00 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -57,7 +57,7 @@ struct fuse_file *fuse_file_alloc(struct fuse_conn *fc)
>  		return NULL;
>  
>  	ff->fc = fc;
> -	ff->reserved_req = fuse_request_alloc();
> +	ff->reserved_req = fuse_request_alloc(fc);
>  	if (unlikely(!ff->reserved_req)) {
>  		kfree(ff);
>  		return NULL;
> @@ -653,7 +653,7 @@ static int fuse_readpages_fill(void *_data, struct page *page)
>  	fuse_wait_on_page_writeback(inode, page->index);
>  
>  	if (req->num_pages &&
> -	    (req->num_pages == FUSE_MAX_PAGES_PER_REQ ||
> +	    (req->num_pages == fc->max_pages ||
>  	     (req->num_pages + 1) * PAGE_CACHE_SIZE > fc->max_read ||
>  	     req->pages[req->num_pages - 1]->index + 1 != page->index)) {
>  		fuse_send_readpages(req, data->file);
> @@ -866,7 +866,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_req *req,
>  		if (!fc->big_writes)
>  			break;
>  	} while (iov_iter_count(ii) && count < fc->max_write &&
> -		 req->num_pages < FUSE_MAX_PAGES_PER_REQ && offset == 0);
> +		 req->num_pages < fc->max_pages && offset == 0);
>  
>  	return count > 0 ? count : err;
>  }
> @@ -1020,8 +1020,9 @@ static void fuse_release_user_pages(struct fuse_req *req, int write)
>  	}
>  }
>  
> -static int fuse_get_user_pages(struct fuse_req *req, const char __user *buf,
> -			       size_t *nbytesp, int write)
> +static int fuse_get_user_pages(struct fuse_conn *fc, struct fuse_req *req,
> +			       const char __user *buf, size_t *nbytesp,
> +			       int write)
>  {
>  	size_t nbytes = *nbytesp;
>  	unsigned long user_addr = (unsigned long) buf;
> @@ -1038,9 +1039,9 @@ static int fuse_get_user_pages(struct fuse_req *req, const char __user *buf,
>  		return 0;
>  	}
>  
> -	nbytes = min_t(size_t, nbytes, FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT);
> +	nbytes = min_t(size_t, nbytes, fc->max_pages << PAGE_SHIFT);
>  	npages = (nbytes + offset + PAGE_SIZE - 1) >> PAGE_SHIFT;
> -	npages = clamp(npages, 1, FUSE_MAX_PAGES_PER_REQ);
> +	npages = clamp(npages, 1, (int)fc->max_pages);
>  	npages = get_user_pages_fast(user_addr, npages, !write, req->pages);
>  	if (npages < 0)
>  		return npages;
> @@ -1077,7 +1078,7 @@ ssize_t fuse_direct_io(struct file *file, const char __user *buf,
>  		size_t nres;
>  		fl_owner_t owner = current->files;
>  		size_t nbytes = min(count, nmax);
> -		int err = fuse_get_user_pages(req, buf, &nbytes, write);
> +		int err = fuse_get_user_pages(fc, req, buf, &nbytes, write);
>  		if (err) {
>  			res = err;
>  			break;
> @@ -1269,7 +1270,7 @@ static int fuse_writepage_locked(struct page *page)
>  
>  	set_page_writeback(page);
>  
> -	req = fuse_request_alloc_nofs();
> +	req = fuse_request_alloc_nofs(fc);
>  	if (!req)
>  		goto err;
>  
> @@ -1695,10 +1696,11 @@ static int fuse_copy_ioctl_iovec_old(struct iovec *dst, void *src,
>  }
>  
>  /* Make sure iov_length() won't overflow */
> -static int fuse_verify_ioctl_iov(struct iovec *iov, size_t count)
> +static int fuse_verify_ioctl_iov(struct fuse_conn *fc, struct iovec *iov,
> +				 size_t count)
>  {
>  	size_t n;
> -	u32 max = FUSE_MAX_PAGES_PER_REQ << PAGE_SHIFT;
> +	u32 max = fc->max_pages << PAGE_SHIFT;
>  
>  	for (n = 0; n < count; n++) {
>  		if (iov->iov_len > (size_t) max)
> @@ -1821,7 +1823,7 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
>  	BUILD_BUG_ON(sizeof(struct fuse_ioctl_iovec) * FUSE_IOCTL_MAX_IOV > PAGE_SIZE);
>  
>  	err = -ENOMEM;
> -	pages = kcalloc(FUSE_MAX_PAGES_PER_REQ, sizeof(pages[0]), GFP_KERNEL);
> +	pages = kcalloc(fc->max_pages, sizeof(pages[0]), GFP_KERNEL);
>  	iov_page = (struct iovec *) __get_free_page(GFP_KERNEL);
>  	if (!pages || !iov_page)
>  		goto out;
> @@ -1860,7 +1862,7 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
>  
>  	/* make sure there are enough buffer pages and init request with them */
>  	err = -ENOMEM;
> -	if (max_pages > FUSE_MAX_PAGES_PER_REQ)
> +	if (max_pages > fc->max_pages)
>  		goto out;
>  	while (num_pages < max_pages) {
>  		pages[num_pages] = alloc_page(GFP_KERNEL | __GFP_HIGHMEM);
> @@ -1943,11 +1945,11 @@ long fuse_do_ioctl(struct file *file, unsigned int cmd, unsigned long arg,
>  		in_iov = iov_page;
>  		out_iov = in_iov + in_iovs;
>  
> -		err = fuse_verify_ioctl_iov(in_iov, in_iovs);
> +		err = fuse_verify_ioctl_iov(fc, in_iov, in_iovs);
>  		if (err)
>  			goto out;
>  
> -		err = fuse_verify_ioctl_iov(out_iov, out_iovs);
> +		err = fuse_verify_ioctl_iov(fc, out_iov, out_iovs);
>  		if (err)
>  			goto out;
>  
> diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
> index 771fb63..46df615 100644
> --- a/fs/fuse/fuse_i.h
> +++ b/fs/fuse/fuse_i.h
> @@ -22,9 +22,10 @@
>  #include <linux/rbtree.h>
>  #include <linux/poll.h>
>  #include <linux/workqueue.h>
> +#include <linux/pipe_fs_i.h>
>  
> -/** Max number of pages that can be used in a single read request */
> -#define FUSE_MAX_PAGES_PER_REQ 32
> +/** Default number of pages that can be used in a single read/write request */
> +#define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32
>  
>  /** Bias for fi->writectr, meaning new writepages must not be sent */
>  #define FUSE_NOWRITE INT_MIN
> @@ -290,12 +291,6 @@ struct fuse_req {
>  		struct fuse_lk_in lk_in;
>  	} misc;
>  
> -	/** page vector */
> -	struct page *pages[FUSE_MAX_PAGES_PER_REQ];
> -
> -	/** number of pages in vector */
> -	unsigned num_pages;
> -
>  	/** offset of data on first page */
>  	unsigned page_offset;
>  
> @@ -313,6 +308,12 @@ struct fuse_req {
>  
>  	/** Request is stolen from fuse_file->reserved_req */
>  	struct file *stolen_file;
> +
> +	/** number of pages in vector */
> +	unsigned num_pages;
> +
> +	/** page vector */
> +	struct page *pages[0];
>  };
>  
>  /**
> @@ -347,6 +348,12 @@ struct fuse_conn {
>  	/** Maximum write size */
>  	unsigned max_write;
>  
> +	/** Maximum number of pages per req */
> +	unsigned max_pages;
> +
> +	/** fuse_req size per connection */
> +	unsigned fuse_req_size;
> +
>  	/** Readers of the connection are waiting on this */
>  	wait_queue_head_t waitq;
>  
> @@ -655,9 +662,9 @@ void fuse_ctl_cleanup(void);
>  /**
>   * Allocate a request
>   */
> -struct fuse_req *fuse_request_alloc(void);
> +struct fuse_req *fuse_request_alloc(struct fuse_conn *fc);
>  
> -struct fuse_req *fuse_request_alloc_nofs(void);
> +struct fuse_req *fuse_request_alloc_nofs(struct fuse_conn *fc);
>  
>  /**
>   * Free a request
> diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
> index 1cd6165..f7f3c5d 100644
> --- a/fs/fuse/inode.c
> +++ b/fs/fuse/inode.c
> @@ -550,6 +550,9 @@ void fuse_conn_init(struct fuse_conn *fc)
>  	atomic_set(&fc->num_waiting, 0);
>  	fc->max_background = FUSE_DEFAULT_MAX_BACKGROUND;
>  	fc->congestion_threshold = FUSE_DEFAULT_CONGESTION_THRESHOLD;
> +	fc->max_pages = FUSE_DEFAULT_MAX_PAGES_PER_REQ;
> +	fc->fuse_req_size = sizeof(struct fuse_req) +
> +			    fc->max_pages * sizeof(struct page *);
>  	fc->khctr = 0;
>  	fc->polled_files = RB_ROOT;
>  	fc->reqctr = 0;
> @@ -774,6 +777,18 @@ static int set_global_limit(const char *val, struct kernel_param *kp)
>  	return 0;
>  }
>  
> +static void set_conn_max_pages(struct fuse_conn *fc, unsigned max_pages)
> +{
> +	unsigned pipe_max_size = pipe_get_max_size();
> +	unsigned pipe_max_pages = DIV_ROUND_UP(pipe_max_size, PAGE_SIZE);
> +
> +	if (max_pages > fc->max_pages) {
> +		fc->max_pages = min_t(unsigned, pipe_max_pages, max_pages);
> +		fc->fuse_req_size = sizeof(struct fuse_req) +
> +				    fc->max_pages * sizeof(struct page *);
> +	}
> +}
> +
>  static void process_init_limits(struct fuse_conn *fc, struct fuse_init_out *arg)
>  {
>  	int cap_sys_admin = capable(CAP_SYS_ADMIN);
> @@ -807,6 +822,7 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_req *req)
>  		fc->conn_error = 1;
>  	else {
>  		unsigned long ra_pages;
> +		unsigned max_pages;
>  
>  		process_init_limits(fc, arg);
>  
> @@ -844,6 +860,8 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_req *req)
>  		fc->minor = arg->minor;
>  		fc->max_write = arg->minor < 5 ? 4096 : arg->max_write;
>  		fc->max_write = max_t(unsigned, 4096, fc->max_write);
> +		max_pages = DIV_ROUND_UP(fc->max_write, PAGE_SIZE);
> +		set_conn_max_pages(fc, max_pages);
>  		fc->conn_init = 1;
>  	}
>  	fc->blocked = 0;
> @@ -880,6 +898,20 @@ static void fuse_free_conn(struct fuse_conn *fc)
>  	kfree(fc);
>  }
>  
> +static void fuse_conn_setup(struct fuse_conn *fc,
> +			    struct fuse_mount_data *d)
> +{
> +	unsigned max_pages;
> +
> +	fc->release = fuse_free_conn;
> +	fc->flags = d->flags;
> +	fc->user_id = d->user_id;
> +	fc->group_id = d->group_id;
> +	fc->max_read = max_t(unsigned, 4096, d->max_read);
> +	max_pages = DIV_ROUND_UP(fc->max_read, PAGE_SIZE);
> +	set_conn_max_pages(fc, max_pages);
> +}
> +
>  static int fuse_bdi_init(struct fuse_conn *fc, struct super_block *sb)
>  {
>  	int err;
> @@ -986,11 +1018,7 @@ static int fuse_fill_super(struct super_block *sb, void *data, int silent)
>  		fc->dont_mask = 1;
>  	sb->s_flags |= MS_POSIXACL;
>  
> -	fc->release = fuse_free_conn;
> -	fc->flags = d.flags;
> -	fc->user_id = d.user_id;
> -	fc->group_id = d.group_id;
> -	fc->max_read = max_t(unsigned, 4096, d.max_read);
> +	fuse_conn_setup(fc, &d);
>  
>  	/* Used by get_root_inode() */
>  	sb->s_fs_info = fc;
> @@ -1003,12 +1031,12 @@ static int fuse_fill_super(struct super_block *sb, void *data, int silent)
>  	/* only now - we want root dentry with NULL ->d_op */
>  	sb->s_d_op = &fuse_dentry_operations;
>  
> -	init_req = fuse_request_alloc();
> +	init_req = fuse_request_alloc(fc);
>  	if (!init_req)
>  		goto err_put_root;
>  
>  	if (is_bdev) {
> -		fc->destroy_req = fuse_request_alloc();
> +		fc->destroy_req = fuse_request_alloc(fc);
>  		if (!fc->destroy_req)
>  			goto err_free_init_req;
>  	}

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH -v2 4/6] fuse: add a sysfs parameter to control the maximum request size
  2012-07-19 12:49   ` Mitsuo Hayasaka
  (?)
@ 2012-08-08 14:43   ` Miklos Szeredi
  -1 siblings, 0 replies; 14+ messages in thread
From: Miklos Szeredi @ 2012-08-08 14:43 UTC (permalink / raw)
  To: Mitsuo Hayasaka
  Cc: Alexander Viro, Andrew Morton, Muthukumar R, fuse-devel,
	linux-kernel, linux-fsdevel, linux-doc, yrl.pp-manager.tt,
	Nikolaus Rath, Liu Yuan, Has-Wen Nienhuys

Mitsuo Hayasaka <mitsuo.hayasaka.hu@hitachi.com> writes:

> Add a max_pages_per_req sysfs paramater to limit the maximum
> read/write request size. It can be changed to arbitrary number
> between 32 and the nr_pages equivalent to pipe_max_size, and the
> 32 pages are set by default.
>
> The sysfs parameter control is required, as follows.
>
> * The libfuse should change the current MIN_BUFSIZE limitation
>   according to the current maximum request size in FUSE. If not,
>   the libfuse must always set MIN_BUFSIZE to the maximum request
>   limit (= [nr_pages (equivalent to pipe_max_size) * 4KB + 0x1000]),
>   which leads to waste of memory.

I don't see the purpose of this sysfs parameter.  Userspace can
calculate the needed buffer size from the max_read/max_write parameters,
can't it?

>
> * It is easy to find and set the paramter to the optimized value
>   in order to improve the read/write throughput, since the
>   maximum request limit does not always provides the highest
>   throughput.

So basically this is a global max_read/max_write limit?  For that a
better solution would be to add it to /etc/fuse.conf.

But I have doubts about whether this is useful.

Thanks,
Miklos

^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2012-08-08 14:42 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-19 12:48 [PATCH -v2 0/6] fuse: make maximum read/write request size tunable Mitsuo Hayasaka
2012-07-19 12:49 ` [PATCH -v2 1/6] pipe: make the maximum pipe size referable from kernel module Mitsuo Hayasaka
2012-07-19 12:49 ` [PATCH -v2 2/6] fuse: make the maximum read/write request size tunable Mitsuo Hayasaka
2012-08-08 14:04   ` Miklos Szeredi
2012-07-19 12:49 ` [PATCH -v2 3/6] fuse: remove cache for fuse request allocation Mitsuo Hayasaka
2012-07-19 12:49   ` Mitsuo Hayasaka
2012-07-19 12:49 ` [PATCH -v2 4/6] fuse: add a sysfs parameter to control the maximum request size Mitsuo Hayasaka
2012-07-19 12:49   ` Mitsuo Hayasaka
2012-08-08 14:43   ` Miklos Szeredi
2012-07-19 12:49 ` [PATCH -v2 5/6] fuse: set default global limit considering tunable " Mitsuo Hayasaka
2012-07-19 12:49   ` Mitsuo Hayasaka
2012-07-19 12:50 ` [PATCH -v2 6/6] fuse: add documentation of sysfs parameter to limit maximum fuse " Mitsuo Hayasaka
2012-07-19 12:50   ` Mitsuo Hayasaka
2012-07-19 17:51   ` Rob Landley

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.