From: David Howells <dhowells@redhat.com>
To: linux-fsdevel@vger.kernel.org
Cc: Jeff Layton <jlayton@redhat.com>,
Matthew Wilcox <willy@infradead.org>,
linux-mm@kvack.org, linux-cachefs@redhat.com,
linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org,
linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
v9fs-developer@lists.sourceforge.net, dhowells@redhat.com,
Trond Myklebust <trond.myklebust@hammerspace.com>,
Anna Schumaker <anna.schumaker@netapp.com>,
Steve French <sfrench@samba.org>,
Dominique Martinet <asmadeus@codewreck.org>,
Jeff Layton <jlayton@redhat.com>,
David Wysochanski <dwysocha@redhat.com>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-cachefs@redhat.com, linux-afs@lists.infradead.org,
linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org,
ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: [PATCH v6 12/30] netfs: Gather stats
Date: Thu, 08 Apr 2021 15:06:32 +0100 [thread overview]
Message-ID: <161789079281.6155.17141344853277186500.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <161789062190.6155.12711584466338493050.stgit@warthog.procyon.org.uk>
Gather statistics from the netfs interface that can be exported through a
seqfile. This is intended to be called by a later patch when viewing
/proc/fs/fscache/stats.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
cc: Matthew Wilcox <willy@infradead.org>
cc: linux-mm@kvack.org
cc: linux-cachefs@redhat.com
cc: linux-afs@lists.infradead.org
cc: linux-nfs@vger.kernel.org
cc: linux-cifs@vger.kernel.org
cc: ceph-devel@vger.kernel.org
cc: v9fs-developer@lists.sourceforge.net
cc: linux-fsdevel@vger.kernel.org
Link: https://lore.kernel.org/r/161118139247.1232039.10556850937548511068.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/161161034669.2537118.2761232524997091480.stgit@warthog.procyon.org.uk/ # v2
Link: https://lore.kernel.org/r/161340397101.1303470.17581910581108378458.stgit@warthog.procyon.org.uk/ # v3
Link: https://lore.kernel.org/r/161539539959.286939.6794352576462965914.stgit@warthog.procyon.org.uk/ # v4
Link: https://lore.kernel.org/r/161653797700.2770958.5801990354413178228.stgit@warthog.procyon.org.uk/ # v5
---
fs/netfs/Kconfig | 15 +++++++++++++
fs/netfs/Makefile | 3 +--
fs/netfs/internal.h | 34 ++++++++++++++++++++++++++++++
fs/netfs/read_helper.c | 23 ++++++++++++++++++++
fs/netfs/stats.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/netfs.h | 1 +
6 files changed, 128 insertions(+), 2 deletions(-)
create mode 100644 fs/netfs/stats.c
diff --git a/fs/netfs/Kconfig b/fs/netfs/Kconfig
index 2ebf90e6ca95..578112713703 100644
--- a/fs/netfs/Kconfig
+++ b/fs/netfs/Kconfig
@@ -6,3 +6,18 @@ config NETFS_SUPPORT
This option enables support for network filesystems, including
helpers for high-level buffered I/O, abstracting out read
segmentation, local caching and transparent huge page support.
+
+config NETFS_STATS
+ bool "Gather statistical information on local caching"
+ depends on NETFS_SUPPORT && PROC_FS
+ help
+ This option causes statistical information to be gathered on local
+ caching and exported through file:
+
+ /proc/fs/fscache/stats
+
+ The gathering of statistics adds a certain amount of overhead to
+ execution as there are a quite a few stats gathered, and on a
+ multi-CPU system these may be on cachelines that keep bouncing
+ between CPUs. On the other hand, the stats are very useful for
+ debugging purposes. Saying 'Y' here is recommended.
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index 4b4eff2ba369..c15bfc966d96 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -1,6 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-netfs-y := \
- read_helper.o
+netfs-y := read_helper.o stats.o
obj-$(CONFIG_NETFS_SUPPORT) := netfs.o
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index ee665c0e7dc8..98b6f4516da1 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -16,8 +16,42 @@
*/
extern unsigned int netfs_debug;
+/*
+ * stats.c
+ */
+#ifdef CONFIG_NETFS_STATS
+extern atomic_t netfs_n_rh_readahead;
+extern atomic_t netfs_n_rh_readpage;
+extern atomic_t netfs_n_rh_rreq;
+extern atomic_t netfs_n_rh_sreq;
+extern atomic_t netfs_n_rh_download;
+extern atomic_t netfs_n_rh_download_done;
+extern atomic_t netfs_n_rh_download_failed;
+extern atomic_t netfs_n_rh_download_instead;
+extern atomic_t netfs_n_rh_read;
+extern atomic_t netfs_n_rh_read_done;
+extern atomic_t netfs_n_rh_read_failed;
+extern atomic_t netfs_n_rh_zero;
+extern atomic_t netfs_n_rh_short_read;
+extern atomic_t netfs_n_rh_write;
+extern atomic_t netfs_n_rh_write_done;
+extern atomic_t netfs_n_rh_write_failed;
+
+
+static inline void netfs_stat(atomic_t *stat)
+{
+ atomic_inc(stat);
+}
+
+static inline void netfs_stat_d(atomic_t *stat)
+{
+ atomic_dec(stat);
+}
+
+#else
#define netfs_stat(x) do {} while(0)
#define netfs_stat_d(x) do {} while(0)
+#endif
/*****************************************************************************/
/*
diff --git a/fs/netfs/read_helper.c b/fs/netfs/read_helper.c
index 799eee7f4ee6..6d6ed30f417e 100644
--- a/fs/netfs/read_helper.c
+++ b/fs/netfs/read_helper.c
@@ -56,6 +56,7 @@ static struct netfs_read_request *netfs_alloc_read_request(
refcount_set(&rreq->usage, 1);
__set_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
ops->init_rreq(rreq, file);
+ netfs_stat(&netfs_n_rh_rreq);
}
return rreq;
@@ -88,6 +89,7 @@ static void netfs_free_read_request(struct work_struct *work)
rreq->netfs_ops->cleanup(rreq->mapping, rreq->netfs_priv);
trace_netfs_rreq(rreq, netfs_rreq_trace_free);
kfree(rreq);
+ netfs_stat_d(&netfs_n_rh_rreq);
}
static void netfs_put_read_request(struct netfs_read_request *rreq, bool was_async)
@@ -117,6 +119,7 @@ static struct netfs_read_subrequest *netfs_alloc_subrequest(
refcount_set(&subreq->usage, 2);
subreq->rreq = rreq;
netfs_get_read_request(rreq);
+ netfs_stat(&netfs_n_rh_sreq);
}
return subreq;
@@ -134,6 +137,7 @@ static void __netfs_put_subrequest(struct netfs_read_subrequest *subreq,
trace_netfs_sreq(subreq, netfs_sreq_trace_free);
kfree(subreq);
+ netfs_stat_d(&netfs_n_rh_sreq);
netfs_put_read_request(rreq, was_async);
}
@@ -156,6 +160,7 @@ static void netfs_clear_unread(struct netfs_read_subrequest *subreq)
static void netfs_fill_with_zeroes(struct netfs_read_request *rreq,
struct netfs_read_subrequest *subreq)
{
+ netfs_stat(&netfs_n_rh_zero);
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
netfs_subreq_terminated(subreq, 0, false);
}
@@ -179,6 +184,7 @@ static void netfs_fill_with_zeroes(struct netfs_read_request *rreq,
static void netfs_read_from_server(struct netfs_read_request *rreq,
struct netfs_read_subrequest *subreq)
{
+ netfs_stat(&netfs_n_rh_download);
rreq->netfs_ops->issue_op(subreq);
}
@@ -288,6 +294,7 @@ static void netfs_rreq_short_read(struct netfs_read_request *rreq,
__clear_bit(NETFS_SREQ_SHORT_READ, &subreq->flags);
__set_bit(NETFS_SREQ_SEEK_DATA_READ, &subreq->flags);
+ netfs_stat(&netfs_n_rh_short_read);
trace_netfs_sreq(subreq, netfs_sreq_trace_resubmit_short);
netfs_get_read_subrequest(subreq);
@@ -319,6 +326,7 @@ static bool netfs_rreq_perform_resubmissions(struct netfs_read_request *rreq)
break;
subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
subreq->error = 0;
+ netfs_stat(&netfs_n_rh_download_instead);
trace_netfs_sreq(subreq, netfs_sreq_trace_download_instead);
netfs_get_read_subrequest(subreq);
atomic_inc(&rreq->nr_rd_ops);
@@ -414,6 +422,17 @@ void netfs_subreq_terminated(struct netfs_read_subrequest *subreq,
subreq->debug_index, subreq->start, subreq->flags,
transferred_or_error);
+ switch (subreq->source) {
+ case NETFS_READ_FROM_CACHE:
+ netfs_stat(&netfs_n_rh_read_done);
+ break;
+ case NETFS_DOWNLOAD_FROM_SERVER:
+ netfs_stat(&netfs_n_rh_download_done);
+ break;
+ default:
+ break;
+ }
+
if (IS_ERR_VALUE(transferred_or_error)) {
subreq->error = transferred_or_error;
goto failed;
@@ -470,8 +489,10 @@ void netfs_subreq_terminated(struct netfs_read_subrequest *subreq,
failed:
if (subreq->source == NETFS_READ_FROM_CACHE) {
+ netfs_stat(&netfs_n_rh_read_failed);
set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
} else {
+ netfs_stat(&netfs_n_rh_download_failed);
set_bit(NETFS_RREQ_FAILED, &rreq->flags);
rreq->error = subreq->error;
}
@@ -653,6 +674,7 @@ void netfs_readahead(struct readahead_control *ractl,
rreq->start = readahead_pos(ractl);
rreq->len = readahead_length(ractl);
+ netfs_stat(&netfs_n_rh_readahead);
trace_netfs_read(rreq, readahead_pos(ractl), readahead_length(ractl),
netfs_read_trace_readahead);
@@ -722,6 +744,7 @@ int netfs_readpage(struct file *file,
rreq->start = page_index(page) * PAGE_SIZE;
rreq->len = thp_size(page);
+ netfs_stat(&netfs_n_rh_readpage);
trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
netfs_get_read_request(rreq);
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
new file mode 100644
index 000000000000..df6ff5718f25
--- /dev/null
+++ b/fs/netfs/stats.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Netfs support statistics
+ *
+ * Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/export.h>
+#include <linux/seq_file.h>
+#include <linux/netfs.h>
+#include "internal.h"
+
+atomic_t netfs_n_rh_readahead;
+atomic_t netfs_n_rh_readpage;
+atomic_t netfs_n_rh_rreq;
+atomic_t netfs_n_rh_sreq;
+atomic_t netfs_n_rh_download;
+atomic_t netfs_n_rh_download_done;
+atomic_t netfs_n_rh_download_failed;
+atomic_t netfs_n_rh_download_instead;
+atomic_t netfs_n_rh_read;
+atomic_t netfs_n_rh_read_done;
+atomic_t netfs_n_rh_read_failed;
+atomic_t netfs_n_rh_zero;
+atomic_t netfs_n_rh_short_read;
+atomic_t netfs_n_rh_write;
+atomic_t netfs_n_rh_write_done;
+atomic_t netfs_n_rh_write_failed;
+
+void netfs_stats_show(struct seq_file *m)
+{
+ seq_printf(m, "RdHelp : RA=%u RP=%u rr=%u sr=%u\n",
+ atomic_read(&netfs_n_rh_readahead),
+ atomic_read(&netfs_n_rh_readpage),
+ atomic_read(&netfs_n_rh_rreq),
+ atomic_read(&netfs_n_rh_sreq));
+ seq_printf(m, "RdHelp : ZR=%u sh=%u\n",
+ atomic_read(&netfs_n_rh_zero),
+ atomic_read(&netfs_n_rh_short_read));
+ seq_printf(m, "RdHelp : DL=%u ds=%u df=%u di=%u\n",
+ atomic_read(&netfs_n_rh_download),
+ atomic_read(&netfs_n_rh_download_done),
+ atomic_read(&netfs_n_rh_download_failed),
+ atomic_read(&netfs_n_rh_download_instead));
+ seq_printf(m, "RdHelp : RD=%u rs=%u rf=%u\n",
+ atomic_read(&netfs_n_rh_read),
+ atomic_read(&netfs_n_rh_read_done),
+ atomic_read(&netfs_n_rh_read_failed));
+ seq_printf(m, "RdHelp : WR=%u ws=%u wf=%u\n",
+ atomic_read(&netfs_n_rh_write),
+ atomic_read(&netfs_n_rh_write_done),
+ atomic_read(&netfs_n_rh_write_failed));
+}
+EXPORT_SYMBOL(netfs_stats_show);
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 8e8c6a4e4dde..db4af80cbae3 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -166,5 +166,6 @@ extern int netfs_readpage(struct file *,
void *);
extern void netfs_subreq_terminated(struct netfs_read_subrequest *, ssize_t, bool);
+extern void netfs_stats_show(struct seq_file *);
#endif /* _LINUX_NETFS_H */
next prev parent reply other threads:[~2021-04-08 14:08 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-08 14:03 [PATCH v6 00/30] Network fs helper library & fscache kiocb API David Howells
2021-04-08 14:04 ` [PATCH v6 01/30] iov_iter: Add ITER_XARRAY David Howells
2021-04-09 1:24 ` Al Viro
2021-04-09 18:10 ` [RFC PATCH 1/2] iov_iter: Remove iov_iter_for_each_range() David Howells
2021-04-09 18:11 ` [RFC PATCH 2/2] iov_iter: Drop the X argument from iterate_all_kinds() and use B instead David Howells
2021-04-09 9:09 ` [PATCH v6 01/30] iov_iter: Add ITER_XARRAY David Howells
2021-04-09 11:44 ` Matthew Wilcox
2021-04-22 13:35 ` Jeff Layton
2021-04-22 13:51 ` David Howells
2021-04-22 14:51 ` Jeff Layton
2021-04-08 14:04 ` [PATCH v6 02/30] mm: Add set/end/wait functions for PG_private_2 David Howells
2021-04-08 14:50 ` Matthew Wilcox
2021-04-08 15:25 ` David Howells
2021-04-08 15:57 ` David Howells
2021-04-08 21:15 ` [RFC][PATCH] mm: Split page_has_private() in two to better handle PG_private_2 David Howells
2021-04-08 23:50 ` Linus Torvalds
2021-04-09 10:59 ` [RFC PATCH 1/3] Make the generic bitops return bool David Howells
2021-04-09 10:59 ` [RFC PATCH 2/3] mm: Return bool from pagebit test functions David Howells
2021-04-09 11:16 ` Matthew Wilcox
2021-04-09 12:00 ` David Howells
2021-04-09 12:35 ` David Howells
2021-04-09 16:03 ` David Howells
2021-04-09 10:59 ` [RFC PATCH 3/3] mm: Split page_has_private() in two to better handle PG_private_2 David Howells
2021-04-09 11:24 ` Matthew Wilcox
2021-04-09 8:14 ` [RFC][PATCH] " David Howells
2021-04-13 15:11 ` [PATCH v7] mm: Add set/end/wait functions for PG_private_2 David Howells
2021-04-13 15:22 ` Matthew Wilcox
2021-04-08 14:04 ` [PATCH v6 03/30] mm/filemap: Pass the file_ra_state in the ractl David Howells
2021-04-08 14:04 ` [PATCH v6 04/30] fs: Document file_ra_state David Howells
2021-04-08 14:04 ` [PATCH v6 05/30] mm: Implement readahead_control pageset expansion David Howells
2021-04-08 14:55 ` Matthew Wilcox
2021-04-08 14:05 ` [PATCH v6 06/30] netfs: Make a netfs helper module David Howells
2021-04-08 14:05 ` [PATCH v6 07/30] netfs: Documentation for helper library David Howells
2021-04-08 14:05 ` [PATCH v6 08/30] netfs, mm: Move PG_fscache helper funcs to linux/netfs.h David Howells
2021-04-08 14:05 ` [PATCH v6 09/30] netfs, mm: Add set/end/wait_on_page_fscache() aliases David Howells
2021-04-08 14:06 ` [PATCH v6 10/30] netfs: Provide readahead and readpage netfs helpers David Howells
2021-04-08 14:06 ` [PATCH v6 11/30] netfs: Add tracepoints David Howells
2021-04-08 14:06 ` David Howells [this message]
2021-04-08 14:06 ` [PATCH v6 13/30] netfs: Add write_begin helper David Howells
2021-04-08 14:06 ` [PATCH v6 14/30] netfs: Define an interface to talk to a cache David Howells
2021-04-08 14:07 ` [PATCH v6 15/30] netfs: Add a tracepoint to log failures that would be otherwise unseen David Howells
2021-04-08 14:07 ` [PATCH v6 16/30] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache David Howells
2021-04-08 14:07 ` [PATCH v6 17/30] afs: Disable use of the fscache I/O routines David Howells
2021-04-08 14:07 ` [PATCH v6 18/30] afs: Pass page into dirty region helpers to provide THP size David Howells
2021-04-08 14:08 ` [PATCH v6 19/30] afs: Print the operation debug_id when logging an unexpected data version David Howells
2021-04-08 14:08 ` [PATCH v6 20/30] afs: Move key to afs_read struct David Howells
2021-04-08 14:08 ` [PATCH v6 21/30] afs: Don't truncate iter during data fetch David Howells
2021-04-08 14:08 ` [PATCH v6 22/30] afs: Log remote unmarshalling errors David Howells
2021-04-08 14:08 ` [PATCH v6 23/30] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2021-04-08 14:09 ` [PATCH v6 24/30] afs: Use ITER_XARRAY for writing David Howells
2021-04-08 14:09 ` [PATCH v6 25/30] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2021-04-08 14:09 ` [PATCH v6 26/30] afs: Extract writeback extension into its own function David Howells
2021-04-08 14:09 ` [PATCH v6 27/30] afs: Prepare for use of THPs David Howells
2021-04-08 14:09 ` [PATCH v6 28/30] afs: Use the fs operation ops to handle FetchData completion David Howells
2021-04-08 14:10 ` [PATCH v6 29/30] afs: Use new netfs lib read helper API David Howells
2021-04-08 14:10 ` [PATCH v6 30/30] afs: Use the netfs_write_begin() helper David Howells
2021-04-08 15:40 ` [PATCH v6 00/30] Network fs helper library & fscache kiocb API Jeff Layton
2021-04-09 12:04 ` David Wysochanski
2021-04-13 14:56 ` David Wysochanski
2021-04-13 16:21 ` Marc Dionne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=161789079281.6155.17141344853277186500.stgit@warthog.procyon.org.uk \
--to=dhowells@redhat.com \
--cc=anna.schumaker@netapp.com \
--cc=asmadeus@codewreck.org \
--cc=ceph-devel@vger.kernel.org \
--cc=dwysocha@redhat.com \
--cc=jlayton@redhat.com \
--cc=linux-afs@lists.infradead.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=sfrench@samba.org \
--cc=trond.myklebust@hammerspace.com \
--cc=v9fs-developer@lists.sourceforge.net \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).