From: Jeff Layton <jlayton@kernel.org>
To: Xiubo Li <xiubli@redhat.com>, Patrick Donnelly <pdonnell@redhat.com>
Cc: "Luís Henriques" <lhenriques@suse.de>,
"Ilya Dryomov" <idryomov@gmail.com>,
"Ceph Development" <ceph-devel@vger.kernel.org>,
linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH] ceph: add remote object copy counter to fs client
Date: Tue, 26 Oct 2021 07:40:51 -0400 [thread overview]
Message-ID: <604199ed389d9286e3fdab6b5acdf65c421df45d.camel@kernel.org> (raw)
In-Reply-To: <785d1435-4a2c-95aa-0573-2de54b4e7b6b@redhat.com>
On Tue, 2021-10-26 at 11:05 +0800, Xiubo Li wrote:
> On 10/22/21 1:30 AM, Patrick Donnelly wrote:
> > On Thu, Oct 21, 2021 at 12:35 PM Jeff Layton <jlayton@kernel.org> wrote:
> > > On Thu, 2021-10-21 at 12:18 -0400, Patrick Donnelly wrote:
> > > > On Thu, Oct 21, 2021 at 11:44 AM Jeff Layton <jlayton@kernel.org> wrote:
> > > > > On Thu, 2021-10-21 at 09:52 -0400, Patrick Donnelly wrote:
> > > > > > On Wed, Oct 20, 2021 at 12:27 PM Jeff Layton <jlayton@kernel.org> wrote:
> > > > > > > On Wed, 2021-10-20 at 15:37 +0100, Luís Henriques wrote:
> > > > > > > > This counter will keep track of the number of remote object copies done on
> > > > > > > > copy_file_range syscalls. This counter will be filesystem per-client, and
> > > > > > > > can be accessed from the client debugfs directory.
> > > > > > > >
> > > > > > > > Cc: Patrick Donnelly <pdonnell@redhat.com>
> > > > > > > > Signed-off-by: Luís Henriques <lhenriques@suse.de>
> > > > > > > > ---
> > > > > > > > This is an RFC to reply to Patrick's request in [0]. Note that I'm not
> > > > > > > > 100% sure about the usefulness of this patch, or if this is the best way
> > > > > > > > to provide the functionality Patrick requested. Anyway, this is just to
> > > > > > > > get some feedback, hence the RFC.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > --
> > > > > > > > Luís
> > > > > > > >
> > > > > > > > [0] https://github.com/ceph/ceph/pull/42720
> > > > > > > >
> > > > > > > I think this would be better integrated into the stats infrastructure.
> > > > > > >
> > > > > > > Maybe you could add a new set of "copy" stats to struct
> > > > > > > ceph_client_metric that tracks the total copy operations done, their
> > > > > > > size and latency (similar to read and write ops)?
> > > > > > I think it's a good idea to integrate this into "stats" but I think a
> > > > > > local debugfs file for some counters is still useful. The "stats"
> > > > > > module is immature at this time and I'd rather not build any qa tests
> > > > > > (yet) that rely on it.
> > > > > >
> > > > > > Can we generalize this patch-set to a file named "op_counters" or
> > > > > > similar and additionally add other OSD ops performed by the kclient?
> > > > > >
> > > > >
> > > > > Tracking this sort of thing is the main purpose of the stats code. I'm
> > > > > really not keen on adding a whole separate set of files for reporting
> > > > > this.
> > > > Maybe I'm confused. Is there some "file" which is already used for
> > > > this type of debugging information? Or do you mean the code for
> > > > sending stats to the MDS to support cephfs-top?
> > > >
> > > > > What's the specific problem with relying on the data in debugfs
> > > > > "metrics" file?
> > > > Maybe no problem? I wasn't aware of a "metrics" file.
> > > >
> > > Yes. For instance:
> > >
> > > # cat /sys/kernel/debug/ceph/*/metrics
> > > item total
> > > ------------------------------------------
> > > opened files / total inodes 0 / 4
> > > pinned i_caps / total inodes 5 / 4
> > > opened inodes / total inodes 0 / 4
> > >
> > > item total avg_lat(us) min_lat(us) max_lat(us) stdev(us)
> > > -----------------------------------------------------------------------------------
> > > read 0 0 0 0 0
> > > write 5 914013 824797 1092343 103476
> > > metadata 79 12856 1572 114572 13262
> > >
> > > item total avg_sz(bytes) min_sz(bytes) max_sz(bytes) total_sz(bytes)
> > > ----------------------------------------------------------------------------------------
> > > read 0 0 0 0 0
> > > write 5 4194304 4194304 4194304 20971520
> > >
> > > item total miss hit
> > > -------------------------------------------------
> > > d_lease 11 0 29
> > > caps 5 68 10702
> > >
> > >
> > > I'm proposing that Luis add new lines for "copy" to go along with the
> > > "read" and "write" ones. The "total" counter should give you a count of
> > > the number of operations.
> > Okay that makes more sense!
> >
> > Side note: I am a bit horrified by how computer-unfriendly that
> > table-formatted data is.
>
> Any suggestion to improve this ?
>
> How about just make the "metric" file writable like a switch ? And as
> default it will show the data as above and if tools want the
> computer-friendly format, just write none-zero to it, then show raw data
> just like:
>
> # cat /sys/kernel/debug/ceph/*/metrics
> opened_files:0
> pinned_i_caps:5
> opened_inodes:0
> total_inodes:4
>
> read_latency:0,0,0,0,0
> write_latency:5,914013,824797,1092343,103476
> metadata_latency:79,12856,1572,114572,13262
>
> read_size:0,0,0,0,0
> write_size:5,4194304,4194304,4194304,20971520
>
> d_lease:11,0,29
> caps:5,68,10702
>
>
I'd rather not multiplex the output of this file based on some input.
That would also be rather hard to do -- write() and read() are two
different syscalls, so you'd need to track a bool (or something) across
them somehow.
Currently, I doubt there are many scripts in the field that scrape this
info and debugfs is specifically excluded from ABI concerns. If we want
to make it more machine-readable (which sounds like a good thing), then
I suggest we just change the output to something like what you have
above and not worry about preserving the "legacy" output.
--
Jeff Layton <jlayton@kernel.org>
next prev parent reply other threads:[~2021-10-26 11:40 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-20 14:37 [RFC PATCH] ceph: add remote object copy counter to fs client Luís Henriques
2021-10-20 16:27 ` Jeff Layton
2021-10-20 16:58 ` Luís Henriques
2021-10-21 13:52 ` Patrick Donnelly
2021-10-21 15:43 ` Jeff Layton
2021-10-21 16:18 ` Patrick Donnelly
2021-10-21 16:35 ` Jeff Layton
2021-10-21 17:30 ` Patrick Donnelly
2021-10-21 17:33 ` Jeff Layton
2021-10-26 3:05 ` Xiubo Li
2021-10-26 11:40 ` Jeff Layton [this message]
2021-10-26 15:31 ` Luís Henriques
2021-10-27 6:46 ` Xiubo Li
2021-10-27 10:01 ` [PATCH] ceph: split 'metric' debugfs file into several files Luís Henriques
2021-10-27 11:53 ` Jeff Layton
2021-10-27 12:00 ` Xiubo Li
2021-10-27 4:52 ` [RFC PATCH] ceph: add remote object copy counter to fs client Xiubo Li
2021-10-25 10:12 ` Luís Henriques
2021-10-25 10:20 ` Jeff Layton
2021-10-25 10:52 ` Luís Henriques
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=604199ed389d9286e3fdab6b5acdf65c421df45d.camel@kernel.org \
--to=jlayton@kernel.org \
--cc=ceph-devel@vger.kernel.org \
--cc=idryomov@gmail.com \
--cc=lhenriques@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=pdonnell@redhat.com \
--cc=xiubli@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).