KVM Archive on lore.kernel.org
 help / color / Atom feed
From: Paolo Bonzini <pbonzini@redhat.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: KVM list <kvm@vger.kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Christian Borntraeger <borntraeger@de.ibm.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Peter Feiner <pfeiner@google.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: "statsfs" API design
Date: Sun, 10 Nov 2019 14:04:58 +0100
Message-ID: <cb52053e-eac0-cbb9-1633-646c7f71b8b3@redhat.com> (raw)
In-Reply-To: <20191109154952.GA1365674@kroah.com>

On 09/11/19 16:49, Greg Kroah-Hartman wrote:
> On Wed, Nov 06, 2019 at 04:56:25PM +0100, Paolo Bonzini wrote:
>> Hi all,
>>
>> statsfs is a proposal for a new Linux kernel synthetic filesystem, to be
>> mounted in /sys/kernel/stats, which exposes subsystem-level statistics
>> in sysfs.  Reading need not be particularly lightweight, but writing
>> must be fast.  Therefore, statistics are gathered at a fine-grain level
>> in order to avoid locking or atomic operations, and then aggregated by
>> statsfs until the desired granularity.
> 
> Wait, reading a statistic from userspace can be slow, but writing to it
> from userspace has to be fast?  Or do you mean the speed is all for
> reading/writing the value within the kernel?

Reading/writing from the kernel.  Reads from a userspace are a superset
of reading from the kernel, writes from userspace are irrelevant.

[...]

>> As you can see, values are basically integers stored somewhere in a
>> struct.   The statsfs_value struct also includes information on which
>> operations (for example sum, min, max, average, count nonzero) it makes
>> sense to expose when the values are aggregated.
> 
> What can userspace do with that info?

The basic usage is logging.  A turbostat-like tool that is able to use
both debugfs and tracepoints is already in tools/kvm/kvm_stat.

> I have some old notes somewhere about what people really want when it
> comes to a good "statistics" datatype, that I was thinking of building
> off of, but that seems independant of what you are doing here, right?
> This is just exporting existing values to userspace in a semi-sane way?

For KVM yes.  But while I'm at it, I'd like the subsystem to be useful
for others so if you can dig out those notes I can integrate that.

> Anyway, I like the idea, but what about how this is exposed to
> userspace?  The criticism of sysfs for statistics is that it is too slow
> to open/read/close lots of files and tough to get "at this moment in
> time these are all the different values" snapshots easily.  How will
> this be addressed here?

Individual files in sysfs *should* be the first format to export
statsfs, since quick scripts are an important usecase.  However, another
advantage of having a higher-level API is that other ways to access the
stats can be added transparently.

The main requirement for that is self-descriptiveness, blindly passing
structs to userspace is certainly the worst format of all.  But I don't
like the idea of JSON or anything ASCII; that adds overhead to both
production and parsing, for no particular reason.   Tracepoints already
do something like that to export arguments, so perhaps there is room to
reuse code or at least some ideas.  It could be binary sysfs files
(again like tracing) or netlink, I haven't thought about it at all.

Paolo


      reply index

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-06 15:56 Paolo Bonzini
2019-11-09 15:49 ` Greg Kroah-Hartman
2019-11-10 13:04   ` Paolo Bonzini [this message]

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cb52053e-eac0-cbb9-1633-646c7f71b8b3@redhat.com \
    --to=pbonzini@redhat.com \
    --cc=alex.williamson@redhat.com \
    --cc=borntraeger@de.ibm.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pfeiner@google.com \
    --cc=rostedt@goodmis.org \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git