From: Kimberly Brown <kimbrownkd@gmail.com>
To: Michael Kelley <mikelley@microsoft.com>,
Long Li <longli@microsoft.com>,
Sasha Levin <Alexander.Levin@microsoft.com>,
Dexuan Cui <decui@microsoft.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Stephen Hemminger <sthemmin@microsoft.com>,
devel@linuxdriverproject.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2] Drivers: hv: vmbus: Expose counters for interrupts and full conditions
Date: Fri, 4 Jan 2019 23:35:18 -0500 [thread overview]
Message-ID: <20190105043518.GA4072@ubu-Virtual-Machine> (raw)
In-Reply-To: <0181213190901.GA3843@ubu-Virtual-Machine>
Counter values for per-channel interrupts and ring buffer full
conditions are useful for investigating performance.
Expose counters in sysfs for 2 types of guest to host interrupts:
1) Interrupts caused by the channel's outbound ring buffer transitioning
from empty to not empty
2) Interrupts caused by the channel's inbound ring buffer transitioning
from full to not full while a packet is waiting for enough buffer space to
become available
Expose 2 counters in sysfs for the number of times that write operations
encountered a full outbound ring buffer:
1) The total number of write operations that encountered a full
condition
2) The number of write operations that were the first to encounter a
full condition
I tested this patch by confirming that the sysfs files were created and
observing the counter values. The values seemed to increase by a
reasonable amount when the Hyper-v related drivers were in use.
Signed-off-by: Kimberly Brown <kimbrownkd@gmail.com>
---
Changes in v2:
- Added mailing lists to the cc list
- Removed the host to guest interrupt counters proposed in v1 because
they were not accurate
- Added full condition counters for the channel's outbound ring buffer
Documentation/ABI/stable/sysfs-bus-vmbus | 33 ++++++++++++++++++++++++
drivers/hv/ring_buffer.c | 14 +++++++++-
drivers/hv/vmbus_drv.c | 32 +++++++++++++++++++++++
include/linux/hyperv.h | 32 +++++++++++++++++++++++
4 files changed, 110 insertions(+), 1 deletion(-)
diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus b/Documentation/ABI/stable/sysfs-bus-vmbus
index 3fed8fdb873d..31dc89989032 100644
--- a/Documentation/ABI/stable/sysfs-bus-vmbus
+++ b/Documentation/ABI/stable/sysfs-bus-vmbus
@@ -146,3 +146,36 @@ KernelVersion: 4.16
Contact: Stephen Hemminger <sthemmin@microsoft.com>
Description: Binary file created by uio_hv_generic for ring buffer
Users: Userspace drivers
+
+What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/intr_in_full
+Date: January 2019
+KernelVersion: 4.21
+Contact: Michael Kelley <mikelley@microsoft.com>
+Description: Number of guest to host interrupts caused by the inbound ring
+ buffer transitioning from full to not full while a packet is
+ waiting for buffer space to become available
+Users: Debugging tools
+
+What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/intr_out_empty
+Date: January 2019
+KernelVersion: 4.21
+Contact: Michael Kelley <mikelley@microsoft.com>
+Description: Number of guest to host interrupts caused by the outbound ring
+ buffer transitioning from empty to not empty
+Users: Debugging tools
+
+What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/out_full_first
+Date: January 2019
+KernelVersion: 4.21
+Contact: Michael Kelley <mikelley@microsoft.com>
+Description: Number of write operations that were the first to encounter an
+ outbound ring buffer full condition
+Users: Debugging tools
+
+What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/out_full_total
+Date: January 2019
+KernelVersion: 4.21
+Contact: Michael Kelley <mikelley@microsoft.com>
+Description: Total number of write operations that encountered an outbound
+ ring buffer full condition
+Users: Debugging tools
diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
index 64d0c85d5161..be2cbd0bea7d 100644
--- a/drivers/hv/ring_buffer.c
+++ b/drivers/hv/ring_buffer.c
@@ -74,8 +74,10 @@ static void hv_signal_on_write(u32 old_write, struct vmbus_channel *channel)
* This is the only case we need to signal when the
* ring transitions from being empty to non-empty.
*/
- if (old_write == READ_ONCE(rbi->ring_buffer->read_index))
+ if (old_write == READ_ONCE(rbi->ring_buffer->read_index)) {
+ ++channel->intr_out_empty;
vmbus_setevent(channel);
+ }
}
/* Get the next write location for the specified ring buffer. */
@@ -273,10 +275,19 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
* is empty since the read index == write index.
*/
if (bytes_avail_towrite <= totalbytes_towrite) {
+ ++channel->out_full_total;
+
+ if (!channel->out_full_flag) {
+ ++channel->out_full_first;
+ channel->out_full_flag = true;
+ }
+
spin_unlock_irqrestore(&outring_info->ring_lock, flags);
return -EAGAIN;
}
+ channel->out_full_flag = false;
+
/* Write to the ring buffer */
next_write_location = hv_get_next_write_location(outring_info);
@@ -531,6 +542,7 @@ void hv_pkt_iter_close(struct vmbus_channel *channel)
if (curr_write_sz <= pending_sz)
return;
+ ++channel->intr_in_full;
vmbus_setevent(channel);
}
EXPORT_SYMBOL_GPL(hv_pkt_iter_close);
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 283d184280af..460b7f75251c 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1445,6 +1445,34 @@ static ssize_t channel_events_show(const struct vmbus_channel *channel, char *bu
}
static VMBUS_CHAN_ATTR(events, S_IRUGO, channel_events_show, NULL);
+static ssize_t channel_intr_in_full_show(const struct vmbus_channel *channel,
+ char *buf)
+{
+ return sprintf(buf, "%llu\n", channel->intr_in_full);
+}
+static VMBUS_CHAN_ATTR(intr_in_full, 0444, channel_intr_in_full_show, NULL);
+
+static ssize_t channel_intr_out_empty_show(const struct vmbus_channel *channel,
+ char *buf)
+{
+ return sprintf(buf, "%llu\n", channel->intr_out_empty);
+}
+static VMBUS_CHAN_ATTR(intr_out_empty, 0444, channel_intr_out_empty_show, NULL);
+
+static ssize_t channel_out_full_first_show(const struct vmbus_channel *channel,
+ char *buf)
+{
+ return sprintf(buf, "%llu\n", channel->out_full_first);
+}
+static VMBUS_CHAN_ATTR(out_full_first, 0444, channel_out_full_first_show, NULL);
+
+static ssize_t channel_out_full_total_show(const struct vmbus_channel *channel,
+ char *buf)
+{
+ return sprintf(buf, "%llu\n", channel->out_full_total);
+}
+static VMBUS_CHAN_ATTR(out_full_total, 0444, channel_out_full_total_show, NULL);
+
static ssize_t subchannel_monitor_id_show(const struct vmbus_channel *channel,
char *buf)
{
@@ -1470,6 +1498,10 @@ static struct attribute *vmbus_chan_attrs[] = {
&chan_attr_latency.attr,
&chan_attr_interrupts.attr,
&chan_attr_events.attr,
+ &chan_attr_intr_in_full.attr,
+ &chan_attr_intr_out_empty.attr,
+ &chan_attr_out_full_first.attr,
+ &chan_attr_out_full_total.attr,
&chan_attr_monitor_id.attr,
&chan_attr_subchannel_id.attr,
NULL
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index b3e24368930a..673f58c4bb1d 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -751,6 +751,27 @@ struct vmbus_channel {
u64 interrupts; /* Host to Guest interrupts */
u64 sig_events; /* Guest to Host events */
+ /* Interrupt counts for 2 types of Guest to Host interrupts */
+ u64 intr_in_full; /* in ring buffer, full to not full */
+ u64 intr_out_empty; /* out ring buffer, empty to not empty */
+
+ /*
+ * The total number of write operations that encountered a full
+ * outbound ring buffer.
+ */
+ u64 out_full_total;
+ /*
+ * The number of write operations that were the first to encounter a
+ * full outbound ring buffer.
+ */
+ u64 out_full_first;
+ /*
+ * Indicates that a full outbound ring buffer was encountered. The flag
+ * is set to true when a full outbound ring buffer is encountered and
+ * set to false when a write to the outbound ring buffer is completed.
+ */
+ bool out_full_flag;
+
/* Channel callback's invoked in softirq context */
struct tasklet_struct callback_event;
void (*onchannel_callback)(void *context);
@@ -938,6 +959,17 @@ static inline void *get_per_channel_state(struct vmbus_channel *c)
static inline void set_channel_pending_send_size(struct vmbus_channel *c,
u32 size)
{
+ if (size) {
+ ++c->out_full_total;
+
+ if (!c->out_full_flag) {
+ ++c->out_full_first;
+ c->out_full_flag = true;
+ }
+ } else {
+ c->out_full_flag = false;
+ }
+
c->outbound.ring_buffer->pending_send_sz = size;
}
--
2.17.1
next parent reply other threads:[~2019-01-05 4:35 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <0181213190901.GA3843@ubu-Virtual-Machine>
2019-01-05 4:35 ` Kimberly Brown [this message]
2019-01-05 21:01 ` [PATCH v2] Drivers: hv: vmbus: Expose counters for interrupts and full conditions Michael Kelley
2019-01-10 3:58 ` Dexuan Cui
2019-01-10 3:46 ` Dexuan Cui
2019-01-17 4:37 ` [PATCH v3] " Kimberly Brown
2019-01-17 6:45 ` Dexuan Cui
2019-01-17 17:11 ` Stephen Hemminger
2019-01-21 16:29 ` Kimberly Brown
2019-01-17 16:04 ` Michael Kelley
2019-02-04 7:13 ` [PATCH v4] " Kimberly Brown
2019-02-04 16:25 ` Michael Kelley
2019-02-15 1:57 ` Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190105043518.GA4072@ubu-Virtual-Machine \
--to=kimbrownkd@gmail.com \
--cc=Alexander.Levin@microsoft.com \
--cc=decui@microsoft.com \
--cc=devel@linuxdriverproject.org \
--cc=haiyangz@microsoft.com \
--cc=kys@microsoft.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=mikelley@microsoft.com \
--cc=sthemmin@microsoft.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).