* [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes
@ 2023-08-04 6:50 chengming.zhou
2023-08-04 6:50 ` [PATCH 2/3] iocost_monitor: print vrate inuse along with base_vrate chengming.zhou
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: chengming.zhou @ 2023-08-04 6:50 UTC (permalink / raw)
To: axboe, tj; +Cc: linux-kernel, zhouchengming
From: Chengming Zhou <zhouchengming@bytedance.com>
When I use iocost_monitor on nvme0n1, this error shows up:
"Could not find ioc for nvme0n1"
There is no kobj in struct queue in recent kernel, it seems that the commit
2bd85221a625 ("block: untangle request_queue refcounting from sysfs")
move the queue kobj to struct gendisk.
Fix it by using mq_kobj which is at the same level with queue kobj.
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
---
tools/cgroup/iocost_monitor.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/cgroup/iocost_monitor.py b/tools/cgroup/iocost_monitor.py
index 0dbbc67400fc..7aa076cb559e 100644
--- a/tools/cgroup/iocost_monitor.py
+++ b/tools/cgroup/iocost_monitor.py
@@ -221,7 +221,7 @@ ioc = None
for i, ptr in radix_tree_for_each(blkcg_root.blkg_tree.address_of_()):
blkg = drgn.Object(prog, 'struct blkcg_gq', address=ptr)
try:
- if devname == blkg.q.kobj.parent.name.string_().decode('utf-8'):
+ if devname == blkg.q.mq_kobj.parent.name.string_().decode('utf-8'):
q_id = blkg.q.id.value_()
if blkg.pd[plid]:
root_iocg = container_of(blkg.pd[plid], 'struct ioc_gq', 'pd')
--
2.41.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/3] iocost_monitor: print vrate inuse along with base_vrate
2023-08-04 6:50 [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes chengming.zhou
@ 2023-08-04 6:50 ` chengming.zhou
2023-08-04 6:50 ` [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms chengming.zhou
2023-08-08 21:46 ` [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes Jens Axboe
2 siblings, 0 replies; 5+ messages in thread
From: chengming.zhou @ 2023-08-04 6:50 UTC (permalink / raw)
To: axboe, tj; +Cc: linux-kernel, zhouchengming
From: Chengming Zhou <zhouchengming@bytedance.com>
The real vrate iocost inuse is not base_vrate, but the atomic vtime_rate.
We need iocost_monitor tool to display this real vrate that iocost use,
to check if the boosted compensated vrate is normal.
Effect after change:
nvme0n1 RUN per=50.0ms cur_per=172116.580:v1040587.433 busy= +0 \
vrate=135.00%:270.00% params=ssd_dfl(CQ)
^
|
this is real vrate inuse
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
---
tools/cgroup/iocost_monitor.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/cgroup/iocost_monitor.py b/tools/cgroup/iocost_monitor.py
index 7aa076cb559e..52ae9d1595b2 100644
--- a/tools/cgroup/iocost_monitor.py
+++ b/tools/cgroup/iocost_monitor.py
@@ -100,6 +100,7 @@ class IocStat:
self.period_at = ioc.period_at.value_() / 1_000_000
self.vperiod_at = ioc.period_at_vtime.value_() / VTIME_PER_SEC
self.vrate_pct = ioc.vtime_base_rate.value_() * 100 / VTIME_PER_USEC
+ self.ivrate_pct = ioc.vtime_rate.counter.value_() * 100 / VTIME_PER_USEC
self.busy_level = ioc.busy_level.value_()
self.autop_idx = ioc.autop_idx.value_()
self.user_cost_model = ioc.user_cost_model.value_()
@@ -119,7 +120,9 @@ class IocStat:
'period_at' : self.period_at,
'period_vtime_at' : self.vperiod_at,
'busy_level' : self.busy_level,
- 'vrate_pct' : self.vrate_pct, }
+ 'vrate_pct' : self.vrate_pct,
+ 'ivrate_pct' : self.ivrate_pct,
+ }
def table_preamble_str(self):
state = ('RUN' if self.running else 'IDLE') if self.enabled else 'OFF'
@@ -127,7 +130,7 @@ class IocStat:
f'per={self.period_ms}ms ' \
f'cur_per={self.period_at:.3f}:v{self.vperiod_at:.3f} ' \
f'busy={self.busy_level:+3} ' \
- f'vrate={self.vrate_pct:6.2f}% ' \
+ f'vrate={self.vrate_pct:6.2f}%:{self.ivrate_pct:6.2f}% ' \
f'params={self.autop_name}'
if self.user_cost_model or self.user_qos_params:
output += f'({"C" if self.user_cost_model else ""}{"Q" if self.user_qos_params else ""})'
--
2.41.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms
2023-08-04 6:50 [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes chengming.zhou
2023-08-04 6:50 ` [PATCH 2/3] iocost_monitor: print vrate inuse along with base_vrate chengming.zhou
@ 2023-08-04 6:50 ` chengming.zhou
2023-08-07 18:34 ` Tejun Heo
2023-08-08 21:46 ` [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes Jens Axboe
2 siblings, 1 reply; 5+ messages in thread
From: chengming.zhou @ 2023-08-04 6:50 UTC (permalink / raw)
To: axboe, tj; +Cc: linux-kernel, zhouchengming
From: Chengming Zhou <zhouchengming@bytedance.com>
The iocg can have three throttled metrics: wait, debt, delay. This patch
add missing wait_ms to IocgStat to show the latest wait_ms of iocg.
As we are here, group iocg usage percents "inflt%" and "usage%" together,
and group iocg throttled metrics "wait", "debt" and "delay" together.
Effect after changes:
nvme0n1 RUN per=50.0ms cur_per=177105.713:v1053528.587 busy= +0 vrate=135.00%:270.00% params=ssd_dfl(CQ)
active weight hweight% inflt% usage% wait debt delay
InterfererGroup0 * 100/ 100 54.28/ 9.09 0.34 24.07 0.00 0.00 0.00
interfered * 84/ 1000 45.72/ 90.91 0.48 41.09 0.00 0.00 0.00
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
---
tools/cgroup/iocost_monitor.py | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/tools/cgroup/iocost_monitor.py b/tools/cgroup/iocost_monitor.py
index 52ae9d1595b2..933c750b319b 100644
--- a/tools/cgroup/iocost_monitor.py
+++ b/tools/cgroup/iocost_monitor.py
@@ -138,7 +138,7 @@ class IocStat:
def table_header_str(self):
return f'{"":25} active {"weight":>9} {"hweight%":>13} {"inflt%":>6} ' \
- f'{"debt":>7} {"delay":>7} {"usage%"}'
+ f'{"usage%":>6} {"wait":>7} {"debt":>7} {"delay":>7}'
class IocgStat:
def __init__(self, iocg):
@@ -164,6 +164,8 @@ class IocgStat:
self.usage = (100 * iocg.usage_delta_us.value_() /
ioc.period_us.value_()) if self.active else 0
+ self.wait_ms = (iocg.stat.wait_us.value_() -
+ iocg.last_stat.wait_us.value_()) / 1000
self.debt_ms = iocg.abs_vdebt.value_() / VTIME_PER_USEC / 1000
if blkg.use_delay.counter.value_() != 0:
self.delay_ms = blkg.delay_nsec.counter.value_() / 1_000_000
@@ -180,9 +182,10 @@ class IocgStat:
'hweight_active_pct' : self.hwa_pct,
'hweight_inuse_pct' : self.hwi_pct,
'inflight_pct' : self.inflight_pct,
+ 'usage_pct' : self.usage,
+ 'wait_ms' : self.wait_ms,
'debt_ms' : self.debt_ms,
'delay_ms' : self.delay_ms,
- 'usage_pct' : self.usage,
'address' : self.address }
return out
@@ -192,9 +195,10 @@ class IocgStat:
f'{round(self.inuse):5}/{round(self.active):5} ' \
f'{self.hwi_pct:6.2f}/{self.hwa_pct:6.2f} ' \
f'{self.inflight_pct:6.2f} ' \
+ f'{min(self.usage, 999):6.2f} ' \
+ f'{self.wait_ms:7.2f} ' \
f'{self.debt_ms:7.2f} ' \
- f'{self.delay_ms:7.2f} '\
- f'{min(self.usage, 999):6.2f}'
+ f'{self.delay_ms:7.2f}'
out = out.rstrip(':')
return out
--
2.41.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms
2023-08-04 6:50 ` [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms chengming.zhou
@ 2023-08-07 18:34 ` Tejun Heo
0 siblings, 0 replies; 5+ messages in thread
From: Tejun Heo @ 2023-08-07 18:34 UTC (permalink / raw)
To: chengming.zhou; +Cc: axboe, linux-kernel, zhouchengming
On Fri, Aug 04, 2023 at 02:50:39PM +0800, chengming.zhou@linux.dev wrote:
> From: Chengming Zhou <zhouchengming@bytedance.com>
>
> The iocg can have three throttled metrics: wait, debt, delay. This patch
> add missing wait_ms to IocgStat to show the latest wait_ms of iocg.
>
> As we are here, group iocg usage percents "inflt%" and "usage%" together,
> and group iocg throttled metrics "wait", "debt" and "delay" together.
>
> Effect after changes:
>
> nvme0n1 RUN per=50.0ms cur_per=177105.713:v1053528.587 busy= +0 vrate=135.00%:270.00% params=ssd_dfl(CQ)
> active weight hweight% inflt% usage% wait debt delay
> InterfererGroup0 * 100/ 100 54.28/ 9.09 0.34 24.07 0.00 0.00 0.00
> interfered * 84/ 1000 45.72/ 90.91 0.48 41.09 0.00 0.00 0.00
>
> Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
For 1-3:
Acked-by: Tejun Heo <tj@kernel.org>
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes
2023-08-04 6:50 [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes chengming.zhou
2023-08-04 6:50 ` [PATCH 2/3] iocost_monitor: print vrate inuse along with base_vrate chengming.zhou
2023-08-04 6:50 ` [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms chengming.zhou
@ 2023-08-08 21:46 ` Jens Axboe
2 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2023-08-08 21:46 UTC (permalink / raw)
To: tj, chengming.zhou; +Cc: linux-kernel, zhouchengming
On Fri, 04 Aug 2023 14:50:37 +0800, chengming.zhou@linux.dev wrote:
> When I use iocost_monitor on nvme0n1, this error shows up:
> "Could not find ioc for nvme0n1"
>
> There is no kobj in struct queue in recent kernel, it seems that the commit
> 2bd85221a625 ("block: untangle request_queue refcounting from sysfs")
> move the queue kobj to struct gendisk.
>
> [...]
Applied, thanks!
[1/3] iocost_monitor: fix kernel queue kobj changes
commit: 2eae9c4912b6cfdfadcd4fa8ac26879e18a504a1
[2/3] iocost_monitor: print vrate inuse along with base_vrate
commit: 8e93c1acd15e6a754c19ef12f6e69641f37e267a
[3/3] iocost_monitor: improve it by adding iocg wait_ms
commit: 68392b002023cb6dadd3d5044268470a7201b313
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-08-08 21:46 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-04 6:50 [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes chengming.zhou
2023-08-04 6:50 ` [PATCH 2/3] iocost_monitor: print vrate inuse along with base_vrate chengming.zhou
2023-08-04 6:50 ` [PATCH 3/3] iocost_monitor: improve it by adding iocg wait_ms chengming.zhou
2023-08-07 18:34 ` Tejun Heo
2023-08-08 21:46 ` [PATCH 1/3] iocost_monitor: fix kernel queue kobj changes Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).