* [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates
2019-09-03 11:14 [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Florian Schmidt
@ 2019-09-03 11:14 ` Florian Schmidt
2019-09-04 20:44 ` Daniel Jordan
2019-09-03 11:14 ` [PATCH 2/2] trace-vmscan-postprocess: fix output table spacing Florian Schmidt
2019-09-04 20:42 ` [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Daniel Jordan
2 siblings, 1 reply; 9+ messages in thread
From: Florian Schmidt @ 2019-09-03 11:14 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Andrew Morton, Daniel Jordan, Kirill Tkhai, linux-doc,
linux-kernel, Florian Schmidt
mm_vmscan_{direct_reclaim_begin,wakeup_kswapd,lru_isolate,lru_shrink_active}
changed their output to the point where the script throws warnings and
errors. Update it to be properly in line with those changes.
Signed-off-by: Florian Schmidt <florian.schmidt@nutanix.com>
---
.../postprocess/trace-vmscan-postprocess.pl | 23 +++++++++----------
1 file changed, 11 insertions(+), 12 deletions(-)
diff --git a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
index 995da15b16ca..6c4e3fde9447 100644
--- a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
+++ b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
@@ -107,14 +107,14 @@ GetOptions(
);
# Defaults for dynamically discovered regex's
-my $regex_direct_begin_default = 'order=([0-9]*) may_writepage=([0-9]*) gfp_flags=([A-Z_|]*)';
+my $regex_direct_begin_default = 'order=([0-9]*) gfp_flags=([A-Z_|]*)';
my $regex_direct_end_default = 'nr_reclaimed=([0-9]*)';
my $regex_kswapd_wake_default = 'nid=([0-9]*) order=([0-9]*)';
my $regex_kswapd_sleep_default = 'nid=([0-9]*)';
-my $regex_wakeup_kswapd_default = 'nid=([0-9]*) zid=([0-9]*) order=([0-9]*) gfp_flags=([A-Z_|]*)';
-my $regex_lru_isolate_default = 'isolate_mode=([0-9]*) classzone_idx=([0-9]*) order=([0-9]*) nr_requested=([0-9]*) nr_scanned=([0-9]*) nr_skipped=([0-9]*) nr_taken=([0-9]*) lru=([a-z_]*)';
+my $regex_wakeup_kswapd_default = 'nid=([0-9]*) order=([0-9]*) gfp_flags=([A-Z_|]*)';
+my $regex_lru_isolate_default = 'isolate_mode=([0-9]*) classzone=([0-9]*) order=([0-9]*) nr_requested=([0-9]*) nr_scanned=([0-9]*) nr_skipped=([0-9]*) nr_taken=([0-9]*) lru=([a-z_]*)';
my $regex_lru_shrink_inactive_default = 'nid=([0-9]*) nr_scanned=([0-9]*) nr_reclaimed=([0-9]*) nr_dirty=([0-9]*) nr_writeback=([0-9]*) nr_congested=([0-9]*) nr_immediate=([0-9]*) nr_activate_anon=([0-9]*) nr_activate_file=([0-9]*) nr_ref_keep=([0-9]*) nr_unmap_fail=([0-9]*) priority=([0-9]*) flags=([A-Z_|]*)';
-my $regex_lru_shrink_active_default = 'lru=([A-Z_]*) nr_scanned=([0-9]*) nr_rotated=([0-9]*) priority=([0-9]*)';
+my $regex_lru_shrink_active_default = 'nid=([0-9]*) nr_active=([0-9]*) nr_deactivated=([0-9]*) nr_referenced=([0-9]*) priority=([0-9]*) flags=([A-Z_]*)';
my $regex_writepage_default = 'page=([0-9a-f]*) pfn=([0-9]*) flags=([A-Z_|]*)';
# Dyanically discovered regex
@@ -184,8 +184,7 @@ sub generate_traceevent_regex {
$regex_direct_begin = generate_traceevent_regex(
"vmscan/mm_vmscan_direct_reclaim_begin",
$regex_direct_begin_default,
- "order", "may_writepage",
- "gfp_flags");
+ "order", "gfp_flags");
$regex_direct_end = generate_traceevent_regex(
"vmscan/mm_vmscan_direct_reclaim_end",
$regex_direct_end_default,
@@ -201,11 +200,11 @@ $regex_kswapd_sleep = generate_traceevent_regex(
$regex_wakeup_kswapd = generate_traceevent_regex(
"vmscan/mm_vmscan_wakeup_kswapd",
$regex_wakeup_kswapd_default,
- "nid", "zid", "order", "gfp_flags");
+ "nid", "order", "gfp_flags");
$regex_lru_isolate = generate_traceevent_regex(
"vmscan/mm_vmscan_lru_isolate",
$regex_lru_isolate_default,
- "isolate_mode", "classzone_idx", "order",
+ "isolate_mode", "classzone", "order",
"nr_requested", "nr_scanned", "nr_skipped", "nr_taken",
"lru");
$regex_lru_shrink_inactive = generate_traceevent_regex(
@@ -218,9 +217,9 @@ $regex_lru_shrink_inactive = generate_traceevent_regex(
$regex_lru_shrink_active = generate_traceevent_regex(
"vmscan/mm_vmscan_lru_shrink_active",
$regex_lru_shrink_active_default,
- "nid", "zid",
- "lru",
- "nr_scanned", "nr_rotated", "priority");
+ "nid",
+ "nr_taken", "nr_active", "nr_deactivated", "nr_referenced",
+ "priority", "flags");
$regex_writepage = generate_traceevent_regex(
"vmscan/mm_vmscan_writepage",
$regex_writepage_default,
@@ -371,7 +370,7 @@ EVENT_PROCESS:
print " $regex_wakeup_kswapd\n";
next;
}
- my $order = $3;
+ my $order = $2;
$perprocesspid{$process_pid}->{MM_VMSCAN_WAKEUP_KSWAPD_PERORDER}[$order]++;
} elsif ($tracepoint eq "mm_vmscan_lru_isolate") {
$details = $6;
--
2.23.0.rc1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates
2019-09-03 11:14 ` [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates Florian Schmidt
@ 2019-09-04 20:44 ` Daniel Jordan
2019-09-11 10:32 ` Florian Schmidt
0 siblings, 1 reply; 9+ messages in thread
From: Daniel Jordan @ 2019-09-04 20:44 UTC (permalink / raw)
To: Florian Schmidt
Cc: Jonathan Corbet, Andrew Morton, Daniel Jordan, Kirill Tkhai,
linux-doc, linux-kernel, Yafang Shao
On Tue, Sep 03, 2019 at 11:14:12AM +0000, Florian Schmidt wrote:
> mm_vmscan_{direct_reclaim_begin,wakeup_kswapd,lru_isolate,lru_shrink_active}
> changed their output to the point where the script throws warnings and
> errors. Update it to be properly in line with those changes.
Could use the appropriate Fixes tags here.
> Signed-off-by: Florian Schmidt <florian.schmidt@nutanix.com>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates
2019-09-04 20:44 ` Daniel Jordan
@ 2019-09-11 10:32 ` Florian Schmidt
0 siblings, 0 replies; 9+ messages in thread
From: Florian Schmidt @ 2019-09-11 10:32 UTC (permalink / raw)
To: Daniel Jordan
Cc: Jonathan Corbet, Andrew Morton, Kirill Tkhai, linux-doc,
linux-kernel, Yafang Shao
Hi Daniel,
On 04/09/2019 21:44, Daniel Jordan wrote:
> On Tue, Sep 03, 2019 at 11:14:12AM +0000, Florian Schmidt wrote:
>> mm_vmscan_{direct_reclaim_begin,wakeup_kswapd,lru_isolate,lru_shrink_active}
>> changed their output to the point where the script throws warnings and
>> errors. Update it to be properly in line with those changes.
>
> Could use the appropriate Fixes tags here.
Good point, I'll add
Fixes: 3481c37ffa1d ("mm/vmscan: drop may_writepage and classzone_idx
from direct reclaim begin template")
Fixes: 3b775998eca7 ("include/trace/events/vmscan.h: drop zone id from
kswapd tracepoints")
in v2.
Thanks for the feedback,
Florian
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 2/2] trace-vmscan-postprocess: fix output table spacing
2019-09-03 11:14 [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Florian Schmidt
2019-09-03 11:14 ` [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates Florian Schmidt
@ 2019-09-03 11:14 ` Florian Schmidt
2019-09-04 20:42 ` [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Daniel Jordan
2 siblings, 0 replies; 9+ messages in thread
From: Florian Schmidt @ 2019-09-03 11:14 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Andrew Morton, Daniel Jordan, Kirill Tkhai, linux-doc,
linux-kernel, Florian Schmidt
Fix spacing so that both the headers in themselves, as well as the
output of the two tables related to each other, are properly aligned.
Signed-off-by: Florian Schmidt <florian.schmidt@nutanix.com>
---
Documentation/trace/postprocess/trace-vmscan-postprocess.pl | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
index 6c4e3fde9447..5a5d70029c41 100644
--- a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
+++ b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl
@@ -575,8 +575,8 @@ sub dump_stats {
# Print out kswapd activity
printf("\n");
- printf("%-" . $max_strlen . "s %8s %10s %8s %8s %8s %8s\n", "Kswapd", "Kswapd", "Order", "Pages", "Pages", "Pages", "Pages");
- printf("%-" . $max_strlen . "s %8s %10s %8s %8s %8s %8s\n", "Instance", "Wakeups", "Re-wakeup", "Scanned", "Rclmed", "Sync-IO", "ASync-IO");
+ printf("%-" . $max_strlen . "s %8s %10s %8s %8s %8s %8s\n", "Kswapd", "Kswapd", "Order", "Pages", "Pages", "Pages", "Pages");
+ printf("%-" . $max_strlen . "s %8s %10s %8s %8s %8s %8s\n", "Instance", "Wakeups", "Re-wakeup", "Scanned", "Rclmed", "Sync-IO", "ASync-IO");
foreach $process_pid (keys %stats) {
if (!$stats{$process_pid}->{MM_VMSCAN_KSWAPD_WAKE}) {
@@ -595,7 +595,7 @@ sub dump_stats {
$total_kswapd_writepage_file_async += $stats{$process_pid}->{MM_VMSCAN_WRITEPAGE_FILE_ASYNC};
$total_kswapd_writepage_anon_async += $stats{$process_pid}->{MM_VMSCAN_WRITEPAGE_ANON_ASYNC};
- printf("%-" . $max_strlen . "s %8d %10d %8u %8u %8i %8u",
+ printf("%-" . $max_strlen . "s %8d %10d %8u %8u %8i %8u ",
$process_pid,
$stats{$process_pid}->{MM_VMSCAN_KSWAPD_WAKE},
$stats{$process_pid}->{HIGH_KSWAPD_REWAKEUP},
--
2.23.0.rc1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output
2019-09-03 11:14 [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Florian Schmidt
2019-09-03 11:14 ` [PATCH 1/2] trace-vmscan-postprocess: sync with tracepoints updates Florian Schmidt
2019-09-03 11:14 ` [PATCH 2/2] trace-vmscan-postprocess: fix output table spacing Florian Schmidt
@ 2019-09-04 20:42 ` Daniel Jordan
2019-09-05 4:32 ` Yafang Shao
2019-09-11 10:30 ` Florian Schmidt
2 siblings, 2 replies; 9+ messages in thread
From: Daniel Jordan @ 2019-09-04 20:42 UTC (permalink / raw)
To: Florian Schmidt
Cc: Jonathan Corbet, Andrew Morton, Daniel Jordan, Kirill Tkhai,
linux-doc, linux-kernel, Yafang Shao
On Tue, Sep 03, 2019 at 11:14:07AM +0000, Florian Schmidt wrote:
> This patch series updates trace-vmscan-postprocess.pl to work without
> throwing warnings and errors which stem from updates to several trace
> points.
Cc Yafang, who made (most of?) these updates.
> 3481c37ffa1d ("mm/vmscan: drop may_writepage and classzone_idx from
> direct reclaim begin template") removed "may_writepage" from
> mm_vmscan_direct_reclaim_begin, and 3b775998eca7
> ("include/trace/events/vmscan.h: drop zone id from kswapd tracepoints")
> removed "zid" from mm_vmscan_wakeup_kswapd. The output of
> mm_vmscan_lru_isolate and mm_vmscan_lru_shrink_active seems to never
> have matched the format of the trace point output since they were
> created, or at least for as long as I can tell. Patch 1 aligns the
> format parsing of the perl script with the current output of the trace
> points.
Thanks, patch 1 fixes the script for me for all tracepoints you touched.
> In addition, the tables that are printed by the script were not properly
> aligned any more, so patch 2 fixes the spacing.
Nit, not for Pages Scanned. With your series I get
Kswapd Kswapd Order Pages Pages Pages Pages
Instance Wakeups Re-wakeup Scanned Rclmed Sync-IO ASync-IO
kswapd0-175 1 0 253694 253691 3 129896 wake-0=1
> A side remark: parsing the trace output for mm_vmscan_lru_shrink_active
> has been in the script ever since it was created in 2010, but at no
> point the parsed output was ever used for anything. I updated the
> parsing code now, but I wonder if we could just get rid of that part...
I wonder if we shouldn't just get rid of the whole script, it's hard to
remember to keep in sync with vmscan changes and I can't think of a way to
remedy that short of having mm regression tests that run this. But your
patches are an improvement for now.
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output
2019-09-04 20:42 ` [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Daniel Jordan
@ 2019-09-05 4:32 ` Yafang Shao
2019-09-05 15:27 ` Daniel Jordan
2019-09-11 10:30 ` Florian Schmidt
1 sibling, 1 reply; 9+ messages in thread
From: Yafang Shao @ 2019-09-05 4:32 UTC (permalink / raw)
To: Daniel Jordan
Cc: Florian Schmidt, Jonathan Corbet, Andrew Morton, Kirill Tkhai,
linux-doc, linux-kernel
On Thu, Sep 5, 2019 at 4:42 AM Daniel Jordan <daniel.m.jordan@oracle.com> wrote:
>
> On Tue, Sep 03, 2019 at 11:14:07AM +0000, Florian Schmidt wrote:
> > This patch series updates trace-vmscan-postprocess.pl to work without
> > throwing warnings and errors which stem from updates to several trace
> > points.
>
> Cc Yafang, who made (most of?) these updates.
>
Yes, I made 3481c37ffa1d and 3b775998eca7 but didn't remeber to update
the scripts in the Document directory.
Thanks for improving it.
> > 3481c37ffa1d ("mm/vmscan: drop may_writepage and classzone_idx from
> > direct reclaim begin template") removed "may_writepage" from
> > mm_vmscan_direct_reclaim_begin, and 3b775998eca7
> > ("include/trace/events/vmscan.h: drop zone id from kswapd tracepoints")
> > removed "zid" from mm_vmscan_wakeup_kswapd. The output of
> > mm_vmscan_lru_isolate and mm_vmscan_lru_shrink_active seems to never
> > have matched the format of the trace point output since they were
> > created, or at least for as long as I can tell. Patch 1 aligns the
> > format parsing of the perl script with the current output of the trace
> > points.
>
> Thanks, patch 1 fixes the script for me for all tracepoints you touched.
>
> > In addition, the tables that are printed by the script were not properly
> > aligned any more, so patch 2 fixes the spacing.
>
> Nit, not for Pages Scanned. With your series I get
>
> Kswapd Kswapd Order Pages Pages Pages Pages
> Instance Wakeups Re-wakeup Scanned Rclmed Sync-IO ASync-IO
> kswapd0-175 1 0 253694 253691 3 129896 wake-0=1
>
> > A side remark: parsing the trace output for mm_vmscan_lru_shrink_active
> > has been in the script ever since it was created in 2010, but at no
> > point the parsed output was ever used for anything. I updated the
> > parsing code now, but I wonder if we could just get rid of that part...
>
> I wonder if we shouldn't just get rid of the whole script, it's hard to
> remember to keep in sync with vmscan changes and I can't think of a way to
> remedy that short of having mm regression tests that run this.
There are some similar scripts under tools/perf/scripts/, i.e.
compaction-times.py.
What about intergrating these vmscan scripts into perf/scripts as well ?
Something like vmscan-times.py...
> But your
> patches are an improvement for now.
Agreed.
Thanks
Yafang
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output
2019-09-05 4:32 ` Yafang Shao
@ 2019-09-05 15:27 ` Daniel Jordan
0 siblings, 0 replies; 9+ messages in thread
From: Daniel Jordan @ 2019-09-05 15:27 UTC (permalink / raw)
To: Yafang Shao
Cc: Daniel Jordan, Florian Schmidt, Jonathan Corbet, Andrew Morton,
Kirill Tkhai, linux-doc, linux-kernel
On Thu, Sep 05, 2019 at 12:32:49PM +0800, Yafang Shao wrote:
> On Thu, Sep 5, 2019 at 4:42 AM Daniel Jordan <daniel.m.jordan@oracle.com> wrote:
> > I wonder if we shouldn't just get rid of the whole script, it's hard to
> > remember to keep in sync with vmscan changes and I can't think of a way to
> > remedy that short of having mm regression tests that run this.
>
> There are some similar scripts under tools/perf/scripts/, i.e.
> compaction-times.py.
> What about intergrating these vmscan scripts into perf/scripts as well ?
> Something like vmscan-times.py...
Could be done, but I don't see how that makes it easier to keep in
sync...unless perf's tests are run regularly.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output
2019-09-04 20:42 ` [PATCH 0/2] trace-vmscan-postprocess: fix parsing and output Daniel Jordan
2019-09-05 4:32 ` Yafang Shao
@ 2019-09-11 10:30 ` Florian Schmidt
1 sibling, 0 replies; 9+ messages in thread
From: Florian Schmidt @ 2019-09-11 10:30 UTC (permalink / raw)
To: Daniel Jordan
Cc: Jonathan Corbet, Andrew Morton, Kirill Tkhai, linux-doc,
linux-kernel, Yafang Shao
Hi Daniel,
On 04/09/2019 21:42, Daniel Jordan wrote:
>> In addition, the tables that are printed by the script were not properly
>> aligned any more, so patch 2 fixes the spacing.
>
> Nit, not for Pages Scanned. With your series I get
>
> Kswapd Kswapd Order Pages Pages Pages Pages
> Instance Wakeups Re-wakeup Scanned Rclmed Sync-IO ASync-IO
> kswapd0-175 1 0 253694 253691 3 129896 wake-0=1
Whoops, you're right, I'll fix that in v2.
> I wonder if we shouldn't just get rid of the whole script, it's hard to
> remember to keep in sync with vmscan changes and I can't think of a way to
> remedy that short of having mm regression tests that run this. But your
> patches are an improvement for now.
That's definitely one possibility. If history has shown that these break
and aren't properly kept in line, they could go. Alternatively, as
Yafang suggested, integrating them into the perf test suite might help
with the issues.
I'll ignore that point for now though and just send a v2 for this patch
series.
Thanks for the feedback,
Florian
^ permalink raw reply [flat|nested] 9+ messages in thread