From: Tvrtko Ursulin <tursulin@ursulin.net> To: igt-dev@lists.freedesktop.org Cc: intel-gfx@lists.freedesktop.org Subject: [PATCH i-g-t 7/9] trace.pl: Fix incomplete request handling Date: Thu, 12 Jul 2018 11:59:56 +0100 [thread overview] Message-ID: <20180712105958.12953-8-tvrtko.ursulin@linux.intel.com> (raw) In-Reply-To: <20180712105958.12953-1-tvrtko.ursulin@linux.intel.com> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Incomplete requests (no notify, no context complete) have to be corrected by looking at the engine timeline, and not the sorted-by-start-time view as was previously used. Per-engine timelines are generated on demand and cached for later use. v2: Find end of current context on the engine timeline instead of just using the next request for adjusting the incomplete start time. v3: Improve scaling with large datasets by only walking each engine timeline once and some caching. (John Harrison) v4: * Fix logic fail from v3. * Refactor the code a bit to separate the stages better. * Do not account batches with unknown duration in avg stats. * Handle two user interrupts with the same seqno. * Handle user interrupt arriving after request_out. v5: * Update for class:instance. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: John Harrison <John.C.Harrison@intel.com> --- scripts/trace.pl | 163 +++++++++++++++++++++++++++++++---------------- 1 file changed, 108 insertions(+), 55 deletions(-) diff --git a/scripts/trace.pl b/scripts/trace.pl index 9bdafa2d7e99..33045f3264f2 100755 --- a/scripts/trace.pl +++ b/scripts/trace.pl @@ -462,16 +462,11 @@ while (<>) { die if exists $db{$key}->{'end'}; $db{$key}->{'end'} = $time; - if (exists $notify{$gkey}) { - $db{$key}->{'notify'} = $notify{$gkey}; - } else { - # No notify so far. Maybe it will arrive later which - # will be handled in the sanitation loop below. - $db{$key}->{'notify'} = $db{$key}->{'end'}; - $db{$key}->{'no-notify'} = 1; - } + $db{$key}->{'notify'} = $notify{$gkey} if exists $notify{$gkey}; } elsif ($tp_name eq 'i915:intel_engine_notify:') { - $notify{global_key($ring, $seqno)} = $time; + my $gkey = global_key($ring, $seqno); + + $notify{$gkey} = $time unless exists $notify{$gkey}; } elsif ($tp_name eq 'i915:intel_gpu_freq_change:') { push @freqs, [$prev_freq_ts, $time, $prev_freq] if $prev_freq; $prev_freq_ts = $time; @@ -484,66 +479,116 @@ while (<>) { my $max_seqno = 0; foreach my $key (keys %db) { my $gkey = global_key($db{$key}->{'ring'}, $db{$key}->{'global'}); - my $notify = $notify{$gkey}; die unless exists $db{$key}->{'start'}; $max_seqno = $db{$key}->{'seqno'} if $db{$key}->{'seqno'} > $max_seqno; - unless (exists $db{$key}->{'end'}) { - # Context complete not received. - $db{$key}->{'no-end'} = 1; + # Notify arrived after context complete? + $db{$key}->{'notify'} = $notify{$gkey} if not exists $db{$key}->{'notify'} + and exists $notify{$gkey}; - if (defined($notify)) { - # No context complete due req merging - use notify. - $db{$key}->{'notify'} = $notify; - $db{$key}->{'end'} = $notify; - } else { - # No notify and no context complete - give up for now. - $db{$key}->{'incomplete'} = 1; - } - } else { - # Notify arrived after context complete. - if (exists $db{$key}->{'no-notify'} and defined($notify)) { - delete $db{$key}->{'no-notify'}; - $db{$key}->{'notify'} = $notify; - } + # No notify but we have end? + $db{$key}->{'notify'} = $db{$key}->{'end'} if exists $db{$key}->{'end'} and + not exists $db{$key}->{'notify'}; + + # If user interrupt arrived out of order push it back to be no later + # than request out. + if (exists $db{$key}->{'end'} and exists $db{$key}->{'notify'} and + $db{$key}->{'notify'} > $db{$key}->{'end'}) { + $db{$key}->{'notify'} = $db{$key}->{'end'}; } } -# Fix up incompletes my $key_count = scalar(keys %db); -foreach my $key (keys %db) { - next unless exists $db{$key}->{'incomplete'}; - # End the incomplete batch at the time next one starts - my $ring = $db{$key}->{'ring'}; - my $ctx = $db{$key}->{'ctx'}; - my $seqno = $db{$key}->{'seqno'}; - my $next_key; - my $i = 1; - my $end; - - do { - $next_key = db_key($ring, $ctx, $seqno + $i); - $i++; - } until ((exists $db{$next_key} and not exists $db{$next_key}->{'incomplete'}) - or $i > $key_count); # ugly stop hack +my %engine_timelines; - if (exists $db{$next_key}) { - $end = $db{$next_key}->{'end'}; - } else { - # No info at all, fake it: - $end = $db{$key}->{'start'} + 999; - } +sub sortEngine { + my $as = $db{$a}->{'global'}; + my $bs = $db{$b}->{'global'}; + my $val; + + $val = $as <=> $bs; - $db{$key}->{'notify'} = $end; - $db{$key}->{'end'} = $end; + die if $val == 0; + + return $val; } -# GPU time accounting -my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count); -my (%submit_avg, %execute_avg, %ctxsave_avg); +sub get_engine_timeline { + my ($ring) = @_; + my @timeline; + + return $engine_timelines{$ring} if exists $engine_timelines{$ring}; + + @timeline = grep { $db{$_}->{'ring'} eq $ring } keys %db; + # FIXME seqno restart + @timeline = sort sortEngine @timeline; + + $engine_timelines{$ring} = \@timeline; + + return \@timeline; +} + +# Fix up coalesced requests by ending them either when the following same +# context request with known end ends, or when a different context starts. +foreach my $gid (sort keys %rings) { + my $ring = $ringmap{$rings{$gid}}; + my $timeline = get_engine_timeline($ring); + my $last_complete = -1; + my $last_ctx = -1; + my $complete; + + foreach my $pos (0..$#{$timeline}) { + my $key = @{$timeline}[$pos]; + my ($ctx, $end); + + next if exists $db{$key}->{'end'}; + + $db{$key}->{'no-end'} = 1; + $ctx = $db{$key}->{'ctx'}; + + if ($pos > $last_complete or $ctx != $last_ctx) { + my $next = $pos; + + undef $complete; + + while ($next < $#{$timeline}) { + my $next_key = ${$timeline}[++$next]; + if ($ctx == $db{$next_key}->{'ctx'} and + exists $db{$next_key}->{'end'}) { + $last_ctx = $db{$next_key}->{'ctx'}; + $last_complete = $next; + $complete = $next_key; + last; + } + } + } + + if (defined $complete) { + if ($ctx == $db{$complete}->{'ctx'}) { + $end = $db{$complete}->{'end'}; + } else { + $end = $db{$complete}->{'start'}; + } + } else { + # No next submission. Use notify if available or give up. + if (exists $db{$key}->{'notify'}) { + $end = $db{$key}->{'notify'}; + } else { + $end = $db{$key}->{'start'}; + $db{$key}->{'incomplete'} = 1; + } + } + + unless (exists $db{$key}->{'notify'}) { + $db{$key}->{'notify'} = $end; + $db{$key}->{'no-notify'} = 1; + } + $db{$key}->{'end'} = $end; + } +} sub sortStart { my $as = $db{$a}->{'start'}; @@ -592,6 +637,10 @@ foreach my $key (@sorted_keys) { @sorted_keys = sort sortStart keys %db if $re_sort; +# GPU time accounting +my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count); +my (%submit_avg, %execute_avg, %ctxsave_avg); + my $last_ts = 0; my $first_ts; my $min_ctx; @@ -614,7 +663,11 @@ foreach my $key (@sorted_keys) { } $db{$key}->{'execute-delay'} = $start - $db{$key}->{'submit'}; $db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'}; - $db{$key}->{'duration'} = $notify - $start; + unless (exists $db{$key}->{'no-notify'}) { + $db{$key}->{'duration'} = $notify - $start; + } else { + $db{$key}->{'duration'} = 0; + } $running{$ring} += $end - $start if $correct_durations or not exists $db{$key}->{'no-end'}; -- 2.17.1 _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
WARNING: multiple messages have this Message-ID (diff)
From: Tvrtko Ursulin <tursulin@ursulin.net> To: igt-dev@lists.freedesktop.org Cc: intel-gfx@lists.freedesktop.org, Tvrtko Ursulin <tvrtko.ursulin@intel.com> Subject: [igt-dev] [PATCH i-g-t 7/9] trace.pl: Fix incomplete request handling Date: Thu, 12 Jul 2018 11:59:56 +0100 [thread overview] Message-ID: <20180712105958.12953-8-tvrtko.ursulin@linux.intel.com> (raw) In-Reply-To: <20180712105958.12953-1-tvrtko.ursulin@linux.intel.com> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Incomplete requests (no notify, no context complete) have to be corrected by looking at the engine timeline, and not the sorted-by-start-time view as was previously used. Per-engine timelines are generated on demand and cached for later use. v2: Find end of current context on the engine timeline instead of just using the next request for adjusting the incomplete start time. v3: Improve scaling with large datasets by only walking each engine timeline once and some caching. (John Harrison) v4: * Fix logic fail from v3. * Refactor the code a bit to separate the stages better. * Do not account batches with unknown duration in avg stats. * Handle two user interrupts with the same seqno. * Handle user interrupt arriving after request_out. v5: * Update for class:instance. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: John Harrison <John.C.Harrison@intel.com> --- scripts/trace.pl | 163 +++++++++++++++++++++++++++++++---------------- 1 file changed, 108 insertions(+), 55 deletions(-) diff --git a/scripts/trace.pl b/scripts/trace.pl index 9bdafa2d7e99..33045f3264f2 100755 --- a/scripts/trace.pl +++ b/scripts/trace.pl @@ -462,16 +462,11 @@ while (<>) { die if exists $db{$key}->{'end'}; $db{$key}->{'end'} = $time; - if (exists $notify{$gkey}) { - $db{$key}->{'notify'} = $notify{$gkey}; - } else { - # No notify so far. Maybe it will arrive later which - # will be handled in the sanitation loop below. - $db{$key}->{'notify'} = $db{$key}->{'end'}; - $db{$key}->{'no-notify'} = 1; - } + $db{$key}->{'notify'} = $notify{$gkey} if exists $notify{$gkey}; } elsif ($tp_name eq 'i915:intel_engine_notify:') { - $notify{global_key($ring, $seqno)} = $time; + my $gkey = global_key($ring, $seqno); + + $notify{$gkey} = $time unless exists $notify{$gkey}; } elsif ($tp_name eq 'i915:intel_gpu_freq_change:') { push @freqs, [$prev_freq_ts, $time, $prev_freq] if $prev_freq; $prev_freq_ts = $time; @@ -484,66 +479,116 @@ while (<>) { my $max_seqno = 0; foreach my $key (keys %db) { my $gkey = global_key($db{$key}->{'ring'}, $db{$key}->{'global'}); - my $notify = $notify{$gkey}; die unless exists $db{$key}->{'start'}; $max_seqno = $db{$key}->{'seqno'} if $db{$key}->{'seqno'} > $max_seqno; - unless (exists $db{$key}->{'end'}) { - # Context complete not received. - $db{$key}->{'no-end'} = 1; + # Notify arrived after context complete? + $db{$key}->{'notify'} = $notify{$gkey} if not exists $db{$key}->{'notify'} + and exists $notify{$gkey}; - if (defined($notify)) { - # No context complete due req merging - use notify. - $db{$key}->{'notify'} = $notify; - $db{$key}->{'end'} = $notify; - } else { - # No notify and no context complete - give up for now. - $db{$key}->{'incomplete'} = 1; - } - } else { - # Notify arrived after context complete. - if (exists $db{$key}->{'no-notify'} and defined($notify)) { - delete $db{$key}->{'no-notify'}; - $db{$key}->{'notify'} = $notify; - } + # No notify but we have end? + $db{$key}->{'notify'} = $db{$key}->{'end'} if exists $db{$key}->{'end'} and + not exists $db{$key}->{'notify'}; + + # If user interrupt arrived out of order push it back to be no later + # than request out. + if (exists $db{$key}->{'end'} and exists $db{$key}->{'notify'} and + $db{$key}->{'notify'} > $db{$key}->{'end'}) { + $db{$key}->{'notify'} = $db{$key}->{'end'}; } } -# Fix up incompletes my $key_count = scalar(keys %db); -foreach my $key (keys %db) { - next unless exists $db{$key}->{'incomplete'}; - # End the incomplete batch at the time next one starts - my $ring = $db{$key}->{'ring'}; - my $ctx = $db{$key}->{'ctx'}; - my $seqno = $db{$key}->{'seqno'}; - my $next_key; - my $i = 1; - my $end; - - do { - $next_key = db_key($ring, $ctx, $seqno + $i); - $i++; - } until ((exists $db{$next_key} and not exists $db{$next_key}->{'incomplete'}) - or $i > $key_count); # ugly stop hack +my %engine_timelines; - if (exists $db{$next_key}) { - $end = $db{$next_key}->{'end'}; - } else { - # No info at all, fake it: - $end = $db{$key}->{'start'} + 999; - } +sub sortEngine { + my $as = $db{$a}->{'global'}; + my $bs = $db{$b}->{'global'}; + my $val; + + $val = $as <=> $bs; - $db{$key}->{'notify'} = $end; - $db{$key}->{'end'} = $end; + die if $val == 0; + + return $val; } -# GPU time accounting -my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count); -my (%submit_avg, %execute_avg, %ctxsave_avg); +sub get_engine_timeline { + my ($ring) = @_; + my @timeline; + + return $engine_timelines{$ring} if exists $engine_timelines{$ring}; + + @timeline = grep { $db{$_}->{'ring'} eq $ring } keys %db; + # FIXME seqno restart + @timeline = sort sortEngine @timeline; + + $engine_timelines{$ring} = \@timeline; + + return \@timeline; +} + +# Fix up coalesced requests by ending them either when the following same +# context request with known end ends, or when a different context starts. +foreach my $gid (sort keys %rings) { + my $ring = $ringmap{$rings{$gid}}; + my $timeline = get_engine_timeline($ring); + my $last_complete = -1; + my $last_ctx = -1; + my $complete; + + foreach my $pos (0..$#{$timeline}) { + my $key = @{$timeline}[$pos]; + my ($ctx, $end); + + next if exists $db{$key}->{'end'}; + + $db{$key}->{'no-end'} = 1; + $ctx = $db{$key}->{'ctx'}; + + if ($pos > $last_complete or $ctx != $last_ctx) { + my $next = $pos; + + undef $complete; + + while ($next < $#{$timeline}) { + my $next_key = ${$timeline}[++$next]; + if ($ctx == $db{$next_key}->{'ctx'} and + exists $db{$next_key}->{'end'}) { + $last_ctx = $db{$next_key}->{'ctx'}; + $last_complete = $next; + $complete = $next_key; + last; + } + } + } + + if (defined $complete) { + if ($ctx == $db{$complete}->{'ctx'}) { + $end = $db{$complete}->{'end'}; + } else { + $end = $db{$complete}->{'start'}; + } + } else { + # No next submission. Use notify if available or give up. + if (exists $db{$key}->{'notify'}) { + $end = $db{$key}->{'notify'}; + } else { + $end = $db{$key}->{'start'}; + $db{$key}->{'incomplete'} = 1; + } + } + + unless (exists $db{$key}->{'notify'}) { + $db{$key}->{'notify'} = $end; + $db{$key}->{'no-notify'} = 1; + } + $db{$key}->{'end'} = $end; + } +} sub sortStart { my $as = $db{$a}->{'start'}; @@ -592,6 +637,10 @@ foreach my $key (@sorted_keys) { @sorted_keys = sort sortStart keys %db if $re_sort; +# GPU time accounting +my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count); +my (%submit_avg, %execute_avg, %ctxsave_avg); + my $last_ts = 0; my $first_ts; my $min_ctx; @@ -614,7 +663,11 @@ foreach my $key (@sorted_keys) { } $db{$key}->{'execute-delay'} = $start - $db{$key}->{'submit'}; $db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'}; - $db{$key}->{'duration'} = $notify - $start; + unless (exists $db{$key}->{'no-notify'}) { + $db{$key}->{'duration'} = $notify - $start; + } else { + $db{$key}->{'duration'} = 0; + } $running{$ring} += $end - $start if $correct_durations or not exists $db{$key}->{'no-end'}; -- 2.17.1 _______________________________________________ igt-dev mailing list igt-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/igt-dev
next prev parent reply other threads:[~2018-07-12 11:00 UTC|newest] Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-07-12 10:59 [PATCH i-g-t 0/9] trace.pl fixes and improvements Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-12 10:59 ` [PATCH i-g-t 1/9] trace.pl: Improve time axis labels Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-12 12:42 ` Tvrtko Ursulin 2018-07-12 12:42 ` Tvrtko Ursulin 2018-07-12 22:35 ` John Harrison 2018-07-12 22:35 ` John Harrison 2018-07-13 9:55 ` [PATCH i-g-t v2 " Tvrtko Ursulin 2018-07-13 9:55 ` [igt-dev] " Tvrtko Ursulin 2018-07-16 17:53 ` John Harrison 2018-07-16 17:53 ` [Intel-gfx] " John Harrison 2018-07-17 8:56 ` [igt-dev] " Tvrtko Ursulin 2018-07-17 8:56 ` [Intel-gfx] " Tvrtko Ursulin 2018-07-17 15:11 ` John Harrison 2018-07-17 15:11 ` John Harrison 2018-07-17 15:31 ` John Harrison 2018-07-17 15:31 ` John Harrison 2018-07-17 16:22 ` Tvrtko Ursulin 2018-07-17 16:22 ` Tvrtko Ursulin 2018-07-17 17:57 ` John Harrison 2018-07-17 17:57 ` John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 2/9] trace.pl: Scale timeline for better precision Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-13 0:21 ` John Harrison 2018-07-13 0:21 ` [igt-dev] " John Harrison 2018-07-13 10:02 ` [PATCH i-g-t v3 " Tvrtko Ursulin 2018-07-13 10:02 ` [igt-dev] " Tvrtko Ursulin 2018-07-16 18:15 ` John Harrison 2018-07-16 18:15 ` [igt-dev] " John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 3/9] trace.pl: Improve readability of graphical timeline representation Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-13 0:51 ` John Harrison 2018-07-13 0:51 ` [Intel-gfx] " John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 4/9] trace.pl: Improve context colouring for large context id's Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-13 0:51 ` John Harrison 2018-07-13 0:51 ` [Intel-gfx] " John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 5/9] trace.pl: Improved key/colours Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-12 11:03 ` Tvrtko Ursulin 2018-07-12 11:03 ` Tvrtko Ursulin 2018-07-13 0:52 ` John Harrison 2018-07-13 0:52 ` [igt-dev] " John Harrison 2018-07-13 9:56 ` [PATCH i-g-t v3 " Tvrtko Ursulin 2018-07-13 9:56 ` [igt-dev] " Tvrtko Ursulin 2018-07-14 0:01 ` John Harrison 2018-07-14 0:01 ` [igt-dev] " John Harrison 2018-07-16 8:49 ` [PATCH i-g-t v4 " Tvrtko Ursulin 2018-07-16 8:49 ` [Intel-gfx] " Tvrtko Ursulin 2018-07-16 19:11 ` John Harrison 2018-07-16 19:11 ` [igt-dev] " John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 6/9] trace.pl: Context save only applies to last request of a bunch Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-13 6:55 ` John Harrison 2018-07-12 10:59 ` Tvrtko Ursulin [this message] 2018-07-12 10:59 ` [igt-dev] [PATCH i-g-t 7/9] trace.pl: Fix incomplete request handling Tvrtko Ursulin 2018-07-13 7:00 ` John Harrison 2018-07-13 7:00 ` [igt-dev] " John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 8/9] trace.pl: Basic preemption support Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-13 0:54 ` John Harrison 2018-07-12 10:59 ` [PATCH i-g-t 9/9] trace.pl: Fix request split mode Tvrtko Ursulin 2018-07-12 10:59 ` [igt-dev] " Tvrtko Ursulin 2018-07-12 14:26 ` [igt-dev] ✓ Fi.CI.BAT: success for trace.pl fixes and improvements (rev3) Patchwork 2018-07-12 15:50 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork 2018-07-13 11:09 ` [igt-dev] ✓ Fi.CI.BAT: success for trace.pl fixes and improvements (rev6) Patchwork 2018-07-13 13:59 ` [igt-dev] ✗ Fi.CI.IGT: failure " Patchwork 2018-07-16 10:03 ` [igt-dev] ✓ Fi.CI.BAT: success for trace.pl fixes and improvements (rev7) Patchwork 2018-07-16 16:41 ` [igt-dev] ✗ Fi.CI.IGT: failure " Patchwork -- strict thread matches above, loose matches on Subject: below -- 2018-05-10 10:19 [PATCH i-g-t 1/9] trace.pl: Add support for colouring context execution Tvrtko Ursulin 2018-05-10 10:19 ` [PATCH i-g-t 7/9] trace.pl: Fix incomplete request handling Tvrtko Ursulin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180712105958.12953-8-tvrtko.ursulin@linux.intel.com \ --to=tursulin@ursulin.net \ --cc=igt-dev@lists.freedesktop.org \ --cc=intel-gfx@lists.freedesktop.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.