[igt-dev] [PATCH i-g-t 00/11] trace.pl fixes

[igt-dev] [PATCH i-g-t 00/11] trace.pl fixes

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Collection of fixes on top of John's recent work.

Problems were mostly in the request split logic which had several issues both
in my original version, and also after John's improvements.

Handling of "incomplete" requests (the ones which received neither notify nor
context complete) was also a bit incorrect.

After this series it seems to work fine, famous last words. Anyway with my test
data it correctly draws the timeline with no overalaps considering both merged
requests and submission to port 1.

I also added context colouring mode so it is easier to follow the timeline and
started using hw_id for context id's for additional readability.

John Harrison (4):
  scripts/trace.pl: More hash key optimisations
  scripts/trace.pl: Sort order
  scripts/trace.pl: Calculate stats only after all munging
  scripts/trace.pl: Simplify 'end' & 'notify' generation

Tvrtko Ursulin (7):
  trace.pl: Use context hw_id as context id
  trace.pl: Move sortQueue near its user
  trace.pl: Move min/max timestamp lookup to last loop
  trace.pl: Fix engine busy accounting in split mode
  trace.pl: Add support for colouring context execution
  trace.pl: Fix incomplete request handling
  trace.pl: Fix request split mode

 scripts/trace.pl | 355 ++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 259 insertions(+), 96 deletions(-)

-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 01/11] scripts/trace.pl: More hash key optimisations

[Tvrtko Ursulin]

From: John Harrison <John.C.Harrison@Intel.com>

Cache the key count value rather than querying the hash every time.
Also assert that the database does not magically change size after the
fixups.

v2: Rename variable according to style guide [Tvrtko]

v3: Reverted accidental style change and added a blank line. [Tvrtko]

Signed-off-by: John Harrison <John.C.Harrison@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
 scripts/trace.pl | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index cb93900dd620..783a4d89065f 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -508,6 +508,7 @@ foreach my $key (keys %db) {
 }
 
 # Fix up incompletes
+my $key_count = scalar(keys %db);
 foreach my $key (keys %db) {
 	next unless exists $db{$key}->{'incomplete'};
 
@@ -522,7 +523,7 @@ foreach my $key (keys %db) {
 		$next_key = db_key($ring, $ctx, $seqno + $i);
 		$i++;
 	} until ((exists $db{$next_key} and not exists $db{$next_key}->{'incomplete'})
-		 or $i > scalar(keys(%db)));  # ugly stop hack
+		 or $i > $key_count);  # ugly stop hack
 
 	if (exists $db{$next_key}) {
 		$db{$key}->{'notify'} = $db{$next_key}->{'end'};
@@ -541,6 +542,8 @@ my $first_ts;
 my @sorted_keys = sort {$db{$a}->{'start'} <=> $db{$b}->{'start'}} keys %db;
 my $re_sort = 0;
 
+die "Database changed size?!" unless scalar(@sorted_keys) == $key_count;
+
 foreach my $key (@sorted_keys) {
 	my $ring = $db{$key}->{'ring'};
 	my $end = $db{$key}->{'end'};
@@ -565,7 +568,7 @@ foreach my $key (@sorted_keys) {
 		do {
 			$next_key = db_key($ring, $ctx, $seqno + $i);
 			$i++;
-		} until (exists $db{$next_key} or $i > scalar(keys(%db)));  # ugly stop hack
+		} until (exists $db{$next_key} or $i > $key_count);  # ugly stop hack
 
 		# 20us tolerance
 		if (exists $db{$next_key} and $db{$next_key}->{'start'} < $start + 20) {
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 02/11] scripts/trace.pl: Sort order

[Tvrtko Ursulin]

From: John Harrison <John.C.Harrison@Intel.com>

Add an extra level to the databse key sort so that the ordering is
deterministic. If the time stamp matches, it now compares the key
itself as well (context/seqno). This makes it much easier to determine
if a change has actually broken anything. Previously back to back runs
with no changes could still produce different output, especially when
adding extra debug output during the calculations.

As the comparison test is now more than a single equation, moved it
out into a separate sort function.

v2: Re-work sort func for readability/performance [Tvrtko]

Signed-off-by: John Harrison <John.C.Harrison@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
 scripts/trace.pl | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 783a4d89065f..48b3bfb2d343 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -539,7 +539,29 @@ my (%submit_avg, %execute_avg, %ctxsave_avg);
 my $last_ts = 0;
 my $first_ts;
 
-my @sorted_keys = sort {$db{$a}->{'start'} <=> $db{$b}->{'start'}} keys %db;
+sub sortStart {
+	my $as = $db{$a}->{'start'};
+	my $bs = $db{$b}->{'start'};
+	my $val;
+
+	$val = $as <=> $bs;
+	$val = $a cmp $b if $val == 0;
+
+	return $val;
+}
+
+sub sortQueue {
+	my $as = $db{$a}->{'queue'};
+	my $bs = $db{$b}->{'queue'};
+	my $val;
+
+	$val = $as <=> $bs;
+	$val = $a cmp $b if $val == 0;
+
+	return $val;
+}
+
+my @sorted_keys = sort sortStart keys %db;
 my $re_sort = 0;
 
 die "Database changed size?!" unless scalar(@sorted_keys) == $key_count;
@@ -589,9 +611,9 @@ foreach my $key (@sorted_keys) {
 	$ctxsave_avg{$ring} += $db{$key}->{'end'} - $db{$key}->{'notify'};
 }
 
-@sorted_keys = sort {$db{$a}->{'start'} <=> $db{$b}->{'start'}} keys %db if $re_sort;
+@sorted_keys = sort sortStart keys %db if $re_sort;
 
-foreach my $ring (keys %batch_avg) {
+foreach my $ring (sort keys %batch_avg) {
 	$batch_avg{$ring} /= $batch_count{$ring};
 	$batch_total_avg{$ring} /= $batch_count{$ring};
 	$submit_avg{$ring} /= $batch_count{$ring};
@@ -831,7 +853,7 @@ print <<ENDHTML;
 ENDHTML
 
 my $i = 0;
-foreach my $key (sort {$db{$a}->{'queue'} <=> $db{$b}->{'queue'}} keys %db) {
+foreach my $key (sort sortQueue keys %db) {
 	my ($name, $ctx, $seqno) = ($db{$key}->{'name'}, $db{$key}->{'ctx'}, $db{$key}->{'seqno'});
 	my ($queue, $start, $notify, $end) = ($db{$key}->{'queue'}, $db{$key}->{'start'}, $db{$key}->{'notify'}, $db{$key}->{'end'});
 	my $submit = $queue + $db{$key}->{'submit-delay'};
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 03/11] scripts/trace.pl: Calculate stats only after all munging

[Tvrtko Ursulin]

From: John Harrison <John.C.Harrison@Intel.com>

There are various statistics being calculated multiple times in
multiple places while the log file is being read in. Some of these are
then re-calculated when the database is munged to correct various
issues with the logs. This patch consolidates the calculations into a
separate pass after all the reading and munging has been done.

Note that this actually produces a different final output as the
'execute-delay' values were not previously being re-calculated after
all the fixups. Thus were based on an incorrect calculation.

v2: Reduce scope of some local variables [Tvrtko]

Signed-off-by: John Harrison <John.C.Harrison@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
 scripts/trace.pl | 54 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 28 insertions(+), 26 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 48b3bfb2d343..6111309dbf3c 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -437,8 +437,7 @@ while (<>) {
 		$req{'global'} = $tp{'global'};
 		$req{'port'} = $tp{'port'};
 		$req{'queue'} = $queue{$key};
-		$req{'submit-delay'} = $submit{$key} - $queue{$key};
-		$req{'execute-delay'} = $req{'start'} - $submit{$key};
+		$req{'submit'} = $submit{$key};
 		$rings{$ring} = $gid++ unless exists $rings{$ring};
 		$ringmap{$rings{$ring}} = $ring;
 		$db{$key} = \%req;
@@ -458,8 +457,6 @@ while (<>) {
 			$db{$key}->{'notify'} = $db{$key}->{'end'};
 			$db{$key}->{'no-notify'} = 1;
 		}
-		$db{$key}->{'duration'} = $db{$key}->{'notify'} - $db{$key}->{'start'};
-		$db{$key}->{'context-complete-delay'} = $db{$key}->{'end'} - $db{$key}->{'notify'};
 	} elsif ($tp_name eq 'i915:intel_engine_notify:') {
 		$notify{global_key($ring, $seqno)} = $time;
 	} elsif ($tp_name eq 'i915:intel_gpu_freq_change:') {
@@ -493,16 +490,11 @@ foreach my $key (keys %db) {
 			$db{$key}->{'notify'} = $db{$key}->{'end'};
 			$db{$key}->{'incomplete'} = 1;
 		}
-
-		$db{$key}->{'duration'} = $db{$key}->{'notify'} - $db{$key}->{'start'};
-		$db{$key}->{'context-complete-delay'} = $db{$key}->{'end'} - $db{$key}->{'notify'};
 	} else {
 		# Notify arrived after context complete.
 		if (exists $db{$key}->{'no-notify'} and exists $notify{$gkey}) {
 			delete $db{$key}->{'no-notify'};
 			$db{$key}->{'notify'} = $notify{$gkey};
-			$db{$key}->{'duration'} = $db{$key}->{'notify'} - $db{$key}->{'start'};
-			$db{$key}->{'context-complete-delay'} = $db{$key}->{'end'} - $db{$key}->{'notify'};
 		}
 	}
 }
@@ -528,8 +520,6 @@ foreach my $key (keys %db) {
 	if (exists $db{$next_key}) {
 		$db{$key}->{'notify'} = $db{$next_key}->{'end'};
 		$db{$key}->{'end'} = $db{$key}->{'notify'};
-		$db{$key}->{'duration'} = $db{$key}->{'notify'} - $db{$key}->{'start'};
-		$db{$key}->{'context-complete-delay'} = $db{$key}->{'end'} - $db{$key}->{'notify'};
 	}
 }
 
@@ -573,17 +563,11 @@ foreach my $key (@sorted_keys) {
 	$first_ts = $db{$key}->{'queue'} if not defined $first_ts or $db{$key}->{'queue'} < $first_ts;
 	$last_ts = $end if $end > $last_ts;
 
-	$running{$ring} += $end - $db{$key}->{'start'} unless exists $db{$key}->{'no-end'};
-	$runnable{$ring} += $db{$key}->{'execute-delay'};
-	$queued{$ring} += $db{$key}->{'start'} - $db{$key}->{'execute-delay'} - $db{$key}->{'queue'};
-
-	$batch_count{$ring}++;
-
 	# correct duration of merged batches
 	if ($correct_durations and exists $db{$key}->{'no-end'}) {
-		my $start = $db{$key}->{'start'};
 		my $ctx = $db{$key}->{'ctx'};
 		my $seqno = $db{$key}->{'seqno'};
+		my $start = $db{$key}->{'start'};
 		my $next_key;
 		my $i = 1;
 
@@ -594,25 +578,43 @@ foreach my $key (@sorted_keys) {
 
 		# 20us tolerance
 		if (exists $db{$next_key} and $db{$next_key}->{'start'} < $start + 20) {
+			my $notify = $db{$key}->{'notify'};
 			$re_sort = 1;
-			$db{$next_key}->{'start'} = $start + $db{$key}->{'duration'};
+			$db{$next_key}->{'start'} = $notify;
 			$db{$next_key}->{'start'} = $db{$next_key}->{'end'} if $db{$next_key}->{'start'} > $db{$next_key}->{'end'};
-			$db{$next_key}->{'duration'} = $db{$next_key}->{'notify'} - $db{$next_key}->{'start'};
-			$end = $db{$key}->{'notify'};
 			die if $db{$next_key}->{'start'} > $db{$next_key}->{'end'};
 		}
-		die if $db{$key}->{'start'} > $db{$key}->{'end'};
+		die if $start > $end;
 	}
+}
+
+@sorted_keys = sort sortStart keys %db if $re_sort;
+
+foreach my $key (@sorted_keys) {
+	my $ring = $db{$key}->{'ring'};
+	my $end = $db{$key}->{'end'};
+	my $start = $db{$key}->{'start'};
+	my $notify = $db{$key}->{'notify'};
+
+	$db{$key}->{'context-complete-delay'} = $end - $notify;
+	$db{$key}->{'execute-delay'} = $start - $db{$key}->{'submit'};
+	$db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'};
+	$db{$key}->{'duration'} = $notify - $start;
+
+	$running{$ring} += $end - $start unless exists $db{$key}->{'no-end'};
+	$runnable{$ring} += $db{$key}->{'execute-delay'};
+	$queued{$ring} += $start - $db{$key}->{'execute-delay'} - $db{$key}->{'queue'};
+
+	$batch_count{$ring}++;
+
 	$batch_avg{$ring} += $db{$key}->{'duration'};
-	$batch_total_avg{$ring} += $end - $db{$key}->{'start'};
+	$batch_total_avg{$ring} += $end - $start;
 
 	$submit_avg{$ring} += $db{$key}->{'submit-delay'};
 	$execute_avg{$ring} += $db{$key}->{'execute-delay'};
-	$ctxsave_avg{$ring} += $db{$key}->{'end'} - $db{$key}->{'notify'};
+	$ctxsave_avg{$ring} += $end - $notify;
 }
 
-@sorted_keys = sort sortStart keys %db if $re_sort;
-
 foreach my $ring (sort keys %batch_avg) {
 	$batch_avg{$ring} /= $batch_count{$ring};
 	$batch_total_avg{$ring} /= $batch_count{$ring};
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 04/11] scripts/trace.pl: Simplify 'end' & 'notify' generation

[Tvrtko Ursulin]

From: John Harrison <John.C.Harrison@Intel.com>

Delay the auto-generation of end/notify values until the point where
everything is known. As opposed to potentially generating them
multiple times with differing values (in the case of 'incomplete'
entries).

v2: More complete description. [Tvrtko]

Signed-off-by: John Harrison <John.C.Harrison@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
 scripts/trace.pl | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 6111309dbf3c..be256ad8b94e 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -467,10 +467,11 @@ while (<>) {
 }
 
 # Sanitation pass to fixup up out of order notify and context complete, and to
-# fine the largest seqno to be used for timeline sorting purposes.
+# find the largest seqno to be used for timeline sorting purposes.
 my $max_seqno = 0;
 foreach my $key (keys %db) {
 	my $gkey = global_key($db{$key}->{'ring'}, $db{$key}->{'global'});
+	my $notify = $notify{$gkey};
 
 	die unless exists $db{$key}->{'start'};
 
@@ -478,23 +479,21 @@ foreach my $key (keys %db) {
 
 	unless (exists $db{$key}->{'end'}) {
 		# Context complete not received.
-		if (exists $notify{$gkey}) {
+		$db{$key}->{'no-end'} = 1;
+
+		if (defined($notify)) {
 			# No context complete due req merging - use notify.
-			$db{$key}->{'notify'} = $notify{$gkey};
-			$db{$key}->{'end'} = $db{$key}->{'notify'};
-			$db{$key}->{'no-end'} = 1;
+			$db{$key}->{'notify'} = $notify;
+			$db{$key}->{'end'} = $notify;
 		} else {
-			# No notify and no context complete - mark it.
-			$db{$key}->{'no-end'} = 1;
-			$db{$key}->{'end'} = $db{$key}->{'start'} + 999;
-			$db{$key}->{'notify'} = $db{$key}->{'end'};
+			# No notify and no context complete - give up for now.
 			$db{$key}->{'incomplete'} = 1;
 		}
 	} else {
 		# Notify arrived after context complete.
-		if (exists $db{$key}->{'no-notify'} and exists $notify{$gkey}) {
+		if (exists $db{$key}->{'no-notify'} and defined($notify)) {
 			delete $db{$key}->{'no-notify'};
-			$db{$key}->{'notify'} = $notify{$gkey};
+			$db{$key}->{'notify'} = $notify;
 		}
 	}
 }
@@ -510,6 +509,7 @@ foreach my $key (keys %db) {
 	my $seqno = $db{$key}->{'seqno'};
 	my $next_key;
 	my $i = 1;
+	my $end;
 
 	do {
 		$next_key = db_key($ring, $ctx, $seqno + $i);
@@ -518,9 +518,14 @@ foreach my $key (keys %db) {
 		 or $i > $key_count);  # ugly stop hack
 
 	if (exists $db{$next_key}) {
-		$db{$key}->{'notify'} = $db{$next_key}->{'end'};
-		$db{$key}->{'end'} = $db{$key}->{'notify'};
+		$end = $db{$next_key}->{'end'};
+	} else {
+		# No info at all, fake it:
+		$end = $db{$key}->{'start'} + 999;
 	}
+
+	$db{$key}->{'notify'} = $end;
+	$db{$key}->{'end'} = $end;
 }
 
 # GPU time accounting
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 05/11] trace.pl: Use context hw_id as context id

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Now that it is available from the kernel, use the generally much smaller
and so more readable, hw_id as the request context id.

This also means context id squashing command line switch has to be
removed.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 22 ++--------------------
 1 file changed, 2 insertions(+), 20 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index be256ad8b94e..958c83c82b8b 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -38,7 +38,6 @@ my %skip_box;
 my $html = 0;
 my $trace = 0;
 my $avg_delay_stats = 0;
-my $squash_context_id = 0;
 my $gpu_timeline = 0;
 
 my @args;
@@ -107,8 +106,6 @@ Usage:
       --html				Generate HTML output.
       --trace cmd			Trace the following command.
       --avg-delay-stats			Print average delay stats.
-      --squash-ctx-id			Squash context id by substracting engine
-					id from ctx id.
       --gpu-timeline			Draw overall GPU busy timeline.
 ENDHELP
 
@@ -142,18 +139,6 @@ sub arg_avg_delay_stats
 	return @_;
 }
 
-sub arg_squash_ctx_id
-{
-	return unless scalar(@_);
-
-	if ($_[0] eq '--squash-ctx-id') {
-		shift @_;
-		$squash_context_id = 1;
-	}
-
-	return @_;
-}
-
 sub arg_gpu_timeline
 {
 	return unless scalar(@_);
@@ -286,7 +271,6 @@ while (@args) {
 	@args = arg_help(@args);
 	@args = arg_html(@args);
 	@args = arg_avg_delay_stats(@args);
-	@args = arg_squash_ctx_id(@args);
 	@args = arg_gpu_timeline(@args);
 	@args = arg_trace(@args);
 	@args = arg_max_items(@args);
@@ -320,8 +304,6 @@ sub sanitize_ctx
 {
 	my ($ctx, $ring) = @_;
 
-	$ctx = $ctx - $ring if $squash_context_id;
-
 	if (exists $ctxdb{$ctx}) {
 		return $ctx . '.' . $ctxdb{$ctx};
 	} else {
@@ -386,8 +368,8 @@ while (<>) {
 		$ring = $tp{'ring'};
 		$seqno = $tp{'seqno'};
 
-		if (exists $tp{'ctx'}) {
-			$ctx = $tp{'ctx'};
+		if (exists $tp{'hw_id'}) {
+			$ctx = $tp{'hw_id'};
 			$orig_ctx = $ctx;
 			$ctx = sanitize_ctx($ctx, $ring);
 			$key = db_key($ring, $ctx, $seqno);
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 05/11] trace.pl: Use context hw_id as context id

[Lionel Landwerlin]

Chris is looking at updating the hw_id per submission or something...
Just letting you know, that might impact your new way of tracking things :(

On 30/01/18 10:11, Tvrtko Ursulin wrote:
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>
> Now that it is available from the kernel, use the generally much smaller
> and so more readable, hw_id as the request context id.
>
> This also means context id squashing command line switch has to be
> removed.
>
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: John Harrison <John.C.Harrison@intel.com>
> ---
>   scripts/trace.pl | 22 ++--------------------
>   1 file changed, 2 insertions(+), 20 deletions(-)
>
> diff --git a/scripts/trace.pl b/scripts/trace.pl
> index be256ad8b94e..958c83c82b8b 100755
> --- a/scripts/trace.pl
> +++ b/scripts/trace.pl
> @@ -38,7 +38,6 @@ my %skip_box;
>   my $html = 0;
>   my $trace = 0;
>   my $avg_delay_stats = 0;
> -my $squash_context_id = 0;
>   my $gpu_timeline = 0;
>   
>   my @args;
> @@ -107,8 +106,6 @@ Usage:
>         --html				Generate HTML output.
>         --trace cmd			Trace the following command.
>         --avg-delay-stats			Print average delay stats.
> -      --squash-ctx-id			Squash context id by substracting engine
> -					id from ctx id.
>         --gpu-timeline			Draw overall GPU busy timeline.
>   ENDHELP
>   
> @@ -142,18 +139,6 @@ sub arg_avg_delay_stats
>   	return @_;
>   }
>   
> -sub arg_squash_ctx_id
> -{
> -	return unless scalar(@_);
> -
> -	if ($_[0] eq '--squash-ctx-id') {
> -		shift @_;
> -		$squash_context_id = 1;
> -	}
> -
> -	return @_;
> -}
> -
>   sub arg_gpu_timeline
>   {
>   	return unless scalar(@_);
> @@ -286,7 +271,6 @@ while (@args) {
>   	@args = arg_help(@args);
>   	@args = arg_html(@args);
>   	@args = arg_avg_delay_stats(@args);
> -	@args = arg_squash_ctx_id(@args);
>   	@args = arg_gpu_timeline(@args);
>   	@args = arg_trace(@args);
>   	@args = arg_max_items(@args);
> @@ -320,8 +304,6 @@ sub sanitize_ctx
>   {
>   	my ($ctx, $ring) = @_;
>   
> -	$ctx = $ctx - $ring if $squash_context_id;
> -
>   	if (exists $ctxdb{$ctx}) {
>   		return $ctx . '.' . $ctxdb{$ctx};
>   	} else {
> @@ -386,8 +368,8 @@ while (<>) {
>   		$ring = $tp{'ring'};
>   		$seqno = $tp{'seqno'};
>   
> -		if (exists $tp{'ctx'}) {
> -			$ctx = $tp{'ctx'};
> +		if (exists $tp{'hw_id'}) {
> +			$ctx = $tp{'hw_id'};
>   			$orig_ctx = $ctx;
>   			$ctx = sanitize_ctx($ctx, $ring);
>   			$key = db_key($ring, $ctx, $seqno);


_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 05/11] trace.pl: Use context hw_id as context id

[Tvrtko Ursulin]

On 30/01/2018 12:29, Lionel Landwerlin wrote:
> Chris is looking at updating the hw_id per submission or something...
> Just letting you know, that might impact your new way of tracking things :(

Hm yeah, maybe even rings a bell that hw_id can be re-used as contexts 
are destroyed an created so would confuse the script without any 
changes. Will double-check. Luckily this patch is completely independent 
from the rest so can simply be dropped in that case.

Regards,

Tvrtko


> On 30/01/18 10:11, Tvrtko Ursulin wrote:
>> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>>
>> Now that it is available from the kernel, use the generally much smaller
>> and so more readable, hw_id as the request context id.
>>
>> This also means context id squashing command line switch has to be
>> removed.
>>
>> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
>> Cc: John Harrison <John.C.Harrison@intel.com>
>> ---
>>   scripts/trace.pl | 22 ++--------------------
>>   1 file changed, 2 insertions(+), 20 deletions(-)
>>
>> diff --git a/scripts/trace.pl b/scripts/trace.pl
>> index be256ad8b94e..958c83c82b8b 100755
>> --- a/scripts/trace.pl
>> +++ b/scripts/trace.pl
>> @@ -38,7 +38,6 @@ my %skip_box;
>>   my $html = 0;
>>   my $trace = 0;
>>   my $avg_delay_stats = 0;
>> -my $squash_context_id = 0;
>>   my $gpu_timeline = 0;
>>   my @args;
>> @@ -107,8 +106,6 @@ Usage:
>>         --html                Generate HTML output.
>>         --trace cmd            Trace the following command.
>>         --avg-delay-stats            Print average delay stats.
>> -      --squash-ctx-id            Squash context id by substracting 
>> engine
>> -                    id from ctx id.
>>         --gpu-timeline            Draw overall GPU busy timeline.
>>   ENDHELP
>> @@ -142,18 +139,6 @@ sub arg_avg_delay_stats
>>       return @_;
>>   }
>> -sub arg_squash_ctx_id
>> -{
>> -    return unless scalar(@_);
>> -
>> -    if ($_[0] eq '--squash-ctx-id') {
>> -        shift @_;
>> -        $squash_context_id = 1;
>> -    }
>> -
>> -    return @_;
>> -}
>> -
>>   sub arg_gpu_timeline
>>   {
>>       return unless scalar(@_);
>> @@ -286,7 +271,6 @@ while (@args) {
>>       @args = arg_help(@args);
>>       @args = arg_html(@args);
>>       @args = arg_avg_delay_stats(@args);
>> -    @args = arg_squash_ctx_id(@args);
>>       @args = arg_gpu_timeline(@args);
>>       @args = arg_trace(@args);
>>       @args = arg_max_items(@args);
>> @@ -320,8 +304,6 @@ sub sanitize_ctx
>>   {
>>       my ($ctx, $ring) = @_;
>> -    $ctx = $ctx - $ring if $squash_context_id;
>> -
>>       if (exists $ctxdb{$ctx}) {
>>           return $ctx . '.' . $ctxdb{$ctx};
>>       } else {
>> @@ -386,8 +368,8 @@ while (<>) {
>>           $ring = $tp{'ring'};
>>           $seqno = $tp{'seqno'};
>> -        if (exists $tp{'ctx'}) {
>> -            $ctx = $tp{'ctx'};
>> +        if (exists $tp{'hw_id'}) {
>> +            $ctx = $tp{'hw_id'};
>>               $orig_ctx = $ctx;
>>               $ctx = sanitize_ctx($ctx, $ring);
>>               $key = db_key($ring, $ctx, $seqno);
> 
> 
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 06/11] trace.pl: Move sortQueue near its user

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Just to clear up some space for incoming code refactoring.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 958c83c82b8b..f30fcac878c1 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -527,17 +527,6 @@ sub sortStart {
 	return $val;
 }
 
-sub sortQueue {
-	my $as = $db{$a}->{'queue'};
-	my $bs = $db{$b}->{'queue'};
-	my $val;
-
-	$val = $as <=> $bs;
-	$val = $a cmp $b if $val == 0;
-
-	return $val;
-}
-
 my @sorted_keys = sort sortStart keys %db;
 my $re_sort = 0;
 
@@ -841,6 +830,17 @@ print <<ENDHTML;
   var items = new vis.DataSet([
 ENDHTML
 
+sub sortQueue {
+	my $as = $db{$a}->{'queue'};
+	my $bs = $db{$b}->{'queue'};
+	my $val;
+
+	$val = $as <=> $bs;
+	$val = $a cmp $b if $val == 0;
+
+	return $val;
+}
+
 my $i = 0;
 foreach my $key (sort sortQueue keys %db) {
 	my ($name, $ctx, $seqno) = ($db{$key}->{'name'}, $db{$key}->{'ctx'}, $db{$key}->{'seqno'});
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 06/11] trace.pl: Move sortQueue near to the user

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Just to clear up some space for incoming code refactoring.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 958c83c82b8b..f30fcac878c1 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -527,17 +527,6 @@ sub sortStart {
 	return $val;
 }
 
-sub sortQueue {
-	my $as = $db{$a}->{'queue'};
-	my $bs = $db{$b}->{'queue'};
-	my $val;
-
-	$val = $as <=> $bs;
-	$val = $a cmp $b if $val == 0;
-
-	return $val;
-}
-
 my @sorted_keys = sort sortStart keys %db;
 my $re_sort = 0;
 
@@ -841,6 +830,17 @@ print <<ENDHTML;
   var items = new vis.DataSet([
 ENDHTML
 
+sub sortQueue {
+	my $as = $db{$a}->{'queue'};
+	my $bs = $db{$b}->{'queue'};
+	my $val;
+
+	$val = $as <=> $bs;
+	$val = $a cmp $b if $val == 0;
+
+	return $val;
+}
+
 my $i = 0;
 foreach my $key (sort sortQueue keys %db) {
 	my ($name, $ctx, $seqno) = ($db{$key}->{'name'}, $db{$key}->{'ctx'}, $db{$key}->{'seqno'});
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 06/11] trace.pl: Move sortQueue near to the user

[Tvrtko Ursulin]

Garbage in the cwd, please ignore this one! :(

On 30/01/2018 10:11, Tvrtko Ursulin wrote:
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> Just to clear up some space for incoming code refactoring.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: John Harrison <John.C.Harrison@intel.com>
> ---
>   scripts/trace.pl | 22 +++++++++++-----------
>   1 file changed, 11 insertions(+), 11 deletions(-)
> 
> diff --git a/scripts/trace.pl b/scripts/trace.pl
> index 958c83c82b8b..f30fcac878c1 100755
> --- a/scripts/trace.pl
> +++ b/scripts/trace.pl
> @@ -527,17 +527,6 @@ sub sortStart {
>   	return $val;
>   }
>   
> -sub sortQueue {
> -	my $as = $db{$a}->{'queue'};
> -	my $bs = $db{$b}->{'queue'};
> -	my $val;
> -
> -	$val = $as <=> $bs;
> -	$val = $a cmp $b if $val == 0;
> -
> -	return $val;
> -}
> -
>   my @sorted_keys = sort sortStart keys %db;
>   my $re_sort = 0;
>   
> @@ -841,6 +830,17 @@ print <<ENDHTML;
>     var items = new vis.DataSet([
>   ENDHTML
>   
> +sub sortQueue {
> +	my $as = $db{$a}->{'queue'};
> +	my $bs = $db{$b}->{'queue'};
> +	my $val;
> +
> +	$val = $as <=> $bs;
> +	$val = $a cmp $b if $val == 0;
> +
> +	return $val;
> +}
> +
>   my $i = 0;
>   foreach my $key (sort sortQueue keys %db) {
>   	my ($name, $ctx, $seqno) = ($db{$key}->{'name'}, $db{$key}->{'ctx'}, $db{$key}->{'seqno'});
> 
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 07/11] trace.pl: Move min/max timestamp lookup to last loop

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

It makes sense to fetch the min and max timestamp only after the
last sort of the array.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index f30fcac878c1..16a1675c6c21 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -513,8 +513,6 @@ foreach my $key (keys %db) {
 # GPU time accounting
 my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count);
 my (%submit_avg, %execute_avg, %ctxsave_avg);
-my $last_ts = 0;
-my $first_ts;
 
 sub sortStart {
 	my $as = $db{$a}->{'start'};
@@ -536,9 +534,6 @@ foreach my $key (@sorted_keys) {
 	my $ring = $db{$key}->{'ring'};
 	my $end = $db{$key}->{'end'};
 
-	$first_ts = $db{$key}->{'queue'} if not defined $first_ts or $db{$key}->{'queue'} < $first_ts;
-	$last_ts = $end if $end > $last_ts;
-
 	# correct duration of merged batches
 	if ($correct_durations and exists $db{$key}->{'no-end'}) {
 		my $ctx = $db{$key}->{'ctx'};
@@ -566,12 +561,18 @@ foreach my $key (@sorted_keys) {
 
 @sorted_keys = sort sortStart keys %db if $re_sort;
 
+my $last_ts = 0;
+my $first_ts;
+
 foreach my $key (@sorted_keys) {
 	my $ring = $db{$key}->{'ring'};
 	my $end = $db{$key}->{'end'};
 	my $start = $db{$key}->{'start'};
 	my $notify = $db{$key}->{'notify'};
 
+	$first_ts = $db{$key}->{'queue'} if not defined $first_ts or $db{$key}->{'queue'} < $first_ts;
+	$last_ts = $end if $end > $last_ts;
+
 	$db{$key}->{'context-complete-delay'} = $end - $notify;
 	$db{$key}->{'execute-delay'} = $start - $db{$key}->{'submit'};
 	$db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'};
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 08/11] trace.pl: Fix engine busy accounting in split mode

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

In split mode all requests have to be added up since they were previously
re-arranged so there is no overlap.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 16a1675c6c21..e39d56c962e7 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -578,7 +578,8 @@ foreach my $key (@sorted_keys) {
 	$db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'};
 	$db{$key}->{'duration'} = $notify - $start;
 
-	$running{$ring} += $end - $start unless exists $db{$key}->{'no-end'};
+	$running{$ring} += $end - $start if $correct_durations or
+					    not exists $db{$key}->{'no-end'};
 	$runnable{$ring} += $db{$key}->{'execute-delay'};
 	$queued{$ring} += $start - $db{$key}->{'execute-delay'} - $db{$key}->{'queue'};
 
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 08/11] trace.pl: Split engine busy accounting in split mode

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

In split mode all requests have to be added up since they were previously
re-arranged so there is no overlap.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 16a1675c6c21..e39d56c962e7 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -578,7 +578,8 @@ foreach my $key (@sorted_keys) {
 	$db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'};
 	$db{$key}->{'duration'} = $notify - $start;
 
-	$running{$ring} += $end - $start unless exists $db{$key}->{'no-end'};
+	$running{$ring} += $end - $start if $correct_durations or
+					    not exists $db{$key}->{'no-end'};
 	$runnable{$ring} += $db{$key}->{'execute-delay'};
 	$queued{$ring} += $start - $db{$key}->{'execute-delay'} - $db{$key}->{'queue'};
 
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 08/11] trace.pl: Split engine busy accounting in split mode

[Tvrtko Ursulin]

Garbage in the cwd, please ignore this one! :(

On 30/01/2018 10:11, Tvrtko Ursulin wrote:
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> In split mode all requests have to be added up since they were previously
> re-arranged so there is no overlap.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: John Harrison <John.C.Harrison@intel.com>
> ---
>   scripts/trace.pl | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/trace.pl b/scripts/trace.pl
> index 16a1675c6c21..e39d56c962e7 100755
> --- a/scripts/trace.pl
> +++ b/scripts/trace.pl
> @@ -578,7 +578,8 @@ foreach my $key (@sorted_keys) {
>   	$db{$key}->{'submit-delay'} = $db{$key}->{'submit'} - $db{$key}->{'queue'};
>   	$db{$key}->{'duration'} = $notify - $start;
>   
> -	$running{$ring} += $end - $start unless exists $db{$key}->{'no-end'};
> +	$running{$ring} += $end - $start if $correct_durations or
> +					    not exists $db{$key}->{'no-end'};
>   	$runnable{$ring} += $db{$key}->{'execute-delay'};
>   	$queued{$ring} += $start - $db{$key}->{'execute-delay'} - $db{$key}->{'queue'};
>   
> 
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 09/11] trace.pl: Add support for colouring context execution

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Add the command line switch which uses different colours for different
context execution boxes.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 41 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 39 insertions(+), 2 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index e39d56c962e7..1ab6c297acde 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -39,6 +39,7 @@ my $html = 0;
 my $trace = 0;
 my $avg_delay_stats = 0;
 my $gpu_timeline = 0;
+my $colour_contexts = 0;
 
 my @args;
 
@@ -107,6 +108,8 @@ Usage:
       --trace cmd			Trace the following command.
       --avg-delay-stats			Print average delay stats.
       --gpu-timeline			Draw overall GPU busy timeline.
+      --colour-contexts / -c		Use different colours for different
+					context execution boxes.
 ENDHELP
 
 		exit 0;
@@ -264,6 +267,20 @@ sub arg_skip_box
 	return @_;
 }
 
+sub arg_colour_contexts
+{
+	return unless scalar(@_);
+
+	if ($_[0] eq '--colour-contexts' or
+	    $_[0] eq '--color-contexts' or
+	    $_[0] eq '-c') {
+		shift @_;
+		$colour_contexts = 1;
+	}
+
+	return @_;
+}
+
 @args = @ARGV;
 while (@args) {
 	my $left = scalar(@args);
@@ -278,6 +295,7 @@ while (@args) {
 	@args = arg_split_requests(@args);
 	@args = arg_ignore_ring(@args);
 	@args = arg_skip_box(@args);
+	@args = arg_colour_contexts(@args);
 
 	last if $left == scalar(@args);
 }
@@ -563,6 +581,7 @@ foreach my $key (@sorted_keys) {
 
 my $last_ts = 0;
 my $first_ts;
+my $min_ctx;
 
 foreach my $key (@sorted_keys) {
 	my $ring = $db{$key}->{'ring'};
@@ -572,6 +591,8 @@ foreach my $key (@sorted_keys) {
 
 	$first_ts = $db{$key}->{'queue'} if not defined $first_ts or $db{$key}->{'queue'} < $first_ts;
 	$last_ts = $end if $end > $last_ts;
+	$min_ctx = $db{$key}->{'ctx'} if not defined $min_ctx or
+					 $db{$key}->{'ctx'} < $min_ctx;
 
 	$db{$key}->{'context-complete-delay'} = $end - $notify;
 	$db{$key}->{'execute-delay'} = $start - $db{$key}->{'submit'};
@@ -703,6 +724,8 @@ foreach my $key (keys %reqwait) {
 say sprintf('GPU: %.2f%% idle, %.2f%% busy',
 	     $flat_busy{'gpu-idle'}, $flat_busy{'gpu-busy'}) unless $html;
 
+my $execute_colour = $colour_contexts ? 'multi-colour' : 'pink';
+
 print <<ENDHTML if $html;
 <!DOCTYPE HTML>
 <html>
@@ -723,7 +746,7 @@ print <<ENDHTML if $html;
 <button onclick="toggleStackSubgroups()">Toggle stacking</button>
 
 <p>
-pink = requests executing on the GPU<br>
+$execute_colour = requests executing on the GPU<br>
 grey = runnable requests waiting for a slot on GPU<br>
 blue = requests waiting on fences and dependencies before they are runnable<br>
 </p>
@@ -843,6 +866,19 @@ sub sortQueue {
 	return $val;
 }
 
+my @html_colours = ( 'Pink', 'Green', 'Purple', 'Gold', 'Cyan', 'Brown',
+		     'DeepPink', 'LightGreen', 'Plum', 'Orange', 'Teal', 'Peru',
+		     'Indigo', 'Khaki', 'Coral', 'Olive' );
+
+sub ctx_colour
+{
+       my ($ctx) = (@_);
+
+       return 'Pink' unless $colour_contexts;
+
+       return $html_colours[($ctx - $min_ctx) % scalar(@html_colours)];
+}
+
 my $i = 0;
 foreach my $key (sort sortQueue keys %db) {
 	my ($name, $ctx, $seqno) = ($db{$key}->{'name'}, $db{$key}->{'ctx'}, $db{$key}->{'seqno'});
@@ -880,7 +916,8 @@ foreach my $key (sort sortQueue keys %db) {
 		if (exists $db{$key}->{'incomplete'}) {
 			$style = 'color: white; background-color: red;';
 		} else {
-			$style = 'color: black; background-color: pink;';
+			$style = 'color: black; background-color: ' .
+				  ctx_colour($ctx) . ';';
 		}
 		$content = "$name <small>$db{$key}->{'port'}</small>";
 		$content .= ' <small><i>???</i></small> ' if exists $db{$key}->{'incomplete'};
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 10/11] trace.pl: Fix incomplete request handling

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Incomplete requests (no notify, no context complete) have to be corrected
by looking at the engine timeline, and not the sorted-by-start-time view
as was previously used.

Per-engine timelines are generated on demand and cached for later use.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 75 ++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 54 insertions(+), 21 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 1ab6c297acde..768bb015da99 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -26,6 +26,8 @@ use strict;
 use warnings;
 use 5.010;
 
+use List::Util;
+
 my $gid = 0;
 my (%db, %queue, %submit, %notify, %rings, %ctxdb, %ringmap, %reqwait);
 my @freqs;
@@ -498,29 +500,60 @@ foreach my $key (keys %db) {
 	}
 }
 
-# Fix up incompletes
 my $key_count = scalar(keys %db);
-foreach my $key (keys %db) {
-	next unless exists $db{$key}->{'incomplete'};
 
-	# End the incomplete batch at the time next one starts
-	my $ring = $db{$key}->{'ring'};
-	my $ctx = $db{$key}->{'ctx'};
-	my $seqno = $db{$key}->{'seqno'};
-	my $next_key;
-	my $i = 1;
+my %engine_timelines;
+
+sub sortEngine {
+	my $as = $db{$a}->{'global'};
+	my $bs = $db{$b}->{'global'};
+	my $val;
+
+	$val = $as <=> $bs;
+
+	die if $val == 0;
+
+	return $val;
+}
+
+sub get_engine_timeline {
+	my ($ring) = @_;
+	my @timeline;
+
+	return $engine_timelines{$ring} if exists $engine_timelines{$ring};
+
+	@timeline = grep { $db{$_}->{'ring'} == $ring } keys %db;
+	# FIXME seqno restart
+	@timeline = sort sortEngine @timeline;
+
+	$engine_timelines{$ring} = \@timeline;
+
+	return \@timeline;
+}
+
+# Fix up incompletes by ending them when the following batch on the same engine
+# timeline starts. Start by filtering out the incomplete requests.
+my @incompletes = grep { exists $db{$_}->{'incomplete'} } keys %db;
+
+foreach my $key (@incompletes) {
+	my $timeline;
+	my $global;
+	my $i;
 	my $end;
 
-	do {
-		$next_key = db_key($ring, $ctx, $seqno + $i);
-		$i++;
-	} until ((exists $db{$next_key} and not exists $db{$next_key}->{'incomplete'})
-		 or $i > $key_count);  # ugly stop hack
+	# Find the next request on the engine timeline.
+	$timeline = get_engine_timeline($db{$key}->{'ring'});
+	$global = $db{$key}->{'global'};
+
+	$i = List::Util::first { ${$timeline}[$_] eq $key } 0..$#{$timeline};
 
-	if (exists $db{$next_key}) {
-		$end = $db{$next_key}->{'end'};
+	$i = $i + 1;
+	if ($i <= $#{$timeline}) {
+		my $next_key = ${$timeline}[$i];
+
+		$end = $db{$next_key}->{'start'};
 	} else {
-		# No info at all, fake it:
+		# No next submission, fake it.
 		$end = $db{$key}->{'start'} + 999;
 	}
 
@@ -528,10 +561,6 @@ foreach my $key (keys %db) {
 	$db{$key}->{'end'} = $end;
 }
 
-# GPU time accounting
-my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count);
-my (%submit_avg, %execute_avg, %ctxsave_avg);
-
 sub sortStart {
 	my $as = $db{$a}->{'start'};
 	my $bs = $db{$b}->{'start'};
@@ -579,6 +608,10 @@ foreach my $key (@sorted_keys) {
 
 @sorted_keys = sort sortStart keys %db if $re_sort;
 
+# GPU time accounting
+my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count);
+my (%submit_avg, %execute_avg, %ctxsave_avg);
+
 my $last_ts = 0;
 my $first_ts;
 my $min_ctx;
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 11/11] trace.pl: Fix request split mode

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Request split mode had several bugs, both in the original version and also
after the recent refactorings.

One big one was that it wasn't considering different submit ports as a
reason to split execution, and also that it was too time based instead of
looking at relevant timelines.

In this refactoring we address the former by using the engine timelines
introduced in the previous patch. Secondary port submissions are moved
to follow the preceding submission as a first step in the correction
process.

In the second step, we add context timelines and use then in a similar
fashion to separate start and end time of coalesced requests. For each
coalesced request we know its boundaries by looking at the engine
timeline (via global seqnos), and we know the previous request it should
only start after, by looking at the context timeline.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: John Harrison <John.C.Harrison@intel.com>
---
 scripts/trace.pl | 129 ++++++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 103 insertions(+), 26 deletions(-)

diff --git a/scripts/trace.pl b/scripts/trace.pl
index 768bb015da99..653215819f34 100755
--- a/scripts/trace.pl
+++ b/scripts/trace.pl
@@ -572,41 +572,118 @@ sub sortStart {
 	return $val;
 }
 
-my @sorted_keys = sort sortStart keys %db;
-my $re_sort = 0;
+my $re_sort = 1;
+my @sorted_keys;
 
-die "Database changed size?!" unless scalar(@sorted_keys) == $key_count;
+sub maybe_sort_keys
+{
+	if ($re_sort) {
+		@sorted_keys = sort sortStart keys %db;
+		$re_sort = 0;
+		die "Database changed size?!" unless scalar(@sorted_keys) ==
+						     $key_count;
+	}
+}
 
-foreach my $key (@sorted_keys) {
-	my $ring = $db{$key}->{'ring'};
-	my $end = $db{$key}->{'end'};
+maybe_sort_keys();
+
+my %ctx_timelines;
+
+sub sortContext {
+	my $as = $db{$a}->{'seqno'};
+	my $bs = $db{$b}->{'seqno'};
+	my $val;
 
-	# correct duration of merged batches
-	if ($correct_durations and exists $db{$key}->{'no-end'}) {
+	$val = $as <=> $bs;
+
+	die if $val == 0;
+
+	return $val;
+}
+
+sub get_ctx_timeline {
+	my ($ctx, $ring, $key) = @_;
+	my @timeline;
+
+	return $ctx_timelines{$key} if exists $ctx_timelines{$key};
+
+	@timeline = grep { $db{$_}->{'ring'} == $ring and
+			   $db{$_}->{'ctx'} == $ctx } @sorted_keys;
+	# FIXME seqno restart
+	@timeline = sort sortContext @timeline;
+
+	$ctx_timelines{$key} = \@timeline;
+
+	return \@timeline;
+}
+
+# Split out merged batches if requested.
+if ($correct_durations) {
+	my @port1;
+	my @no_end;
+
+	# Shift port1 requests start time to after the previous port0 context on
+	# the same timeline has finished.
+	@port1 = grep { $db{$_}->{'port'} == 1 } @sorted_keys;
+	foreach my $key (@port1) {
+		my $timeline = get_engine_timeline($db{$key}->{'ring'});
+		my $i = List::Util::first { ${$timeline}[$_] eq $key } 0..$#{$timeline};
+		my $prev_key;
+		my $start;
+
+		die if $i == 0;
+
+		$prev_key = ${$timeline}[$i - 1];
+		$start = $db{$prev_key}->{'end'};
+		$db{$key}->{'start'} = $start;
+		die if $start > $db{$key}->{'end'};
+
+		# Timeline has just been modified so we'll need to adjust it.
+		# First delete the current key from the timeline.
+		splice @{$timeline}, $i, 1;
+		# Now find the new correct position for the key and insert it
+		# there.
+		while ($i < $#{$timeline}) {
+			last if $db{${$timeline}[$i]}->{'global'} >= $db{$key}->{'global'};
+			$i = $i + 1;
+		}
+		splice @{$timeline}, $i, 0, $key;
+		$re_sort = 1;
+	}
+
+	maybe_sort_keys();
+
+	# Batch with no-end (no request_out) means it was submitted as part of
+	# colaesced context. This means it's start time should be set to the end
+	# time of a precedeing request on this timeline.
+	@no_end = grep { exists $db{$_}->{'no-end'} } @sorted_keys;
+	foreach my $key (@no_end) {
 		my $ctx = $db{$key}->{'ctx'};
-		my $seqno = $db{$key}->{'seqno'};
-		my $start = $db{$key}->{'start'};
-		my $next_key;
-		my $i = 1;
-
-		do {
-			$next_key = db_key($ring, $ctx, $seqno + $i);
-			$i++;
-		} until (exists $db{$next_key} or $i > $key_count);  # ugly stop hack
-
-		# 20us tolerance
-		if (exists $db{$next_key} and $db{$next_key}->{'start'} < $start + 20) {
-			my $notify = $db{$key}->{'notify'};
+		my $ring = $db{$key}->{'ring'};
+		my $tkey = $ctx . '/' . $ring;
+		my $timeline = get_ctx_timeline($ctx, $ring, $tkey);
+		my $i;
+
+		$i = List::Util::first { ${$timeline}[$_] eq $key } 0..$#{$timeline};
+		if ($i > 0) {
+			my $prev_key = ${$timeline}[$i - 1];
+
+			if ($db{$prev_key}->{'global'} == $db{$key}->{'global'}) {
+				$db{$key}->{'start'} = $db{$prev_key}->{'notify'};
+				$re_sort = 1;
+			}
+		}
+
+		if ($i < $#{$timeline}) {
+			my $next_key = ${$timeline}[$i + 1];
+
+			$db{$next_key}->{'start'} = $db{$key}->{'notify'};
 			$re_sort = 1;
-			$db{$next_key}->{'start'} = $notify;
-			$db{$next_key}->{'start'} = $db{$next_key}->{'end'} if $db{$next_key}->{'start'} > $db{$next_key}->{'end'};
-			die if $db{$next_key}->{'start'} > $db{$next_key}->{'end'};
 		}
-		die if $start > $end;
 	}
 }
 
-@sorted_keys = sort sortStart keys %db if $re_sort;
+maybe_sort_keys();
 
 # GPU time accounting
 my (%running, %runnable, %queued, %batch_avg, %batch_total_avg, %batch_count);
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for trace.pl fixes

[Patchwork]

== Series Details ==

Series: trace.pl fixes
URL   : https://patchwork.freedesktop.org/series/37330/
State : success

== Summary ==

IGT patchset tested on top of latest successful build
098a4013434dafda61fa47bde08c29b36f436e9b igt/prime_mmap_coherency: Close dmabuf after use

with latest DRM-Tip kernel build CI_DRM_3698
f355c45d5d64 drm-tip: 2018y-01m-30d-14h-04m-35s UTC integration manifest

No testlist changes.

Test kms_pipe_crc_basic:
        Subgroup suspend-read-crc-pipe-b:
                pass       -> INCOMPLETE (fi-snb-2520m) fdo#103713

fdo#103713 https://bugs.freedesktop.org/show_bug.cgi?id=103713

fi-bdw-5557u     total:288  pass:267  dwarn:0   dfail:0   fail:0   skip:21  time:423s
fi-bdw-gvtdvm    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:429s
fi-blb-e6850     total:288  pass:223  dwarn:1   dfail:0   fail:0   skip:64  time:374s
fi-bsw-n3050     total:288  pass:242  dwarn:0   dfail:0   fail:0   skip:46  time:496s
fi-bwr-2160      total:288  pass:183  dwarn:0   dfail:0   fail:0   skip:105 time:282s
fi-bxt-dsi       total:288  pass:258  dwarn:0   dfail:0   fail:0   skip:30  time:486s
fi-byt-j1900     total:288  pass:253  dwarn:0   dfail:0   fail:0   skip:35  time:471s
fi-byt-n2820     total:288  pass:249  dwarn:0   dfail:0   fail:0   skip:39  time:469s
fi-cfl-s2        total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:571s
fi-elk-e7500     total:224  pass:168  dwarn:10  dfail:0   fail:0   skip:45 
fi-gdg-551       total:288  pass:179  dwarn:0   dfail:0   fail:1   skip:108 time:282s
fi-glk-1         total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:515s
fi-hsw-4770      total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:392s
fi-hsw-4770r     total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:402s
fi-ilk-650       total:288  pass:228  dwarn:0   dfail:0   fail:0   skip:60  time:414s
fi-ivb-3520m     total:288  pass:259  dwarn:0   dfail:0   fail:0   skip:29  time:458s
fi-ivb-3770      total:288  pass:255  dwarn:0   dfail:0   fail:0   skip:33  time:419s
fi-kbl-7500u     total:288  pass:263  dwarn:1   dfail:0   fail:0   skip:24  time:462s
fi-kbl-7560u     total:288  pass:269  dwarn:0   dfail:0   fail:0   skip:19  time:498s
fi-kbl-7567u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:456s
fi-kbl-r         total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:507s
fi-skl-6260u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:432s
fi-skl-6600u     total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:517s
fi-skl-6700hq    total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:529s
fi-skl-6700k2    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:489s
fi-skl-6770hq    total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:493s
fi-skl-guc       total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:418s
fi-skl-gvtdvm    total:288  pass:265  dwarn:0   dfail:0   fail:0   skip:23  time:433s
fi-snb-2520m     total:245  pass:211  dwarn:0   dfail:0   fail:0   skip:33 
fi-snb-2600      total:288  pass:248  dwarn:0   dfail:0   fail:0   skip:40  time:399s
Blacklisted hosts:
fi-glk-dsi       total:288  pass:258  dwarn:0   dfail:0   fail:0   skip:30  time:472s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_834/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev