linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Steven Sistare <steven.sistare@oracle.com>
To: mingo@redhat.com, peterz@infradead.org
Cc: subhra.mazumdar@oracle.com, dhaval.giani@oracle.com,
	daniel.m.jordan@oracle.com, pavel.tatashin@microsoft.com,
	matt@codeblueprint.co.uk, umgwanakikbuti@gmail.com,
	riel@redhat.com, jbacik@fb.com, juri.lelli@redhat.com,
	valentin.schneider@arm.com, vincent.guittot@linaro.org,
	quentin.perret@arm.com, linux-kernel@vger.kernel.org
Subject: hackbench run scripts
Date: Fri, 9 Nov 2018 10:02:37 -0500	[thread overview]
Message-ID: <0eaa3ee9-64d6-4739-eec9-e28befa0e97f@oracle.com> (raw)
In-Reply-To: <1541767840-93588-1-git-send-email-steven.sistare@oracle.com>

[-- Attachment #1: Type: text/plain, Size: 1935 bytes --]

Hi folks,
  I am attaching the bash helper scripts I used to run and post-process hackbench
in case you find them useful.  They compute the statistics and print a nicely
formatted result:

  feat - Enable/disable one or more sched_features.
  runmany - run a command many times and print average time and stdev.
  hackdiff - Print difference between 2 identical series of runmany hackbench runs.

You supply the top level script that runs a series of groups.  Example:

---- hackseries ---------------
#!/bin/sh

iters=50000
groups="1 2 3 4"
runs=5

tmpfile=temp$$.out
rm -f $tmpfile

runall() {
    for g in $groups ; do
        echo hackbench $g process $iters | tee -a $tmpfile
        runmany $runs hackbench $g process $iters | tee -a $tmpfile
    done
}

sudo feat NO_STEAL ; runall ; echo
sudo feat STEAL    ; runall ; echo
hackdiff < $tmpfile
rm -f $tmpfile
--------------------------------

$ hackseries
hackbench 1 process 50000
 4.096  4.090  4.073  4.129  4.092     Average 4.096   stdev 0.5%
hackbench 2 process 50000
 7.215  7.214  7.242  7.208  7.224     Average 7.220   stdev 0.2%
hackbench 3 process 50000
10.082 10.035 10.049 10.068 10.082     Average 10.063   stdev 0.2%
hackbench 4 process 50000
12.851 12.948 12.905 12.916 12.994     Average 12.922   stdev 0.4%

hackbench 1 process 50000
 3.193  3.261  3.257  3.223  3.247     Average 3.236   stdev 0.9%
hackbench 2 process 50000
 6.020  5.960  6.003  6.008  5.998     Average 5.997   stdev 0.4%
hackbench 3 process 50000
 8.598  8.692  8.536  8.661  8.468     Average 8.591   stdev 1.1%
hackbench 4 process 50000
11.201 11.148 11.053 11.174 11.127     Average 11.140   stdev 0.5%

          --- base --    --- new ---
groups    time %stdev    time %stdev  %speedup
     1   4.096    0.5   3.236    0.9      26.5
     2   7.220    0.2   5.997    0.4      20.3
     3  10.063    0.2   8.591    1.1      17.1
     4  12.922    0.4  11.140    0.5      15.9

- Steve

[-- Attachment #2: feat --]
[-- Type: text/plain, Size: 836 bytes --]

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
# 
# Author: Steve Sistare <steven.sistare@oracle.com>
#
# Usage: feat [feature1] [feature2] ...
#   Enable/disable one or more sched_features.
#   With no arguments, print the currently enabled features.

if [[ ! -r /sys/kernel/debug ]]; then
        echo "Permission denied. You must be root."
        exit 1
elif [[ ! -r /sys/kernel/debug/sched_features ]]; then
        echo "debugfs not mounted.  Please run as root:"
        echo "    mount -t debugfs none /sys/kernel/debug"
        exit 1
fi

if [ $# == 0 ] ; then
        cat /sys/kernel/debug/sched_features | sed 's/ /\n/g'
else
        for feat in $* ; do
                if ! echo $feat > /sys/kernel/debug/sched_features ; then
                        echo Unrecognized feature $feat
                fi
        done
fi

exit 0

[-- Attachment #3: runmany --]
[-- Type: text/plain, Size: 1574 bytes --]

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Author: Steve Sistare <steven.sistare@oracle.com>
#
# Usage: runmany <repeat> command args ...
#   Run command <repeat> times, extract time, and print average and stdev.
#
# Command must print a line of the form "Time %f" (eg, like hackbench).


# Convert float $1 to integral fixed-point value with $2 decimal places.
#
ftoi()
{
	declare -i tens=10**$2
	echo $(echo "scale=0; ($tens * $1) / 1" | bc -q)
}

# Convert integral fixed-point $1 with $2 decimal places to floating 
# point string.
#
itof()
{
	declare -i i frac tens x=$1
	[[ $x -lt 0 ]] && sign='-' x=(-$x)
	tens=10**$2
	i=$x/tens
	frac=$x%tens
	printf "%s%d.%0${2}d" "$sign" $i $frac
}

# Return the average of all arguments.
#
average()
{
        declare -i x avg=0

	for x in $* ; do
                avg=avg+x
        done
        avg=avg/$#
        echo $avg
}

# Return the stdev of all args, as a percent of the average, as a float with
# 6 decimal places.
#
stdev()
{
        declare -i x var=0
        declare -i avg=$(average $@)

	for x in $* ; do
                x=x-avg
                var=var+x*x
        done
        echo "scale=6; sqrt($var / ($# - 1)) * 100 / $avg" | bc -q
}

declare -i n=$1
declare -a -i t

if [[ $n -eq 0 ]] ; then
	echo "usage: runmany num command args ..."
	exit 1
fi

cmd="${@:2:$#}"

for ((i=0; i<$n; i=i+1)) ; do
	t1=$($cmd | grep Time | awk '{print $2}')
	t[$i]=$(ftoi $t1 3)
	printf "%6.3f " $t1
done

avg=$(average ${t[*]})
favg=$(itof avg 3)
std=$(stdev ${t[*]})
printf "    Average %.3f   stdev %.1f%%\n" $favg $std


[-- Attachment #4: hackdiff --]
[-- Type: text/plain, Size: 2161 bytes --]

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Author: Steve Sistare <steven.sistare@oracle.com>
#
# Usage: hackdiff
#   Print difference between 2 identical series of hackbench runs
#
# Reads lines from stdin of this form (eg, from runmany):
#  hackbench 1 process 100000
#  5.753  5.421  5.254  6.044  5.989     Average 5.692   stdev 6.1%
#
# The first occurrence of a hackbench group line is the base series, and the
# next occurrence with identical parameters is the new series.

# Convert float $1 to integral fixed-point value with $2 decimal places
#
ftoi()
{
        declare -i tens=10**$2
        echo $(echo "scale=0; ($tens * $1) / 1" | bc -q)
}

# Convert integral fixed-point $1 with $2 decimal places to floating point
# string
#
itof()
{
        declare -i i frac tens x=$1
        tens=10**$2
	[[ $x -lt 0 ]] && sign='-' x=(-$x)
        i=$x/tens
        frac=$x%tens
        printf "%s%d.%0${2}d" "$sign" $i $frac
}

declare -a t1 t2 t3 groups
declare -a words
declare -a t1  t2  d1  d2 words groups seen_group
declare -i i t1d t2d t3 n=0

after=false

while read buf ; do
	if [[ ${buf:0:9} == "hackbench" ]]; then
		words=($buf)
		group=${words[1]}
		if [[ ${seen_group[$group]} == true ]]; then
			if [[ $after == false ]]; then
				n=0
				after=true
			fi
			if [[ ${groups[$n]} != $group ]]; then
				echo "expected group ${groups[$n]} " \
				     "but found $group"
				exit 1
			fi
		fi
		seen_group[$group]=true
		groups[$n]=$group

	elif $(echo $buf | grep -q Average) ; then
		words=($(echo $buf | sed ' s/^.*Average *//'))
		if [[ $after == true ]]; then
			t2[$n]=${words[0]}
			d=${words[2]}
			d2[$n]=${d%\%}
		else
			t1[$n]=${words[0]}
			d=${words[2]}
			d1[$n]=${d%\%}
		fi
		n=($n+1)
	fi
done

printf "%21s  %13s\n" "--- base --" "--- new ---"
printf "%6s  %6s %6s  %6s %6s  %8s\n" \
    groups  time \%stdev  time \%stdev  \%speedup

for ((i=0; i<$n; i++)) ; do
	t1d=$(ftoi ${t1[$i]} 3)
	t2d=$(ftoi ${t2[$i]} 3)
        t3=($t1d*1000/$t2d-1000)
        t3=($t3*100)
        speedup=$(itof $t3 3)
	printf "%6d  %6.3f %6.1f  %6.3f %6.1f  %8.1f\n" \
	    ${groups[$i]} ${t1[$i]} ${d1[$i]} ${t2[$i]} ${d2[$i]} $speedup
done


      parent reply	other threads:[~2018-11-09 15:03 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-09 12:50 [PATCH v3 00/10] steal tasks to improve CPU utilization Steve Sistare
2018-11-09 12:50 ` [PATCH v3 01/10] sched: Provide sparsemask, a reduced contention bitmap Steve Sistare
2018-11-27 15:16   ` Steven Sistare
2018-11-28  1:19     ` Omar Sandoval
2018-12-06 16:07       ` Steven Sistare
2018-12-06 18:19         ` Omar Sandoval
2018-11-09 12:50 ` [PATCH v3 02/10] sched/topology: Provide hooks to allocate data shared per LLC Steve Sistare
2018-11-09 12:50 ` [PATCH v3 03/10] sched/topology: Provide cfs_overload_cpus bitmap Steve Sistare
2018-11-09 17:38   ` Valentin Schneider
2018-11-19 17:32     ` Steven Sistare
2018-11-20 12:52       ` Valentin Schneider
2018-11-12 16:42   ` Valentin Schneider
2018-11-19 17:33     ` Steven Sistare
2018-11-20 12:42       ` Valentin Schneider
2018-11-26 19:06         ` Steven Sistare
2018-12-03 16:56           ` Valentin Schneider
2018-12-06 16:40             ` Steven Sistare
2018-12-06 17:28               ` Valentin Schneider
2018-11-09 12:50 ` [PATCH v3 04/10] sched/fair: Dynamically update cfs_overload_cpus Steve Sistare
2018-11-09 12:50 ` [PATCH v3 05/10] sched/fair: Hoist idle_stamp up from idle_balance Steve Sistare
2018-11-09 19:07   ` Valentin Schneider
2018-11-19 17:31     ` Steven Sistare
2018-11-20 10:24       ` Valentin Schneider
2018-11-09 12:50 ` [PATCH v3 06/10] sched/fair: Generalize the detach_task interface Steve Sistare
2018-11-09 12:50 ` [PATCH v3 07/10] sched/fair: Provide can_migrate_task_llc Steve Sistare
2018-11-09 12:50 ` [PATCH v3 08/10] sched/fair: Steal work from an overloaded CPU when CPU goes idle Steve Sistare
2018-11-09 12:50 ` [PATCH v3 09/10] sched/fair: disable stealing if too many NUMA nodes Steve Sistare
2018-11-09 12:50 ` [PATCH v3 10/10] sched/fair: Provide idle search schedstats Steve Sistare
2018-11-10 17:08   ` kbuild test robot
2018-11-09 15:02 ` Steven Sistare [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0eaa3ee9-64d6-4739-eec9-e28befa0e97f@oracle.com \
    --to=steven.sistare@oracle.com \
    --cc=daniel.m.jordan@oracle.com \
    --cc=dhaval.giani@oracle.com \
    --cc=jbacik@fb.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=matt@codeblueprint.co.uk \
    --cc=mingo@redhat.com \
    --cc=pavel.tatashin@microsoft.com \
    --cc=peterz@infradead.org \
    --cc=quentin.perret@arm.com \
    --cc=riel@redhat.com \
    --cc=subhra.mazumdar@oracle.com \
    --cc=umgwanakikbuti@gmail.com \
    --cc=valentin.schneider@arm.com \
    --cc=vincent.guittot@linaro.org \
    --subject='Re: hackbench run scripts' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).