[PATCH v3 0/4] scripts: add stack{usage,delta} scripts

[PATCH v3 0/4] scripts: add stack{usage,delta} scripts

[Rasmus Villemoes]

On Wed, Aug 19 2015, Michal Marek <mmarek@suse.cz> wrote:

> Do you plan any new iterations, or is this final despite the [RFC]?
> If it's final, I will apply it.

It's final, modulo a few typos I spotted. I took the opportunity to
fix those and send with a PATCH prefix.

===

The current checkstack.pl script has a few problems, stemming from the
overly simplistic attempt at parsing objdump output with regular
expressions. Since gcc 4.6 introduced the -fstack-usage option, we can
now get the exact stack use instead of resorting to ad hoc methods.

This introduces two small scripts. One for running make with KCFLAGS
set to -fstack-usage, followed by collecting the generated .su files
in a single output file. Another for taking two such output files and
computing the changes in stack use.

2/4 and 3/4 may be too small by themselves; they can easily be
squashed into 1/4.

v2: Use KCFLAGS instead of EXTRA_CFLAGS. A few more details in commit
messages. Simpler option handling in stackusage. Removed accidental
leftover debug prints.

v3: Fix a few typos.

Rasmus Villemoes (4):
  scripts: add stackusage script
  .gitignore: add *.su pattern
  kbuild: remove *.su files generated by -fstack-usage
  scripts: add stackdelta script

 .gitignore         |  1 +
 Makefile           |  1 +
 scripts/stackdelta | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 scripts/stackusage | 33 ++++++++++++++++++++++++++++++
 4 files changed, 94 insertions(+)
 create mode 100755 scripts/stackdelta
 create mode 100755 scripts/stackusage

-- 
2.1.3

[PATCH v3 1/4] scripts: add stackusage script

[Rasmus Villemoes]

The current checkstack.pl script has a few problems, stemming from the
overly simplistic attempt at parsing objdump output with regular
expressions: For example, on x86_64 it doesn't take the push
instruction into account, making it consistently underestimate the
real stack use, and it also doesn't capture stack pointer adjustments
of exactly 128 bytes [1].

Since newer gcc (>= 4.6) knows about -fstack-usage, we might as well
take the information straight from the horse's mouth. This patch
introduces scripts/stackusage, which is a simple wrapper for running
make with KCFLAGS set to -fstack-usage. Example use is

scripts/stackusage -o out.su -j8 lib/

The script understands "-o foo" for writing to 'foo' and -h for a
trivial help text; anything else is passed to make.

Afterwards, we find all newly created .su files, massage them a
little, sort by stack use and write the result to a single output
file.

Note that the function names printed by (at least) gcc 4.7 are
sometimes useless. For example, the first three lines of out.su
generated above are

./lib/decompress_bunzip2.c:155  get_next_block  448     static
./lib/decompress_unlzma.c:537   unlzma  336     static
./lib/vsprintf.c:616    8       304     static

That function '8' is really the static symbol_string(), but it has
been subject to 'interprocedural scalar replacement of aggregates', so
its name in the object file is 'symbol_string.isra.8'. gcc 5.0 doesn't
have this problem; it uses the full name as seen in the object file.

[1] Since gcc encodes that by

48 83 c4 80             add    $0xffffffffffffff80,%rsp

and not

48 81 ec 80 00 00 00    sub    $0x80,%rsp

since -128 fits in an imm8.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 scripts/stackusage | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
 create mode 100755 scripts/stackusage

diff --git a/scripts/stackusage b/scripts/stackusage
new file mode 100755
index 000000000000..8cf26640ef8a
--- /dev/null
+++ b/scripts/stackusage
@@ -0,0 +1,33 @@
+#!/bin/sh
+
+outfile=""
+now=`date +%s`
+
+while [ $# -gt 0 ]
+do
+    case "$1" in
+        -o)
+	    outfile="$2"
+	    shift 2;;
+	-h)
+	    echo "usage: $0 [-o outfile] <make options/args>"
+	    exit 0;;
+	*)  break;;
+    esac
+done
+
+if [ -z "$outfile" ]
+then
+    outfile=`mktemp --tmpdir stackusage.$$.XXXX`
+fi
+
+KCFLAGS="${KCFLAGS} -fstack-usage" make "$@"
+
+# Prepend directory name to file names, remove column information,
+# make file:line/function/size/type properly tab-separated.
+find . -name '*.su' -newermt "@${now}" -print |                     \
+    xargs perl -MFile::Basename -pe                                 \
+        '$d = dirname($ARGV); s#([^:]+:[0-9]+):[0-9]+:#$d/$1\t#;' | \
+    sort -k3,3nr > "${outfile}"
+
+echo "$0: output written to ${outfile}"
-- 
2.1.3

[PATCH v3 2/4] .gitignore: add *.su pattern

[Rasmus Villemoes]

Ignore the *.su files generated by using the gcc option -fstack-usage.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 4ad4a98b884b..9e51ead66a55 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,6 +36,7 @@
 modules.builtin
 Module.symvers
 *.dwo
+*.su
 
 #
 # Top-level generic files
-- 
2.1.3

[PATCH v3 3/4] kbuild: remove *.su files generated by -fstack-usage

[Rasmus Villemoes]

Make sure 'make clean' removes *.su files generated by the gcc option
-fstack-usage.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 6e88c371b32f..a3b3499f94e5 100644
--- a/Makefile
+++ b/Makefile
@@ -1433,6 +1433,7 @@ clean: $(clean-dirs)
 		\( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
 		-o -name '*.ko.*' \
 		-o -name '*.dwo'  \
+		-o -name '*.su'  \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
 		-o -name modules.builtin -o -name '.tmp_*.o.*' \
-- 
2.1.3

[PATCH v3 4/4] scripts: add stackdelta script

[Rasmus Villemoes]

This adds a simple perl script for reading two files as produced by
the stackusage script and computing the changes in stack usage. For
example:

$ scripts/stackusage -o /tmp/old.su CC=gcc-4.7 -j8 fs/ext4/
$ scripts/stackusage -o /tmp/new.su CC=gcc-5.0 -j8 fs/ext4/
$ scripts/stackdelta /tmp/{old,new}.su | sort -k5,5g

shows that gcc 5.0 generally produces less stack-hungry code than gcc
4.7. Obviously, the script can also be used for measuring the effect
of commits, .config tweaks or whatnot.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
---
 scripts/stackdelta | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 scripts/stackdelta

diff --git a/scripts/stackdelta b/scripts/stackdelta
new file mode 100755
index 000000000000..48eabf2f48f8
--- /dev/null
+++ b/scripts/stackdelta
@@ -0,0 +1,59 @@
+#!/usr/bin/perl
+
+# Read two files produced by the stackusage script, and show the
+# delta between them.
+#
+# Currently, only shows changes for functions listed in both files. We
+# could add an option to show also functions which have vanished or
+# appeared (which would often be due to gcc making other inlining
+# decisions).
+#
+# Another possible option would be a minimum absolute value for the
+# delta.
+#
+# A third possibility is for sorting by delta, but that can be
+# achieved by piping to sort -k5,5g.
+
+sub read_stack_usage_file {
+    my %su;
+    my $f = shift;
+    open(my $fh, '<', $f)
+	or die "cannot open $f: $!";
+    while (<$fh>) {
+	chomp;
+	my ($file, $func, $size, $type) = split;
+	# Old versions of gcc (at least 4.7) have an annoying quirk in
+	# that a (static) function whose name has been changed into
+	# for example ext4_find_unwritten_pgoff.isra.11 will show up
+	# in the .su file with a name of just "11". Since such a
+	# numeric suffix is likely to change across different
+	# commits/compilers/.configs or whatever else we're trying to
+	# tweak, we can't really track those functions, so we just
+	# silently skip them.
+	#
+	# Newer gcc (at least 5.0) report the full name, so again,
+	# since the suffix is likely to change, we strip it.
+	next if $func =~ m/^[0-9]+$/;
+	$func =~ s/\..*$//;
+	# Line numbers are likely to change; strip those.
+	$file =~ s/:[0-9]+$//;
+	$su{"${file}\t${func}"} = {size => $size, type => $type};
+    }
+    close($fh);
+    return \%su;
+}
+
+@ARGV == 2
+    or die "usage: $0 <old> <new>";
+
+my $old = read_stack_usage_file($ARGV[0]);
+my $new = read_stack_usage_file($ARGV[1]);
+my @common = sort grep {exists $new->{$_}} keys %$old;
+for (@common) {
+    my $x = $old->{$_}{size};
+    my $y = $new->{$_}{size};
+    my $delta = $y - $x;
+    if ($delta) {
+	printf "%s\t%d\t%d\t%+d\n", $_, $x, $y, $delta;
+    }
+}
-- 
2.1.3

Re: [PATCH v3 0/4] scripts: add stack{usage,delta} scripts

[Michal Marek]

On 2015-08-20 11:53, Rasmus Villemoes wrote:
> On Wed, Aug 19 2015, Michal Marek <mmarek@suse.cz> wrote:
> 
>> Do you plan any new iterations, or is this final despite the [RFC]?
>> If it's final, I will apply it.
> 
> It's final, modulo a few typos I spotted. I took the opportunity to
> fix those and send with a PATCH prefix.

Thanks. I applied the series to kbuild.git#misc now.

Michal