[PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Hitoshi Mitake]

Adding general performance benchmarking subcommand to perf.
This patch adds bench/sched-pipe.c.

bench/sched-pipe.c is a benchmark program
to measure performance of pipe() system call.
This benchmark is based on pipe-test-1m.c by Ingo Molnar.
http://people.redhat.com/mingo/cfs-scheduler/tools/pipe-test-1m.c

Example of use:
% perf bench sched pipe               # Use default options
4.575  	     	   		      # Result is time
% perf bench sched pipe -l 2000	      # This option means "loop 2000 counts"
0.024

Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
---
 tools/perf/bench/sched-pipe.c |   89 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 89 insertions(+), 0 deletions(-)
 create mode 100644 tools/perf/bench/sched-pipe.c

diff --git a/tools/perf/bench/sched-pipe.c b/tools/perf/bench/sched-pipe.c
new file mode 100644
index 0000000..3f22f54
--- /dev/null
+++ b/tools/perf/bench/sched-pipe.c
@@ -0,0 +1,89 @@
+/*
+ *
+ * builtin-bench-pipe.c
+ *
+ * pipe: Benchmark for pipe()
+ *
+ * Based on pipe-test-1m.c by Ingo Molnar <mingo@redhat.com>
+ *  http://people.redhat.com/mingo/cfs-scheduler/tools/pipe-test-1m.c
+ * Ported to perf by Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
+ *
+ */
+
+#include "../perf.h"
+#include "../util/util.h"
+#include "../util/parse-options.h"
+#include "../builtin.h"
+#include "/bench.h"
+
+#include <unistd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <signal.h>
+#include <sys/wait.h>
+#include <linux/unistd.h>
+#include <string.h>
+#include <errno.h>
+#include <assert.h>
+#include <sys/time.h>
+
+#define LOOPS_DEFAULT 1000000
+static int loops = LOOPS_DEFAULT;
+
+static const struct option options[] = {
+	OPT_INTEGER('l', "loop", &loops,
+		    "Specify number of loops"),
+	OPT_END()
+};
+
+static const char * const bench_sched_pipe_usage[] = {
+	"perf bench sched pipe <options>",
+	NULL
+};
+
+int bench_sched_pipe(int argc, const char **argv,
+		     const char *prefix __used)
+{
+	int pipe_1[2], pipe_2[2];
+	int m = 0, i;
+	struct timeval start, stop, diff;
+
+	/*
+	 * why does "ret" exist?
+	 * discarding returned value of read(), write()
+	 * causes error in building environment for perf
+	 */
+	int ret;
+	pid_t pid;
+
+	argc = parse_options(argc, argv, options,
+			     bench_sched_pipe_usage, 0);
+
+	assert(!pipe(pipe_1));
+	assert(!pipe(pipe_2));
+
+	pid = fork();
+	assert(pid >= 0);
+
+	gettimeofday(&start, NULL);
+
+	if (!pid) {
+		for (i = 0; i < loops; i++) {
+			ret = read(pipe_1[0], &m, sizeof(int));
+			ret = write(pipe_2[1], &m, sizeof(int));
+		}
+	} else {
+		for (i = 0; i < loops; i++) {
+			ret = write(pipe_1[1], &m, sizeof(int));
+			ret = read(pipe_2[0], &m, sizeof(int));
+		}
+	}
+
+	gettimeofday(&stop, NULL);
+	timersub(&stop, &start, &diff);
+	if (!pid)
+		printf("%lu.%03lu\n",
+		       diff.tv_sec, diff.tv_usec/1000);
+
+	return 0;
+}
-- 
1.5.6.5

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Ingo Molnar]

* Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:

> 
> Adding general performance benchmarking subcommand to perf.
> This patch adds bench/sched-pipe.c.
> 
> bench/sched-pipe.c is a benchmark program
> to measure performance of pipe() system call.
> This benchmark is based on pipe-test-1m.c by Ingo Molnar.
> http://people.redhat.com/mingo/cfs-scheduler/tools/pipe-test-1m.c
> 
> Example of use:
> % perf bench sched pipe               # Use default options
> 4.575  	     	   		      # Result is time
> % perf bench sched pipe -l 2000	      # This option means "loop 2000 counts"
> 0.024

ok, -l makes sense.

Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
should perhaps print something like:

  % perf bench sched pipe

  (executing 1000000 pipe operations between two tasks)

     4.575 usecs per op
    218579 ops/sec   

?

	Ingo

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Hitoshi Mitake]

From: Ingo Molnar <mingo@elte.hu>
Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
Date: Tue, 3 Nov 2009 18:29:26 +0100

> 
> * Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:
> 
> > 
> > Adding general performance benchmarking subcommand to perf.
> > This patch adds bench/sched-pipe.c.
> > 
> > bench/sched-pipe.c is a benchmark program
> > to measure performance of pipe() system call.
> > This benchmark is based on pipe-test-1m.c by Ingo Molnar.
> > http://people.redhat.com/mingo/cfs-scheduler/tools/pipe-test-1m.c
> > 
> > Example of use:
> > % perf bench sched pipe               # Use default options
> > 4.575  	     	   		      # Result is time
> > % perf bench sched pipe -l 2000	      # This option means "loop 2000 counts"
> > 0.024
> 
> ok, -l makes sense.
> 
> Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> should perhaps print something like:
> 
>   % perf bench sched pipe
> 
>   (executing 1000000 pipe operations between two tasks)
> 
>      4.575 usecs per op
>     218579 ops/sec   
> 
> ?

I have to admit that single float value output is too simple.
So I'll fix the default output.

But, I believe that simple form makes sense for
processing by scripts or graph tools like gnuplot.
I'll add the option (may be --simple) to switch
friendliness of outputs.

Tools like automatic graph generator
will be helpful for us. I'll write it after
completion of current work.

I'll send new patch series including help improvement
you told in previous mail later.

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Ingo Molnar]

* Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:

> From: Ingo Molnar <mingo@elte.hu>
> Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
> Date: Tue, 3 Nov 2009 18:29:26 +0100
> 
> > 
> > * Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:
> > 
> > > 
> > > Adding general performance benchmarking subcommand to perf.
> > > This patch adds bench/sched-pipe.c.
> > > 
> > > bench/sched-pipe.c is a benchmark program
> > > to measure performance of pipe() system call.
> > > This benchmark is based on pipe-test-1m.c by Ingo Molnar.
> > > http://people.redhat.com/mingo/cfs-scheduler/tools/pipe-test-1m.c
> > > 
> > > Example of use:
> > > % perf bench sched pipe               # Use default options
> > > 4.575  	     	   		      # Result is time
> > > % perf bench sched pipe -l 2000	      # This option means "loop 2000 counts"
> > > 0.024
> > 
> > ok, -l makes sense.
> > 
> > Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> > should perhaps print something like:
> > 
> >   % perf bench sched pipe
> > 
> >   (executing 1000000 pipe operations between two tasks)
> > 
> >      4.575 usecs per op
> >     218579 ops/sec   
> > 
> > ?
> 
> I have to admit that single float value output is too simple.
> So I'll fix the default output.
> 
> But, I believe that simple form makes sense for
> processing by scripts or graph tools like gnuplot.
> I'll add the option (may be --simple) to switch
> friendliness of outputs.

Btw., could you make it Git-ish, i.e.:

 --format=short

or:

 --format=simple

Eventually more format options might be added.

	Ingo

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Hitoshi Mitake]

From: Ingo Molnar <mingo@elte.hu>
Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
Date: Sun, 8 Nov 2009 12:30:13 +0100

> > > 
> > > Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> > > should perhaps print something like:
> > > 
> > >   % perf bench sched pipe
> > > 
> > >   (executing 1000000 pipe operations between two tasks)
> > > 
> > >      4.575 usecs per op
> > >     218579 ops/sec   
> > > 
> > > ?
> > 
> > I have to admit that single float value output is too simple.
> > So I'll fix the default output.
> > 
> > But, I believe that simple form makes sense for
> > processing by scripts or graph tools like gnuplot.
> > I'll add the option (may be --simple) to switch
> > friendliness of outputs.
> 
> Btw., could you make it Git-ish, i.e.:
> 
>  --format=short
> 
> or:
> 
>  --format=simple
> 
> Eventually more format options might be added.

It's good idea.
I have to admit that reserving -s for simple output is not good idea.
I'll do this.

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Ingo Molnar]

* Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:

> From: Ingo Molnar <mingo@elte.hu>
> Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
> Date: Sun, 8 Nov 2009 12:30:13 +0100
> 
> > > > 
> > > > Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> > > > should perhaps print something like:
> > > > 
> > > >   % perf bench sched pipe
> > > > 
> > > >   (executing 1000000 pipe operations between two tasks)
> > > > 
> > > >      4.575 usecs per op
> > > >     218579 ops/sec   
> > > > 
> > > > ?
> > > 
> > > I have to admit that single float value output is too simple.
> > > So I'll fix the default output.
> > > 
> > > But, I believe that simple form makes sense for
> > > processing by scripts or graph tools like gnuplot.
> > > I'll add the option (may be --simple) to switch
> > > friendliness of outputs.
> > 
> > Btw., could you make it Git-ish, i.e.:
> > 
> >  --format=short
> > 
> > or:
> > 
> >  --format=simple
> > 
> > Eventually more format options might be added.
> 
> It's good idea.
> I have to admit that reserving -s for simple output is not good idea.
> I'll do this.

I think --format=simple will be used by scripts mostly, so it doesnt 
matter that it's longer to type. We try to save the shorter options for 
humans and be conservative with them.

Another angle is coherency between different subcommands - and '-s' is 
already used as -s/--sort in other perf subcommands, which does not 
match up with the '-s/--simple' usage.

We try to match what the Git project does here - a good deal of 
infrastructure code in perf came from Git and i find Git very easy to 
use and it's managed well.

It's not a hard rule: not all option name incoherencies are fixable or 
avoidable, and there's no big problem if something slips in - i just 
wanted to mention so that you can keep an eye on it when developing new 
features for perf bench.

	Ingo

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Hitoshi Mitake]

From: Ingo Molnar <mingo@elte.hu>
Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
Date: Mon, 9 Nov 2009 08:55:12 +0100

> 
> * Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:
> 
> > From: Ingo Molnar <mingo@elte.hu>
> > Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
> > Date: Sun, 8 Nov 2009 12:30:13 +0100
> > 
> > > > > 
> > > > > Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> > > > > should perhaps print something like:
> > > > > 
> > > > >   % perf bench sched pipe
> > > > > 
> > > > >   (executing 1000000 pipe operations between two tasks)
> > > > > 
> > > > >      4.575 usecs per op
> > > > >     218579 ops/sec   
> > > > > 
> > > > > ?
> > > > 
> > > > I have to admit that single float value output is too simple.
> > > > So I'll fix the default output.
> > > > 
> > > > But, I believe that simple form makes sense for
> > > > processing by scripts or graph tools like gnuplot.
> > > > I'll add the option (may be --simple) to switch
> > > > friendliness of outputs.
> > > 
> > > Btw., could you make it Git-ish, i.e.:
> > > 
> > >  --format=short
> > > 
> > > or:
> > > 
> > >  --format=simple
> > > 
> > > Eventually more format options might be added.
> > 
> > It's good idea.
> > I have to admit that reserving -s for simple output is not good idea.
> > I'll do this.
> 
> I think --format=simple will be used by scripts mostly, so it doesnt 
> matter that it's longer to type. We try to save the shorter options for 
> humans and be conservative with them.
> 
> Another angle is coherency between different subcommands - and '-s' is 
> already used as -s/--sort in other perf subcommands, which does not 
> match up with the '-s/--simple' usage.
> 
> We try to match what the Git project does here - a good deal of 
> infrastructure code in perf came from Git and i find Git very easy to 
> use and it's managed well.
> 
> It's not a hard rule: not all option name incoherencies are fixable or 
> avoidable, and there's no big problem if something slips in - i just 
> wanted to mention so that you can keep an eye on it when developing new 
> features for perf bench.
> 
> 	Ingo
> 

I added --format as option of bench subcommand,
not of each suites.
Because I thought that flavors of formatting are common thing
across the suites.

Example of use:
% ./perf bench sched pipe           # with no style specify
(executing 1000000 pipe operations between two tasks)

        Total time:5.855 sec
                5.855061 usecs/op
                170792 ops/sec
% ./perf bench --format=simple sched pipe # specified simple
               ^^^^^^^^^^^^^^^	# <- --format is here
5.988

How do you think?
I'll send patch series later.

Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.

[Ingo Molnar]

* Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:

> From: Ingo Molnar <mingo@elte.hu>
> Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
> Date: Mon, 9 Nov 2009 08:55:12 +0100
> 
> > 
> > * Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp> wrote:
> > 
> > > From: Ingo Molnar <mingo@elte.hu>
> > > Subject: Re: [PATCH v2 3/7] Adding general performance benchmarking subcommand to perf.
> > > Date: Sun, 8 Nov 2009 12:30:13 +0100
> > > 
> > > > > > 
> > > > > > Shouldnt we output the unit of measurement, i.e. '4.575 usecs'? Also, we 
> > > > > > should perhaps print something like:
> > > > > > 
> > > > > >   % perf bench sched pipe
> > > > > > 
> > > > > >   (executing 1000000 pipe operations between two tasks)
> > > > > > 
> > > > > >      4.575 usecs per op
> > > > > >     218579 ops/sec   
> > > > > > 
> > > > > > ?
> > > > > 
> > > > > I have to admit that single float value output is too simple.
> > > > > So I'll fix the default output.
> > > > > 
> > > > > But, I believe that simple form makes sense for
> > > > > processing by scripts or graph tools like gnuplot.
> > > > > I'll add the option (may be --simple) to switch
> > > > > friendliness of outputs.
> > > > 
> > > > Btw., could you make it Git-ish, i.e.:
> > > > 
> > > >  --format=short
> > > > 
> > > > or:
> > > > 
> > > >  --format=simple
> > > > 
> > > > Eventually more format options might be added.
> > > 
> > > It's good idea.
> > > I have to admit that reserving -s for simple output is not good idea.
> > > I'll do this.
> > 
> > I think --format=simple will be used by scripts mostly, so it doesnt 
> > matter that it's longer to type. We try to save the shorter options for 
> > humans and be conservative with them.
> > 
> > Another angle is coherency between different subcommands - and '-s' is 
> > already used as -s/--sort in other perf subcommands, which does not 
> > match up with the '-s/--simple' usage.
> > 
> > We try to match what the Git project does here - a good deal of 
> > infrastructure code in perf came from Git and i find Git very easy to 
> > use and it's managed well.
> > 
> > It's not a hard rule: not all option name incoherencies are fixable or 
> > avoidable, and there's no big problem if something slips in - i just 
> > wanted to mention so that you can keep an eye on it when developing new 
> > features for perf bench.
> > 
> > 	Ingo
> > 
> 
> I added --format as option of bench subcommand,
> not of each suites.
> Because I thought that flavors of formatting are common thing
> across the suites.

yeah. Maybe it might be librarized into util/ as well in the future, if 
another perf subcommand wants to pick it up.

> 
> Example of use:
> % ./perf bench sched pipe           # with no style specify
> (executing 1000000 pipe operations between two tasks)
> 
>         Total time:5.855 sec
>                 5.855061 usecs/op
>                 170792 ops/sec
> % ./perf bench --format=simple sched pipe # specified simple
>                ^^^^^^^^^^^^^^^	# <- --format is here
> 5.988
> 
> How do you think?
> I'll send patch series later.

Looks good - i picked them up.

Thanks,

	Ingo