From: Mike Galbraith <bitbucket@online.de>
To: Len Brown <lenb@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>, Alex Shi <alex.shi@intel.com>,
mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de,
akpm@linux-foundation.org, arjan@linux.intel.com, pjt@google.com,
namhyung@kernel.org, morten.rasmussen@arm.com,
vincent.guittot@linaro.org, gregkh@linuxfoundation.org,
preeti@linux.vnet.ibm.com, viresh.kumar@linaro.org,
linux-kernel@vger.kernel.org, len.brown@intel.com,
rafael.j.wysocki@intel.com, jkosina@suse.cz,
clark.williams@gmail.com, tony.luck@intel.com,
keescook@chromium.org, mgorman@suse.de, riel@redhat.com,
Linux PM list <linux-pm@vger.kernel.org>
Subject: Re: [patch v7 0/21] sched: power aware scheduling
Date: Tue, 30 Apr 2013 07:16:12 +0200 [thread overview]
Message-ID: <1367298972.4616.41.camel@marge.simpson.net> (raw)
In-Reply-To: <1366989087.30242.11.camel@marge.simpson.net>
On Fri, 2013-04-26 at 17:11 +0200, Mike Galbraith wrote:
> On Wed, 2013-04-17 at 17:53 -0400, Len Brown wrote:
> > On 04/12/2013 12:48 PM, Mike Galbraith wrote:
> > > On Fri, 2013-04-12 at 18:23 +0200, Borislav Petkov wrote:
> > >> On Fri, Apr 12, 2013 at 04:46:50PM +0800, Alex Shi wrote:
> > >>> Thanks a lot for comments, Len!
> > >>
> > >> AFAICT, you kinda forgot to answer his most important question:
> > >>
> > >>> These numbers suggest that this patch series simultaneously
> > >>> has a negative impact on performance and energy required
> > >>> to retire the workload. Why do it?
> > >
> > > Hm. When I tested AIM7 compute on a NUMA box, there was a marked
> > > throughput increase at the low to moderate load end of the test spectrum
> > > IIRC. Fully repeatable. There were also other benefits unrelated to
> > > power, ie mitigation of the evil face of select_idle_sibling(). I
> > > rather liked what I saw during ~big box test-drive.
> > >
> > > (just saying there are other aspects besides joules in there)
> >
> > Mike,
> >
> > Can you re-run your AIM7 measurement with turbo-mode and HT-mode disabled,
> > and then independently re-enable them?
> >
> > If you still see the performance benefit, then that proves
> > that the scheduler hacks are not about tricking into
> > turbo mode, but something else.
>
> I did that today, neither turbo nor HT affected the performance gain. I
> used the same box and patch set as tested before (v4), but plugged into
> linus HEAD. "powersaving" AIM7 numbers are ~identical to those I posted
> before, "performance" is lower at the low end of AIM7 test spectrum, but
> as before, delta goes away once the load becomes hefty.
Well now, that's not exactly what I expected to see for AIM7 compute.
Filesystem is munching cycles otherwise used for compute when load is
spread across the whole box vs consolidated.
performance
PerfTop: 35 irqs/sec kernel:94.3% exact: 0.0% [1000Hz cycles], (all, 80 CPUs)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ______________________________ ________________________________________
9367.00 15.5% jbd2_journal_put_journal_head /lib/modules/3.9.0-default/build/vmlinux
7658.00 12.7% jbd2_journal_add_journal_head /lib/modules/3.9.0-default/build/vmlinux
7042.00 11.7% jbd2_journal_grab_journal_head /lib/modules/3.9.0-default/build/vmlinux
4433.00 7.4% sieve /abuild/mike/aim7/multitask
3248.00 5.4% jbd_lock_bh_state /lib/modules/3.9.0-default/build/vmlinux
3034.00 5.0% do_get_write_access /lib/modules/3.9.0-default/build/vmlinux
2058.00 3.4% mul_double /abuild/mike/aim7/multitask
2038.00 3.4% add_double /abuild/mike/aim7/multitask
1365.00 2.3% native_write_msr_safe /lib/modules/3.9.0-default/build/vmlinux
1333.00 2.2% __find_get_block /lib/modules/3.9.0-default/build/vmlinux
1213.00 2.0% add_long /abuild/mike/aim7/multitask
1208.00 2.0% add_int /abuild/mike/aim7/multitask
1084.00 1.8% __wait_on_bit_lock /lib/modules/3.9.0-default/build/vmlinux
1065.00 1.8% div_double /abuild/mike/aim7/multitask
901.00 1.5% intel_idle /lib/modules/3.9.0-default/build/vmlinux
812.00 1.3% _raw_spin_lock_irqsave /lib/modules/3.9.0-default/build/vmlinux
559.00 0.9% jbd2_journal_dirty_metadata /lib/modules/3.9.0-default/build/vmlinux
464.00 0.8% copy_user_generic_string /lib/modules/3.9.0-default/build/vmlinux
455.00 0.8% div_int /abuild/mike/aim7/multitask
430.00 0.7% string_rtns_1 /abuild/mike/aim7/multitask
419.00 0.7% strncat /lib64/libc-2.11.3.so
412.00 0.7% wake_bit_function /lib/modules/3.9.0-default/build/vmlinux
347.00 0.6% jbd2_journal_cancel_revoke /lib/modules/3.9.0-default/build/vmlinux
346.00 0.6% ext4_mark_iloc_dirty /lib/modules/3.9.0-default/build/vmlinux
306.00 0.5% __brelse /lib/modules/3.9.0-default/build/vmlinux
powersaving
PerfTop: 59 irqs/sec kernel:78.0% exact: 0.0% [1000Hz cycles], (all, 80 CPUs)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ______________________________ ________________________________________
6383.00 22.5% sieve /abuild/mike/aim7/multitask
2380.00 8.4% mul_double /abuild/mike/aim7/multitask
2375.00 8.4% add_double /abuild/mike/aim7/multitask
1678.00 5.9% add_long /abuild/mike/aim7/multitask
1633.00 5.8% add_int /abuild/mike/aim7/multitask
1338.00 4.7% div_double /abuild/mike/aim7/multitask
770.00 2.7% strncat /lib64/libc-2.11.3.so
698.00 2.5% string_rtns_1 /abuild/mike/aim7/multitask
678.00 2.4% copy_user_generic_string /lib/modules/3.9.0-default/build/vmlinux
569.00 2.0% div_int /abuild/mike/aim7/multitask
329.00 1.2% jbd2_journal_put_journal_head /lib/modules/3.9.0-default/build/vmlinux
306.00 1.1% array_rtns /abuild/mike/aim7/multitask
298.00 1.1% do_get_write_access /lib/modules/3.9.0-default/build/vmlinux
270.00 1.0% jbd2_journal_add_journal_head /lib/modules/3.9.0-default/build/vmlinux
258.00 0.9% _int_malloc /lib64/libc-2.11.3.so
251.00 0.9% __find_get_block /lib/modules/3.9.0-default/build/vmlinux
236.00 0.8% __memset /lib/modules/3.9.0-default/build/vmlinux
224.00 0.8% jbd2_journal_grab_journal_head /lib/modules/3.9.0-default/build/vmlinux
221.00 0.8% intel_idle /lib/modules/3.9.0-default/build/vmlinux
161.00 0.6% jbd_lock_bh_state /lib/modules/3.9.0-default/build/vmlinux
161.00 0.6% start_this_handle /lib/modules/3.9.0-default/build/vmlinux
153.00 0.5% __GI_memset /lib64/libc-2.11.3.so
147.00 0.5% ext4_do_update_inode /lib/modules/3.9.0-default/build/vmlinux
135.00 0.5% jbd2_journal_stop /lib/modules/3.9.0-default/build/vmlinux
123.00 0.4% jbd2_journal_dirty_metadata /lib/modules/3.9.0-default/build/vmlinux
performance
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
14 7 0 47716456 255124 674808 0 0 0 0 6183 93733 1 3 95 1 0
0 0 0 47791912 255152 602068 0 0 0 2671 14526 49606 2 2 94 1 0
1 0 0 47794384 255152 603796 0 0 0 0 68 111 0 0 100 0 0
8 6 0 47672340 255156 730040 0 0 0 0 36249 103961 2 8 86 4 0
0 0 0 47793976 255216 604616 0 0 0 2686 5322 6379 2 1 97 0 0
0 0 0 47799128 255216 603108 0 0 0 0 62 106 0 0 100 0 0
3 0 0 47795972 255300 603136 0 0 0 2626 39115 146228 3 5 88 3 0
0 0 0 47797176 255300 603284 0 0 0 43 128 216 0 0 100 0 0
0 0 0 47803244 255300 602580 0 0 0 0 78 124 0 0 100 0 0
0 0 0 47789120 255336 603940 0 0 0 2676 14085 85798 3 3 92 1 0
powersaving
0 0 0 47820780 255516 590292 0 0 0 31 81 126 0 0 100 0 0
0 0 0 47823712 255516 589376 0 0 0 0 107 190 0 0 100 0 0
0 0 0 47826608 255516 588060 0 0 0 0 76 130 0 0 100 0 0
0 0 0 47811260 255632 602080 0 0 0 2678 106 200 0 0 100 0 0
0 0 0 47812548 255632 601892 0 0 0 0 69 110 0 0 100 0 0
0 0 0 47808284 255680 604400 0 0 0 2668 1588 3451 4 2 94 0 0
0 0 0 47810300 255680 603624 0 0 0 0 77 124 0 0 100 0 0
20 3 0 47760764 255720 643744 0 0 1 0 948 2817 2 1 97 0 0
0 0 0 47817828 255756 602400 0 0 1 2703 984 797 2 0 98 0 0
0 0 0 47819548 255756 602532 0 0 0 0 93 158 0 0 100 0 0
1 0 0 47819312 255792 603080 0 0 0 2661 1774 3348 4 2 94 0 0
0 0 0 47821912 255800 602608 0 0 0 2 66 107 0 0 100 0 0
Invisible ink is pretty expensive stuff.
-Mike
next prev parent reply other threads:[~2013-04-30 5:17 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-04 2:00 [patch v7 0/21] sched: power aware scheduling Alex Shi
2013-04-04 2:00 ` [patch v7 01/21] Revert "sched: Introduce temporary FAIR_GROUP_SCHED dependency for load-tracking" Alex Shi
2013-04-04 2:00 ` [patch v7 02/21] sched: set initial value of runnable avg for new forked task Alex Shi
2013-05-06 3:24 ` Preeti U Murthy
2013-04-04 2:00 ` [patch v7 03/21] sched: add sched balance policies in kernel Alex Shi
2013-04-04 2:00 ` [patch v7 04/21] sched: add sysfs interface for sched_balance_policy selection Alex Shi
2013-04-04 2:00 ` [patch v7 05/21] sched: log the cpu utilization at rq Alex Shi
2013-05-06 3:26 ` Preeti U Murthy
2013-05-06 5:22 ` Alex Shi
2013-05-06 12:03 ` Phil Carmody
2013-05-06 12:35 ` Alex Shi
2013-05-06 21:19 ` Paul Turner
2013-04-04 2:00 ` [patch v7 06/21] sched: add new sg/sd_lb_stats fields for incoming fork/exec/wake balancing Alex Shi
2013-04-04 2:00 ` [patch v7 07/21] sched: move sg/sd_lb_stats struct ahead Alex Shi
2013-04-04 2:00 ` [patch v7 08/21] sched: scale_rt_power rename and meaning change Alex Shi
2013-04-04 2:00 ` [patch v7 09/21] sched: get rq potential maximum utilization Alex Shi
2013-04-04 2:00 ` [patch v7 10/21] sched: add power aware scheduling in fork/exec/wake Alex Shi
2013-04-04 2:00 ` [patch v7 11/21] sched: add sched_burst_threshold_ns as wakeup burst indicator Alex Shi
2013-04-04 2:00 ` [patch v7 12/21] sched: using avg_idle to detect bursty wakeup Alex Shi
2013-04-04 2:00 ` [patch v7 13/21] sched: packing transitory tasks in wakeup power balancing Alex Shi
2013-04-04 2:00 ` [patch v7 14/21] sched: add power/performance balance allow flag Alex Shi
2013-04-04 2:00 ` [patch v7 15/21] sched: pull all tasks from source group Alex Shi
2013-04-04 5:59 ` Namhyung Kim
2013-04-06 11:49 ` Alex Shi
2013-04-04 2:00 ` [patch v7 16/21] sched: no balance for prefer_sibling in power scheduling Alex Shi
2013-05-06 3:26 ` Preeti U Murthy
2013-04-04 2:00 ` [patch v7 17/21] sched: add new members of sd_lb_stats Alex Shi
2013-04-04 2:00 ` [patch v7 18/21] sched: power aware load balance Alex Shi
2013-04-04 2:01 ` [patch v7 19/21] sched: lazy power balance Alex Shi
2013-04-04 2:01 ` [patch v7 20/21] sched: don't do power balance on share cpu power domain Alex Shi
2013-04-08 3:17 ` Preeti U Murthy
2013-04-08 3:25 ` Preeti U Murthy
2013-04-08 4:19 ` Alex Shi
2013-04-04 2:01 ` [patch v7 21/21] sched: make sure select_tas_rq_fair get a cpu Alex Shi
2013-04-11 21:02 ` [patch v7 0/21] sched: power aware scheduling Len Brown
2013-04-12 8:46 ` Alex Shi
2013-04-12 16:23 ` Borislav Petkov
2013-04-12 16:48 ` Mike Galbraith
2013-04-12 17:12 ` Borislav Petkov
2013-04-14 1:36 ` Alex Shi
2013-04-17 21:53 ` Len Brown
2013-04-18 1:51 ` Mike Galbraith
2013-04-26 15:11 ` Mike Galbraith
2013-04-30 5:16 ` Mike Galbraith [this message]
2013-04-30 8:30 ` Mike Galbraith
2013-04-30 8:41 ` Ingo Molnar
2013-04-30 9:35 ` Mike Galbraith
2013-04-30 9:49 ` Mike Galbraith
2013-04-30 9:56 ` Mike Galbraith
2013-05-17 8:06 ` Preeti U Murthy
2013-05-20 1:01 ` Alex Shi
2013-05-20 2:30 ` Preeti U Murthy
2013-04-14 1:28 ` Alex Shi
2013-04-14 5:10 ` Alex Shi
2013-04-14 15:59 ` Borislav Petkov
2013-04-15 6:04 ` Alex Shi
2013-04-15 6:16 ` Alex Shi
2013-04-15 9:52 ` Borislav Petkov
2013-04-15 13:50 ` Alex Shi
2013-04-15 23:12 ` Borislav Petkov
2013-04-16 0:22 ` Alex Shi
2013-04-16 10:24 ` Borislav Petkov
2013-04-17 1:18 ` Alex Shi
2013-04-17 7:38 ` Borislav Petkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1367298972.4616.41.camel@marge.simpson.net \
--to=bitbucket@online.de \
--cc=akpm@linux-foundation.org \
--cc=alex.shi@intel.com \
--cc=arjan@linux.intel.com \
--cc=bp@alien8.de \
--cc=clark.williams@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jkosina@suse.cz \
--cc=keescook@chromium.org \
--cc=len.brown@intel.com \
--cc=lenb@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=preeti@linux.vnet.ibm.com \
--cc=rafael.j.wysocki@intel.com \
--cc=riel@redhat.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=vincent.guittot@linaro.org \
--cc=viresh.kumar@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).