From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: LKML <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Eric Dumazet <edumazet@google.com>,
Frederic Weisbecker <fweisbec@gmail.com>,
Chris Mason <clm@fb.com>, Arjan van de Ven <arjan@infradead.org>,
rt@linutronix.de, Rik van Riel <riel@redhat.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
George Spelvin <linux@sciencehorizons.net>,
Len Brown <lenb@kernel.org>
Subject: Re: [patch V2 00/20] timer: Refactor the timer wheel
Date: Mon, 20 Jun 2016 08:05:55 -0700 [thread overview]
Message-ID: <20160620150555.GR3923@linux.vnet.ibm.com> (raw)
In-Reply-To: <20160617121134.417319325@linutronix.de>
On Fri, Jun 17, 2016 at 01:26:28PM -0000, Thomas Gleixner wrote:
> This is the second version of the timer wheel rework series. The first series
> can be found here:
>
> http://lkml.kernel.org/r/20160613070440.950649741@linutronix.de
>
> The series is also available in git:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.timers
Ran some longer rcutorture tests, and the scripting complained about
hangs. This turned out to be due to the 12.5% uncertainty, so I fixed
this by switching the rcutorture stop-test timer to hrtimers. Things are
now working as well as before, with the exception of SRCU, for which I
am getting lots of grace-period stall complaints. This came as a bit
of a surprise. Anyway, I will be reviewing SRCU for timing dependencies.
Thanx, Paul
> Changes vs. V1:
>
> - Addressed the review comments of V1
>
> - Fixed the fallout in tty/metag (noticed by Arjan)
> - Renamed the hlist helper (noticed by Paolo/George)
> - Used the proper mask in get_timer_base() (noticed by Richard)
> - Fixed the inverse state check in internal_add_timer() (noticed by Richard)
> - Simplified the macro maze, removed wrapper (noticed by George)
> - Reordered data retrieval in run_timer() (noticed by George)
>
> - Removed cascading completely
>
> We have a hard cutoff of expiry times at the capacity of the last wheel
> level now. Timers which insist on timeouts longer than that, i.e. ~6days,
> will expire at the cutoff, i.e. ~6 days. From our data gathering the
> largest timeouts are 5 days (networking contrack), which are well in the
> capacity.
>
> To achieve this capacity with HZ=1000 without increasing the storage size
> by another level, we reduced the granularity of the first wheel level from
> 1ms to 4ms. According to our data, there is no user which relies on that
> 1ms granularity and 99% of those timers are canceled before expiry.
>
> As a side effect there is the benefit of better batching in the first level
> which helps networking to avoid rearming timers in the hotpath.
>
> We gathered more data about performance and batching. Compared to mainline the
> following changes have been observed:
>
> - The bad outliers in mainline when the timer wheel needs to be forwarded
> after a long idle sleep are completely gone.
>
> - The total cpu time used for timer softirq processing is significantly
> reduced. Depending on the HZ setting and workload this ranges from factor
> 2 to 6.
>
> - The average invocation period of the timer softirq on an idle system
> increases significantly. Depending on the HZ settings and workload this
> ranges from factor 1.5 to 5. That means that the residency in deep
> c-states should be improved. Have not yet have time to verify this with
> the power tools.
>
> Thanks,
>
> tglx
>
> ---
> arch/x86/kernel/apic/x2apic_uv_x.c | 4
> arch/x86/kernel/cpu/mcheck/mce.c | 4
> block/genhd.c | 5
> drivers/cpufreq/powernv-cpufreq.c | 5
> drivers/mmc/host/jz4740_mmc.c | 2
> drivers/net/ethernet/tile/tilepro.c | 4
> drivers/power/bq27xxx_battery.c | 5
> drivers/tty/metag_da.c | 4
> drivers/tty/mips_ejtag_fdc.c | 4
> drivers/usb/host/ohci-hcd.c | 1
> drivers/usb/host/xhci.c | 2
> include/linux/list.h | 10
> include/linux/timer.h | 30
> kernel/time/tick-internal.h | 1
> kernel/time/tick-sched.c | 46 -
> kernel/time/timer.c | 1099 +++++++++++++++++++++---------------
> lib/random32.c | 1
> net/ipv4/inet_connection_sock.c | 7
> net/ipv4/inet_timewait_sock.c | 5
> 19 files changed, 725 insertions(+), 514 deletions(-)
>
>
next prev parent reply other threads:[~2016-06-20 15:37 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-17 13:26 [patch V2 00/20] timer: Refactor the timer wheel Thomas Gleixner
2016-06-17 13:26 ` [patch V2 01/20] timer: Make pinned a timer property Thomas Gleixner
2016-06-17 13:26 ` [patch V2 02/20] x86/apic/uv: Initialize timer as pinned Thomas Gleixner
2016-06-17 13:26 ` [patch V2 03/20] x86/mce: " Thomas Gleixner
2016-06-17 13:26 ` [patch V2 05/20] driver/net/ethernet/tile: " Thomas Gleixner
2016-06-21 18:14 ` Peter Zijlstra
2016-06-17 13:26 ` [patch V2 04/20] cpufreq/powernv: " Thomas Gleixner
2016-06-17 13:26 ` [patch V2 06/20] drivers/tty/metag_da: " Thomas Gleixner
2016-06-17 13:26 ` [patch V2 07/20] drivers/tty/mips_ejtag: " Thomas Gleixner
2016-06-17 13:26 ` [patch V2 08/20] net/ipv4/inet: Initialize timers " Thomas Gleixner
2016-06-17 13:26 ` [patch V2 09/20] timer: Remove mod_timer_pinned Thomas Gleixner
2016-06-17 13:26 ` [patch V2 10/20] hlist: Add hlist_is_singular_node() helper Thomas Gleixner
2016-06-17 13:26 ` [patch V2 11/20] timer: Give a few structs and members proper names Thomas Gleixner
2016-06-17 13:26 ` [patch V2 12/20] timer: Switch to a non cascading wheel Thomas Gleixner
2016-06-18 9:55 ` George Spelvin
2016-06-24 10:06 ` Thomas Gleixner
2016-06-17 13:26 ` [patch V2 13/20] timer: Remove slack leftovers Thomas Gleixner
2016-06-17 13:26 ` [patch V2 14/20] timer: Move __run_timers() function Thomas Gleixner
2016-06-17 13:26 ` [patch V2 15/20] timer: Optimize collect timers for NOHZ Thomas Gleixner
2016-06-17 13:26 ` [patch V2 16/20] tick/sched: Remove pointless empty function Thomas Gleixner
2016-06-17 13:26 ` [patch V2 17/20] timer: Forward wheel clock whenever possible Thomas Gleixner
2016-06-17 13:26 ` [patch V2 18/20] timer: Only wake softirq if necessary Thomas Gleixner
2016-06-17 13:26 ` [patch V2 19/20] timer: Split out index calculation Thomas Gleixner
2016-06-17 13:26 ` [patch V2 20/20] timer: Optimization for same expiry time in mod_timer() Thomas Gleixner
2016-06-17 13:48 ` [patch V2 00/20] timer: Refactor the timer wheel Eric Dumazet
2016-06-17 13:57 ` Thomas Gleixner
2016-06-17 14:25 ` Eric Dumazet
2016-06-20 13:56 ` Thomas Gleixner
2016-06-20 14:46 ` Arjan van de Ven
2016-06-20 14:46 ` Thomas Gleixner
2016-06-20 14:49 ` Arjan van de Ven
2016-06-20 19:03 ` Rik van Riel
2016-06-21 2:48 ` Eric Dumazet
2016-06-17 14:26 ` Arjan van de Ven
2016-06-20 15:05 ` Paul E. McKenney [this message]
2016-06-20 15:13 ` Thomas Gleixner
2016-06-20 15:41 ` Paul E. McKenney
2016-06-22 7:37 ` Mike Galbraith
2016-06-22 8:44 ` Thomas Gleixner
2016-06-22 9:06 ` Mike Galbraith
2016-06-22 13:37 ` Mike Galbraith
2016-06-22 10:28 ` [LTP] " Cyril Hrubis
2016-06-23 8:27 ` Thomas Gleixner
2016-06-23 11:47 ` Cyril Hrubis
2016-06-23 13:58 ` George Spelvin
2016-06-23 14:10 ` Thomas Gleixner
2016-06-23 15:11 ` Cyril Hrubis
2016-06-23 15:21 ` Thomas Gleixner
2016-06-23 16:31 ` Cyril Hrubis
2016-06-26 19:00 ` Pavel Machek
2016-06-26 19:21 ` Arjan van de Ven
2016-06-26 20:02 ` Pavel Machek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160620150555.GR3923@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=arjan@infradead.org \
--cc=clm@fb.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=lenb@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@sciencehorizons.net \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=rt@linutronix.de \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).