* [PATCH] Revert "clocksource: Make clocksource validation work for all clocksources" @ 2018-11-12 18:56 Michael Zhivich 2018-11-12 22:23 ` John Stultz 0 siblings, 1 reply; 3+ messages in thread From: Michael Zhivich @ 2018-11-12 18:56 UTC (permalink / raw) To: linux-kernel; +Cc: omosnace, arnd, jason.wessel, john.stultz, Michael Zhivich Revert commit 1f45f1f33c8c ("clocksource: Make clocksource validation work for all clocksources") to restore correct clocksource_delta() computation for clocksources that wrap frequently, while retaining the check for tsc drifting. Truncating result of clocksource_delta() to 0 causes incorrect behavior for clocksources that wrap frequently (e.g. acpi_pm which is only 24-bit wide). In particular, large time deltas (e.g. last = 0x000000, now = 0x800000) will be incorrectly computed as 0. If acpi_pm is used as the clocksource watchdog, and machine is under heavy load, the time period for the watchdog check may be significantly longer than the requested 0.5 seconds. If the watchdog check is delayed by 2 seconds (observed behavior), then acpi_pm time delta will be 2.5 sec * 3579545 ticks/sec = 8948863 = 0x888c3f which will be treated as negative and truncated to 0. This behavior will cause tsc to be incorrectly declared unstable in clocksource_watchdog(), as it no longer agrees with acpi_pm. Signed-off-by: Michael Zhivich <mzhivich@akamai.com> --- kernel/time/timekeeping_internal.h | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/kernel/time/timekeeping_internal.h b/kernel/time/timekeeping_internal.h index bcbb52db2256..21463e20d008 100644 --- a/kernel/time/timekeeping_internal.h +++ b/kernel/time/timekeeping_internal.h @@ -18,11 +18,7 @@ static inline u64 clocksource_delta(u64 now, u64 last, u64 mask) { u64 ret = (now - last) & mask; - /* - * Prevent time going backwards by checking the MSB of mask in - * the result. If set, return 0. - */ - return ret & ~(mask >> 1) ? 0 : ret; + return (s64) ret > 0 ? ret : 0; } #else static inline u64 clocksource_delta(u64 now, u64 last, u64 mask) -- 2.19.1 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] Revert "clocksource: Make clocksource validation work for all clocksources" 2018-11-12 18:56 [PATCH] Revert "clocksource: Make clocksource validation work for all clocksources" Michael Zhivich @ 2018-11-12 22:23 ` John Stultz 2018-11-13 19:52 ` Zhivich, Michael 0 siblings, 1 reply; 3+ messages in thread From: John Stultz @ 2018-11-12 22:23 UTC (permalink / raw) To: Michael Zhivich; +Cc: lkml, Ondrej Mosnacek, Arnd Bergmann, Jason Wessel On Mon, Nov 12, 2018 at 10:56 AM, Michael Zhivich <mzhivich@akamai.com> wrote: > Revert commit 1f45f1f33c8c ("clocksource: Make clocksource validation work > for all clocksources") to restore correct clocksource_delta() computation > for clocksources that wrap frequently, while retaining the check for tsc > drifting. > > Truncating result of clocksource_delta() to 0 causes incorrect behavior for > clocksources that wrap frequently (e.g. acpi_pm which is only 24-bit wide). > In particular, large time deltas (e.g. last = 0x000000, now = 0x800000) > will be incorrectly computed as 0. > > If acpi_pm is used as the clocksource watchdog, and machine is under heavy > load, the time period for the watchdog check may be significantly longer > than the requested 0.5 seconds. If the watchdog check is delayed by 2 > seconds (observed behavior), then acpi_pm time delta will be > > 2.5 sec * 3579545 ticks/sec = 8948863 = 0x888c3f > > which will be treated as negative and truncated to 0. This behavior will > cause tsc to be incorrectly declared unstable in clocksource_watchdog(), as > it no longer agrees with acpi_pm. Thanks for raising this issue and submitting the patch! Yea, this is a concern particularly with quick wrapping clocksources. Though I worry that if you're already blocking the watchdog from running for 2.5 seconds, you're likely to also block the watchdog for more then 5 seconds, which if I'm remembering would result in the same problem? In other words, does this really solve the problem, or does it just push the bar a little further out? So, I'm wondering to really fix this, do we need to find some way to raise the priority of the clocksource watchdog, so it isn't deferred for quite so long? thanks -john ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] Revert "clocksource: Make clocksource validation work for all clocksources" 2018-11-12 22:23 ` John Stultz @ 2018-11-13 19:52 ` Zhivich, Michael 0 siblings, 0 replies; 3+ messages in thread From: Zhivich, Michael @ 2018-11-13 19:52 UTC (permalink / raw) To: John Stultz Cc: lkml, Ondrej Mosnacek, Arnd Bergmann, Jason Wessel, Hunt, Joshua, Banerjee, Debabrata On 11/12/18, 5:23 PM, "John Stultz" <john.stultz@linaro.org> wrote: On Mon, Nov 12, 2018 at 10:56 AM, Michael Zhivich <mzhivich@akamai.com> wrote: > Revert commit 1f45f1f33c8c ("clocksource: Make clocksource validation work > for all clocksources") to restore correct clocksource_delta() computation > for clocksources that wrap frequently, while retaining the check for tsc > drifting. > > Truncating result of clocksource_delta() to 0 causes incorrect behavior for > clocksources that wrap frequently (e.g. acpi_pm which is only 24-bit wide). > In particular, large time deltas (e.g. last = 0x000000, now = 0x800000) > will be incorrectly computed as 0. > > If acpi_pm is used as the clocksource watchdog, and machine is under heavy > load, the time period for the watchdog check may be significantly longer > than the requested 0.5 seconds. If the watchdog check is delayed by 2 > seconds (observed behavior), then acpi_pm time delta will be > > 2.5 sec * 3579545 ticks/sec = 8948863 = 0x888c3f > > which will be treated as negative and truncated to 0. This behavior will > cause tsc to be incorrectly declared unstable in clocksource_watchdog(), as > it no longer agrees with acpi_pm. Thanks for raising this issue and submitting the patch! Yea, this is a concern particularly with quick wrapping clocksources. Though I worry that if you're already blocking the watchdog from running for 2.5 seconds, you're likely to also block the watchdog for more then 5 seconds, which if I'm remembering would result in the same problem? In other words, does this really solve the problem, or does it just push the bar a little further out? So, I'm wondering to really fix this, do we need to find some way to raise the priority of the clocksource watchdog, so it isn't deferred for quite so long? thanks -john Thanks for the quick response. In principle, I agree that a proper solution would have to ensure that watchdog timer is not blocked for too long. My understanding is that watchdog work is triggered via TIMER softirq and will get pushed to ksoftirqd when the system is busy. In particular, it appears that do_softirq() and invoke_softirq() both check ksoftirqd_running() before deciding to actually do work. One interesting bit is that ksoftirq_running() will disregard an active ksoftirqd thread and return "false" when HI_SOFTIRQ or TASKLET_SOFTIRQ is set (resulting in at least a single pass over pending softirqs). I think it would make sense to add TIMER_SOFTIRQ to this exception list as well. In fact, looking back at the commit log, I'm finding similar thoughts on the subject: commit 3c53776e29f81719efcf8f7a6e30cdf753bee94d Author: Linus Torvalds <torvalds@linux-foundation.org> Date: Mon Jan 8 11:51:04 2018 -0800 Mark HI and TASKLET softirq synchronous ... We should probably also consider the timer softirqs to be synchronous and not be delayed to ksoftirqd (since they were the issue with the earlier watchdog problems), but that should be done as a separate patch. This does only the tasklet cases. If that makes sense, I'm happy to write up a patch. Thanks, ~ Michael ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-11-13 19:52 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-11-12 18:56 [PATCH] Revert "clocksource: Make clocksource validation work for all clocksources" Michael Zhivich 2018-11-12 22:23 ` John Stultz 2018-11-13 19:52 ` Zhivich, Michael
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.