* [patch V2 01/64] tile: Convert VDSO timekeeping to the precise mechanism
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 02/64] timekeeping: Simplify arch_gettimeoffset() Thomas Gleixner
` (63 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: tile-fix-broken-vdso.patch --]
[-- Type: text/plain, Size: 2131 bytes --]
The code was only halfarsed converted to the new VSDO update mechanism
and still uses the inaccurate base value which lacks the fractional
part of xtime_nsec. Fix it up.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/tile/kernel/time.c | 9 ++++-----
arch/tile/kernel/vdso/vgettimeofday.c | 7 ++++---
2 files changed, 8 insertions(+), 8 deletions(-)
Index: tip/arch/tile/kernel/time.c
===================================================================
--- tip.orig/arch/tile/kernel/time.c
+++ tip/arch/tile/kernel/time.c
@@ -260,7 +260,6 @@ void update_vsyscall_tz(void)
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec wall_time = tk_xtime(tk);
struct timespec *wtm = &tk->wall_to_monotonic;
struct clocksource *clock = tk->clock;
@@ -271,12 +270,12 @@ void update_vsyscall(struct timekeeper *
++vdso_data->tb_update_count;
smp_wmb();
vdso_data->xtime_tod_stamp = clock->cycle_last;
- vdso_data->xtime_clock_sec = wall_time.tv_sec;
- vdso_data->xtime_clock_nsec = wall_time.tv_nsec;
+ vdso_data->xtime_clock_sec = tk->xtime_sec;
+ vdso_data->xtime_clock_nsec = tk->xtime_nsec;
vdso_data->wtom_clock_sec = wtm->tv_sec;
vdso_data->wtom_clock_nsec = wtm->tv_nsec;
- vdso_data->mult = clock->mult;
- vdso_data->shift = clock->shift;
+ vdso_data->mult = tk->mult;
+ vdso_data->shift = tk->shift;
smp_wmb();
++vdso_data->tb_update_count;
}
Index: tip/arch/tile/kernel/vdso/vgettimeofday.c
===================================================================
--- tip.orig/arch/tile/kernel/vdso/vgettimeofday.c
+++ tip/arch/tile/kernel/vdso/vgettimeofday.c
@@ -83,10 +83,11 @@ int __vdso_gettimeofday(struct timeval *
if (count & 1)
continue;
- cycles = (get_cycles() - vdso_data->xtime_tod_stamp);
- ns = (cycles * vdso_data->mult) >> vdso_data->shift;
sec = vdso_data->xtime_clock_sec;
- ns += vdso_data->xtime_clock_nsec;
+ cycles = get_cycles() - vdso_data->xtime_tod_stamp;
+ ns = (cycles * vdso_data->mult) + vdso_data->xtime_clock_nsec;
+ ns >>= vdso_data->shift;
+
if (ns >= NSEC_PER_SEC) {
ns -= NSEC_PER_SEC;
sec += 1;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 02/64] timekeeping: Simplify arch_gettimeoffset()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
2014-07-16 21:03 ` [patch V2 01/64] tile: Convert VDSO timekeeping to the precise mechanism Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 03/64] hrtimer: Cleanup hrtimer accessors to the timekepeing state Thomas Gleixner
` (62 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Geert Uytterhoeven
[-- Attachment #1: timekeeping-simplify-arch-offset.patch --]
[-- Type: text/plain, Size: 2038 bytes --]
Provide a default stub function instead of having the extra
conditional. Cuts binary size on a m68k build by ~100 bytes.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
---
kernel/time/timekeeping.c | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -153,16 +153,10 @@ static void tk_setup_internals(struct ti
/* Timekeeper helper functions. */
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
-u32 (*arch_gettimeoffset)(void);
-
-u32 get_arch_timeoffset(void)
-{
- if (likely(arch_gettimeoffset))
- return arch_gettimeoffset();
- return 0;
-}
+static u32 default_arch_gettimeoffset(void) { return 0; }
+u32 (*arch_gettimeoffset)(void) = default_arch_gettimeoffset;
#else
-static inline u32 get_arch_timeoffset(void) { return 0; }
+static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
static inline s64 timekeeping_get_ns(struct timekeeper *tk)
@@ -182,7 +176,7 @@ static inline s64 timekeeping_get_ns(str
nsec >>= tk->shift;
/* If arch requires, add in get_arch_timeoffset() */
- return nsec + get_arch_timeoffset();
+ return nsec + arch_gettimeoffset();
}
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
@@ -202,7 +196,7 @@ static inline s64 timekeeping_get_ns_raw
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
/* If arch requires, add in get_arch_timeoffset() */
- return nsec + get_arch_timeoffset();
+ return nsec + arch_gettimeoffset();
}
static RAW_NOTIFIER_HEAD(pvclock_gtod_chain);
@@ -282,7 +276,7 @@ static void timekeeping_forward_now(stru
tk->xtime_nsec += cycle_delta * tk->mult;
/* If arch requires, add in get_arch_timeoffset() */
- tk->xtime_nsec += (u64)get_arch_timeoffset() << tk->shift;
+ tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
tk_normalize_xtime(tk);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 03/64] hrtimer: Cleanup hrtimer accessors to the timekepeing state
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
2014-07-16 21:03 ` [patch V2 01/64] tile: Convert VDSO timekeeping to the precise mechanism Thomas Gleixner
2014-07-16 21:03 ` [patch V2 02/64] timekeeping: Simplify arch_gettimeoffset() Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 04/64] ktime: Kill non-scalar ktime_t implementation for 2038 Thomas Gleixner
` (61 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-ed502d8.patch --]
[-- Type: text/plain, Size: 6077 bytes --]
From: John Stultz <john.stultz@linaro.org>
Rather then having two similar but totally different implementations
that provide timekeeping state to the hrtimer code, try to unify the
two implementations to be more simliar.
Thus this clarifies ktime_get_update_offsets to
ktime_get_update_offsets_now and changes get_xtime... to
ktime_get_update_offsets_tick.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 9 ++++++---
include/linux/time.h | 2 --
kernel/time/hrtimer.c | 23 ++++++++++-------------
kernel/time/timekeeping.c | 36 +++++++++++++++++++++++-------------
4 files changed, 39 insertions(+), 31 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -331,9 +331,12 @@ extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
-extern ktime_t ktime_get_update_offsets(ktime_t *offs_real, ktime_t *offs_boot,
- ktime_t *offs_tai);
-
+extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
DECLARE_PER_CPU(struct tick_device, tick_cpu_device);
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -133,8 +133,6 @@ unsigned long get_seconds(void);
struct timespec current_kernel_time(void);
struct timespec __current_kernel_time(void); /* does not take xtime_lock */
struct timespec get_monotonic_coarse(void);
-void get_xtime_and_monotonic_and_sleep_offset(struct timespec *xtim,
- struct timespec *wtom, struct timespec *sleep);
void timekeeping_inject_sleeptime(struct timespec *delta);
#define CURRENT_TIME (current_kernel_time())
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -114,21 +114,18 @@ static inline int hrtimer_clockid_to_bas
*/
static void hrtimer_get_softirq_time(struct hrtimer_cpu_base *base)
{
- ktime_t xtim, mono, boot;
- struct timespec xts, tom, slp;
- s32 tai_offset;
-
- get_xtime_and_monotonic_and_sleep_offset(&xts, &tom, &slp);
- tai_offset = timekeeping_get_tai_offset();
-
- xtim = timespec_to_ktime(xts);
- mono = ktime_add(xtim, timespec_to_ktime(tom));
- boot = ktime_add(mono, timespec_to_ktime(slp));
+ ktime_t xtim, mono, boot, tai;
+ ktime_t off_real, off_boot, off_tai;
+
+ mono = ktime_get_update_offsets_tick(&off_real, &off_boot, &off_tai);
+ boot = ktime_add(mono, off_boot);
+ xtim = ktime_add(mono, off_real);
+ tai = ktime_add(xtim, off_tai);
+
base->clock_base[HRTIMER_BASE_REALTIME].softirq_time = xtim;
base->clock_base[HRTIMER_BASE_MONOTONIC].softirq_time = mono;
base->clock_base[HRTIMER_BASE_BOOTTIME].softirq_time = boot;
- base->clock_base[HRTIMER_BASE_TAI].softirq_time =
- ktime_add(xtim, ktime_set(tai_offset, 0));
+ base->clock_base[HRTIMER_BASE_TAI].softirq_time = tai;
}
/*
@@ -673,7 +670,7 @@ static inline ktime_t hrtimer_update_bas
ktime_t *offs_boot = &base->clock_base[HRTIMER_BASE_BOOTTIME].offset;
ktime_t *offs_tai = &base->clock_base[HRTIMER_BASE_TAI].offset;
- return ktime_get_update_offsets(offs_real, offs_boot, offs_tai);
+ return ktime_get_update_offsets_now(offs_real, offs_boot, offs_tai);
}
/*
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1581,29 +1581,39 @@ void do_timer(unsigned long ticks)
}
/**
- * get_xtime_and_monotonic_and_sleep_offset() - get xtime, wall_to_monotonic,
- * and sleep offsets.
- * @xtim: pointer to timespec to be set with xtime
- * @wtom: pointer to timespec to be set with wall_to_monotonic
- * @sleep: pointer to timespec to be set with time in suspend
+ * ktime_get_update_offsets_tick - hrtimer helper
+ * @offs_real: pointer to storage for monotonic -> realtime offset
+ * @offs_boot: pointer to storage for monotonic -> boottime offset
+ * @offs_tai: pointer to storage for monotonic -> clock tai offset
+ *
+ * Returns monotonic time at last tick and various offsets
*/
-void get_xtime_and_monotonic_and_sleep_offset(struct timespec *xtim,
- struct timespec *wtom, struct timespec *sleep)
+ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real, ktime_t *offs_boot,
+ ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
- unsigned long seq;
+ struct timespec ts;
+ ktime_t now;
+ unsigned int seq;
do {
seq = read_seqcount_begin(&timekeeper_seq);
- *xtim = tk_xtime(tk);
- *wtom = tk->wall_to_monotonic;
- *sleep = tk->total_sleep_time;
+
+ ts = tk_xtime(tk);
+
+ *offs_real = tk->offs_real;
+ *offs_boot = tk->offs_boot;
+ *offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&timekeeper_seq, seq));
+
+ now = ktime_set(ts.tv_sec, ts.tv_nsec);
+ now = ktime_sub(now, *offs_real);
+ return now;
}
#ifdef CONFIG_HIGH_RES_TIMERS
/**
- * ktime_get_update_offsets - hrtimer helper
+ * ktime_get_update_offsets_now - hrtimer helper
* @offs_real: pointer to storage for monotonic -> realtime offset
* @offs_boot: pointer to storage for monotonic -> boottime offset
* @offs_tai: pointer to storage for monotonic -> clock tai offset
@@ -1611,7 +1621,7 @@ void get_xtime_and_monotonic_and_sleep_o
* Returns current monotonic time and updates the offsets
* Called from hrtimer_interrupt() or retrigger_next_event()
*/
-ktime_t ktime_get_update_offsets(ktime_t *offs_real, ktime_t *offs_boot,
+ktime_t ktime_get_update_offsets_now(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 04/64] ktime: Kill non-scalar ktime_t implementation for 2038
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (2 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 03/64] hrtimer: Cleanup hrtimer accessors to the timekepeing state Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 05/64] ktime: Sanitize ktime_to_us/ms conversion Thomas Gleixner
` (60 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-d63a235.patch --]
[-- Type: text/plain, Size: 12482 bytes --]
From: John Stultz <john.stultz@linaro.org>
The non-scalar ktime_t implementation is basically a timespec
which has to be changed to support dates past 2038 on 32bit
systems.
This patch removes the non-scalar ktime_t implementation, forcing
the scalar s64 nanosecond version on all architectures.
This may have additional performance overhead on some 32bit
systems when converting between ktime_t and timespec structures,
however the majority of 32bit systems (arm and i386) were already
using scalar ktime_t, so no performance regressions will be seen
on those platforms.
On affected platforms, I'm open to finding optimizations, including
avoiding converting to timespecs where possible.
[ tglx: We can now cleanup the ktime_t.tv64 mess, but thats a
different issue and we can throw a coccinelle script at it ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/arm/Kconfig | 1
arch/hexagon/Kconfig | 1
arch/s390/Kconfig | 1
arch/x86/Kconfig | 1
include/linux/ktime.h | 173 ----------------------------------------------
include/linux/time.h | 11 +-
kernel/time/Kconfig | 4 -
kernel/time/hrtimer.c | 54 --------------
kernel/time/timekeeping.c | 7 -
9 files changed, 7 insertions(+), 246 deletions(-)
Index: tip/arch/arm/Kconfig
===================================================================
--- tip.orig/arch/arm/Kconfig
+++ tip/arch/arm/Kconfig
@@ -64,7 +64,6 @@ config ARM
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
select IRQ_FORCED_THREADING
- select KTIME_SCALAR
select MODULES_USE_ELF_REL
select NO_BOOTMEM
select OLD_SIGACTION
Index: tip/arch/hexagon/Kconfig
===================================================================
--- tip.orig/arch/hexagon/Kconfig
+++ tip/arch/hexagon/Kconfig
@@ -23,7 +23,6 @@ config HEXAGON
select GENERIC_IOMAP
select GENERIC_SMP_IDLE_THREAD
select STACKTRACE_SUPPORT
- select KTIME_SCALAR
select GENERIC_CLOCKEVENTS
select GENERIC_CLOCKEVENTS_BROADCAST
select MODULES_USE_ELF_RELA
Index: tip/arch/s390/Kconfig
===================================================================
--- tip.orig/arch/s390/Kconfig
+++ tip/arch/s390/Kconfig
@@ -137,7 +137,6 @@ config S390
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16 if 32BIT
select HAVE_VIRT_CPU_ACCOUNTING
- select KTIME_SCALAR if 32BIT
select MODULES_USE_ELF_RELA
select NO_BOOTMEM
select OLD_SIGACTION
Index: tip/arch/x86/Kconfig
===================================================================
--- tip.orig/arch/x86/Kconfig
+++ tip/arch/x86/Kconfig
@@ -111,7 +111,6 @@ config X86
select ARCH_CLOCKSOURCE_DATA
select GENERIC_CLOCKEVENTS_BROADCAST if X86_64 || (X86_32 && X86_LOCAL_APIC)
select GENERIC_TIME_VSYSCALL
- select KTIME_SCALAR if X86_32
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select HAVE_CONTEXT_TRACKING if X86_64
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -27,43 +27,19 @@
/*
* ktime_t:
*
- * On 64-bit CPUs a single 64-bit variable is used to store the hrtimers
+ * A single 64-bit variable is used to store the hrtimers
* internal representation of time values in scalar nanoseconds. The
* design plays out best on 64-bit CPUs, where most conversions are
* NOPs and most arithmetic ktime_t operations are plain arithmetic
* operations.
*
- * On 32-bit CPUs an optimized representation of the timespec structure
- * is used to avoid expensive conversions from and to timespecs. The
- * endian-aware order of the tv struct members is chosen to allow
- * mathematical operations on the tv64 member of the union too, which
- * for certain operations produces better code.
- *
- * For architectures with efficient support for 64/32-bit conversions the
- * plain scalar nanosecond based representation can be selected by the
- * config switch CONFIG_KTIME_SCALAR.
*/
union ktime {
s64 tv64;
-#if BITS_PER_LONG != 64 && !defined(CONFIG_KTIME_SCALAR)
- struct {
-# ifdef __BIG_ENDIAN
- s32 sec, nsec;
-# else
- s32 nsec, sec;
-# endif
- } tv;
-#endif
};
typedef union ktime ktime_t; /* Kill this */
-/*
- * ktime_t definitions when using the 64-bit scalar representation:
- */
-
-#if (BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)
-
/**
* ktime_set - Set a ktime_t variable from a seconds/nanoseconds value
* @secs: seconds to set
@@ -123,153 +99,6 @@ static inline ktime_t timeval_to_ktime(s
/* Convert ktime_t to nanoseconds - NOP in the scalar storage format: */
#define ktime_to_ns(kt) ((kt).tv64)
-#else /* !((BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)) */
-
-/*
- * Helper macros/inlines to get the ktime_t math right in the timespec
- * representation. The macros are sometimes ugly - their actual use is
- * pretty okay-ish, given the circumstances. We do all this for
- * performance reasons. The pure scalar nsec_t based code was nice and
- * simple, but created too many 64-bit / 32-bit conversions and divisions.
- *
- * Be especially aware that negative values are represented in a way
- * that the tv.sec field is negative and the tv.nsec field is greater
- * or equal to zero but less than nanoseconds per second. This is the
- * same representation which is used by timespecs.
- *
- * tv.sec < 0 and 0 >= tv.nsec < NSEC_PER_SEC
- */
-
-/* Set a ktime_t variable to a value in sec/nsec representation: */
-static inline ktime_t ktime_set(const long secs, const unsigned long nsecs)
-{
- return (ktime_t) { .tv = { .sec = secs, .nsec = nsecs } };
-}
-
-/**
- * ktime_sub - subtract two ktime_t variables
- * @lhs: minuend
- * @rhs: subtrahend
- *
- * Return: The remainder of the subtraction.
- */
-static inline ktime_t ktime_sub(const ktime_t lhs, const ktime_t rhs)
-{
- ktime_t res;
-
- res.tv64 = lhs.tv64 - rhs.tv64;
- if (res.tv.nsec < 0)
- res.tv.nsec += NSEC_PER_SEC;
-
- return res;
-}
-
-/**
- * ktime_add - add two ktime_t variables
- * @add1: addend1
- * @add2: addend2
- *
- * Return: The sum of @add1 and @add2.
- */
-static inline ktime_t ktime_add(const ktime_t add1, const ktime_t add2)
-{
- ktime_t res;
-
- res.tv64 = add1.tv64 + add2.tv64;
- /*
- * performance trick: the (u32) -NSEC gives 0x00000000Fxxxxxxx
- * so we subtract NSEC_PER_SEC and add 1 to the upper 32 bit.
- *
- * it's equivalent to:
- * tv.nsec -= NSEC_PER_SEC
- * tv.sec ++;
- */
- if (res.tv.nsec >= NSEC_PER_SEC)
- res.tv64 += (u32)-NSEC_PER_SEC;
-
- return res;
-}
-
-/**
- * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable
- * @kt: addend
- * @nsec: the scalar nsec value to add
- *
- * Return: The sum of @kt and @nsec in ktime_t format.
- */
-extern ktime_t ktime_add_ns(const ktime_t kt, u64 nsec);
-
-/**
- * ktime_sub_ns - Subtract a scalar nanoseconds value from a ktime_t variable
- * @kt: minuend
- * @nsec: the scalar nsec value to subtract
- *
- * Return: The subtraction of @nsec from @kt in ktime_t format.
- */
-extern ktime_t ktime_sub_ns(const ktime_t kt, u64 nsec);
-
-/**
- * timespec_to_ktime - convert a timespec to ktime_t format
- * @ts: the timespec variable to convert
- *
- * Return: A ktime_t variable with the converted timespec value.
- */
-static inline ktime_t timespec_to_ktime(const struct timespec ts)
-{
- return (ktime_t) { .tv = { .sec = (s32)ts.tv_sec,
- .nsec = (s32)ts.tv_nsec } };
-}
-
-/**
- * timeval_to_ktime - convert a timeval to ktime_t format
- * @tv: the timeval variable to convert
- *
- * Return: A ktime_t variable with the converted timeval value.
- */
-static inline ktime_t timeval_to_ktime(const struct timeval tv)
-{
- return (ktime_t) { .tv = { .sec = (s32)tv.tv_sec,
- .nsec = (s32)(tv.tv_usec *
- NSEC_PER_USEC) } };
-}
-
-/**
- * ktime_to_timespec - convert a ktime_t variable to timespec format
- * @kt: the ktime_t variable to convert
- *
- * Return: The timespec representation of the ktime value.
- */
-static inline struct timespec ktime_to_timespec(const ktime_t kt)
-{
- return (struct timespec) { .tv_sec = (time_t) kt.tv.sec,
- .tv_nsec = (long) kt.tv.nsec };
-}
-
-/**
- * ktime_to_timeval - convert a ktime_t variable to timeval format
- * @kt: the ktime_t variable to convert
- *
- * Return: The timeval representation of the ktime value.
- */
-static inline struct timeval ktime_to_timeval(const ktime_t kt)
-{
- return (struct timeval) {
- .tv_sec = (time_t) kt.tv.sec,
- .tv_usec = (suseconds_t) (kt.tv.nsec / NSEC_PER_USEC) };
-}
-
-/**
- * ktime_to_ns - convert a ktime_t variable to scalar nanoseconds
- * @kt: the ktime_t variable to convert
- *
- * Return: The scalar nanoseconds representation of @kt.
- */
-static inline s64 ktime_to_ns(const ktime_t kt)
-{
- return (s64) kt.tv.sec * NSEC_PER_SEC + kt.tv.nsec;
-}
-
-#endif /* !((BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR)) */
/**
* ktime_equal - Compares two ktime_t variables to see if they are equal
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -19,6 +19,10 @@ extern struct timezone sys_tz;
#define TIME_T_MAX (time_t)((1UL << ((sizeof(time_t) << 3) - 1)) - 1)
+/* Located here for timespec_valid_strict */
+#define KTIME_MAX ((s64)~((u64)1 << 63))
+#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+
static inline int timespec_equal(const struct timespec *a,
const struct timespec *b)
{
@@ -84,13 +88,6 @@ static inline struct timespec timespec_s
return ts_delta;
}
-#define KTIME_MAX ((s64)~((u64)1 << 63))
-#if (BITS_PER_LONG == 64)
-# define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
-#else
-# define KTIME_SEC_MAX LONG_MAX
-#endif
-
/*
* Returns true if the timespec is norm, false if denorm:
*/
Index: tip/kernel/time/Kconfig
===================================================================
--- tip.orig/kernel/time/Kconfig
+++ tip/kernel/time/Kconfig
@@ -20,10 +20,6 @@ config GENERIC_TIME_VSYSCALL
config GENERIC_TIME_VSYSCALL_OLD
bool
-# ktime_t scalar 64bit nsec representation
-config KTIME_SCALAR
- bool
-
# Old style timekeeping
config ARCH_USES_GETTIMEOFFSET
bool
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -261,60 +261,6 @@ lock_hrtimer_base(const struct hrtimer *
* too large for inlining:
*/
#if BITS_PER_LONG < 64
-# ifndef CONFIG_KTIME_SCALAR
-/**
- * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable
- * @kt: addend
- * @nsec: the scalar nsec value to add
- *
- * Returns the sum of kt and nsec in ktime_t format
- */
-ktime_t ktime_add_ns(const ktime_t kt, u64 nsec)
-{
- ktime_t tmp;
-
- if (likely(nsec < NSEC_PER_SEC)) {
- tmp.tv64 = nsec;
- } else {
- unsigned long rem = do_div(nsec, NSEC_PER_SEC);
-
- /* Make sure nsec fits into long */
- if (unlikely(nsec > KTIME_SEC_MAX))
- return (ktime_t){ .tv64 = KTIME_MAX };
-
- tmp = ktime_set((long)nsec, rem);
- }
-
- return ktime_add(kt, tmp);
-}
-
-EXPORT_SYMBOL_GPL(ktime_add_ns);
-
-/**
- * ktime_sub_ns - Subtract a scalar nanoseconds value from a ktime_t variable
- * @kt: minuend
- * @nsec: the scalar nsec value to subtract
- *
- * Returns the subtraction of @nsec from @kt in ktime_t format
- */
-ktime_t ktime_sub_ns(const ktime_t kt, u64 nsec)
-{
- ktime_t tmp;
-
- if (likely(nsec < NSEC_PER_SEC)) {
- tmp.tv64 = nsec;
- } else {
- unsigned long rem = do_div(nsec, NSEC_PER_SEC);
-
- tmp = ktime_set((long)nsec, rem);
- }
-
- return ktime_sub(kt, tmp);
-}
-
-EXPORT_SYMBOL_GPL(ktime_sub_ns);
-# endif /* !CONFIG_KTIME_SCALAR */
-
/*
* Divide a ktime value by a nanosecond value
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -344,11 +344,8 @@ ktime_t ktime_get(void)
nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- /*
- * Use ktime_set/ktime_add_ns to create a proper ktime on
- * 32-bit architectures without CONFIG_KTIME_SCALAR.
- */
- return ktime_add_ns(ktime_set(secs, 0), nsecs);
+
+ return ktime_set(secs, nsecs);
}
EXPORT_SYMBOL_GPL(ktime_get);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 05/64] ktime: Sanitize ktime_to_us/ms conversion
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (3 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 04/64] ktime: Kill non-scalar ktime_t implementation for 2038 Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 06/64] ktime: Change ktime_set() to take 64bit seconds value Thomas Gleixner
` (59 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: ktime-sanitize-conversion.patch --]
[-- Type: text/plain, Size: 2232 bytes --]
With the plain nanoseconds based ktime_t we can simply use
ktime_divns() instead of going through loops and hoops of
timespec/timeval conversion.
Reported-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 6 ------
include/linux/ktime.h | 12 ++++++++----
kernel/time/hrtimer.c | 1 +
3 files changed, 9 insertions(+), 10 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -457,12 +457,6 @@ extern void hrtimer_run_pending(void);
/* Bootup initialization: */
extern void __init hrtimers_init(void);
-#if BITS_PER_LONG < 64
-extern u64 ktime_divns(const ktime_t kt, s64 div);
-#else /* BITS_PER_LONG < 64 */
-# define ktime_divns(kt, div) (u64)((kt).tv64 / (div))
-#endif
-
/* Show pending timers: */
extern void sysrq_timer_list_show(void);
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -157,16 +157,20 @@ static inline bool ktime_before(const kt
return ktime_compare(cmp1, cmp2) < 0;
}
+#if BITS_PER_LONG < 64
+extern u64 ktime_divns(const ktime_t kt, s64 div);
+#else /* BITS_PER_LONG < 64 */
+# define ktime_divns(kt, div) (u64)((kt).tv64 / (div))
+#endif
+
static inline s64 ktime_to_us(const ktime_t kt)
{
- struct timeval tv = ktime_to_timeval(kt);
- return (s64) tv.tv_sec * USEC_PER_SEC + tv.tv_usec;
+ return ktime_divns(kt, NSEC_PER_USEC);
}
static inline s64 ktime_to_ms(const ktime_t kt)
{
- struct timeval tv = ktime_to_timeval(kt);
- return (s64) tv.tv_sec * MSEC_PER_SEC + tv.tv_usec / USEC_PER_MSEC;
+ return ktime_divns(kt, NSEC_PER_MSEC);
}
static inline s64 ktime_us_delta(const ktime_t later, const ktime_t earlier)
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -280,6 +280,7 @@ u64 ktime_divns(const ktime_t kt, s64 di
return dclc;
}
+EXPORT_SYMBOL_GPL(ktime_divns);
#endif /* BITS_PER_LONG >= 64 */
/*
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 06/64] ktime: Change ktime_set() to take 64bit seconds value
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (4 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 05/64] ktime: Sanitize ktime_to_us/ms conversion Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 07/64] time64: Add time64.h header and define struct timespec64 Thomas Gleixner
` (58 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-696511b.patch --]
[-- Type: text/plain, Size: 1148 bytes --]
From: John Stultz <john.stultz@linaro.org>
In order to support dates past 2038 on 32bit systems, ktime_set()
needs to handle 64bit second values.
[ tglx: Removed the BITS_PER_LONG check ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/ktime.h | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -47,13 +47,12 @@ typedef union ktime ktime_t; /* Kill th
*
* Return: The ktime_t representation of the value.
*/
-static inline ktime_t ktime_set(const long secs, const unsigned long nsecs)
+static inline ktime_t ktime_set(const s64 secs, const unsigned long nsecs)
{
-#if (BITS_PER_LONG == 64)
if (unlikely(secs >= KTIME_SEC_MAX))
return (ktime_t){ .tv64 = KTIME_MAX };
-#endif
- return (ktime_t) { .tv64 = (s64)secs * NSEC_PER_SEC + (s64)nsecs };
+
+ return (ktime_t) { .tv64 = secs * NSEC_PER_SEC + (s64)nsecs };
}
/* Subtract two ktime_t variables. rem = lhs -rhs: */
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 07/64] time64: Add time64.h header and define struct timespec64
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (5 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 06/64] ktime: Change ktime_set() to take 64bit seconds value Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:03 ` [patch V2 08/64] time: More core infrastructure for timespec64 Thomas Gleixner
` (57 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-d8e0c56.patch --]
[-- Type: text/plain, Size: 6139 bytes --]
From: John Stultz <john.stultz@linaro.org>
Define the timespec64 structure and standard helper functions.
[ tglx: Make it 32bit only. 64bit really can map timespec to timespec64 ]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/time.h | 15 ----
include/linux/time64.h | 163 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 164 insertions(+), 14 deletions(-)
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -4,25 +4,12 @@
# include <linux/cache.h>
# include <linux/seqlock.h>
# include <linux/math64.h>
-#include <uapi/linux/time.h>
+# include <linux/time64.h>
extern struct timezone sys_tz;
-/* Parameters used to convert the timespec values: */
-#define MSEC_PER_SEC 1000L
-#define USEC_PER_MSEC 1000L
-#define NSEC_PER_USEC 1000L
-#define NSEC_PER_MSEC 1000000L
-#define USEC_PER_SEC 1000000L
-#define NSEC_PER_SEC 1000000000L
-#define FSEC_PER_SEC 1000000000000000LL
-
#define TIME_T_MAX (time_t)((1UL << ((sizeof(time_t) << 3) - 1)) - 1)
-/* Located here for timespec_valid_strict */
-#define KTIME_MAX ((s64)~((u64)1 << 63))
-#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
-
static inline int timespec_equal(const struct timespec *a,
const struct timespec *b)
{
Index: tip/include/linux/time64.h
===================================================================
--- /dev/null
+++ tip/include/linux/time64.h
@@ -0,0 +1,163 @@
+#ifndef _LINUX_TIME64_H
+#define _LINUX_TIME64_H
+
+#include <uapi/linux/time.h>
+
+typedef __s64 time64_t;
+
+/*
+ * This wants to go into uapi/linux/time.h once we agreed about the
+ * userspace interfaces.
+ */
+#if __BITS_PER_LONG == 64
+# define timespec64 timespec
+#else
+struct timespec64 {
+ time64_t tv_sec; /* seconds */
+ long tv_nsec; /* nanoseconds */
+};
+#endif
+
+/* Parameters used to convert the timespec values: */
+#define MSEC_PER_SEC 1000L
+#define USEC_PER_MSEC 1000L
+#define NSEC_PER_USEC 1000L
+#define NSEC_PER_MSEC 1000000L
+#define USEC_PER_SEC 1000000L
+#define NSEC_PER_SEC 1000000000L
+#define FSEC_PER_SEC 1000000000000000LL
+
+/* Located here for timespec[64]_valid_strict */
+#define KTIME_MAX ((s64)~((u64)1 << 63))
+#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+
+#if __BITS_PER_LONG == 64
+
+# define timespec64_equal timespec_equal
+# define timespec64_compare timespec_compare
+# define set_normalized_timespec64 set_normalized_timespec
+# define timespec64_add_safe timespec_add_safe
+# define timespec64_add timespec_add
+# define timespec64_sub timespec_sub
+# define timespec64_valid timespec_valid
+# define timespec64_valid_strict timespec_valid_strict
+# define timespec64_to_ns timespec_to_ns
+# define ns_to_timespec64 ns_to_timespec
+# define timespec64_add_ns timespec_add_ns
+
+#else
+
+static inline int timespec64_equal(const struct timespec64 *a,
+ const struct timespec64 *b)
+{
+ return (a->tv_sec == b->tv_sec) && (a->tv_nsec == b->tv_nsec);
+}
+
+/*
+ * lhs < rhs: return <0
+ * lhs == rhs: return 0
+ * lhs > rhs: return >0
+ */
+static inline int timespec64_compare(const struct timespec64 *lhs, const struct timespec64 *rhs)
+{
+ if (lhs->tv_sec < rhs->tv_sec)
+ return -1;
+ if (lhs->tv_sec > rhs->tv_sec)
+ return 1;
+ return lhs->tv_nsec - rhs->tv_nsec;
+}
+
+extern void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec);
+
+/*
+ * timespec64_add_safe assumes both values are positive and checks for
+ * overflow. It will return TIME_T_MAX if the returned value would be
+ * smaller then either of the arguments.
+ */
+extern struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
+ const struct timespec64 rhs);
+
+
+static inline struct timespec64 timespec64_add(struct timespec64 lhs,
+ struct timespec64 rhs)
+{
+ struct timespec64 ts_delta;
+ set_normalized_timespec64(&ts_delta, lhs.tv_sec + rhs.tv_sec,
+ lhs.tv_nsec + rhs.tv_nsec);
+ return ts_delta;
+}
+
+/*
+ * sub = lhs - rhs, in normalized form
+ */
+static inline struct timespec64 timespec64_sub(struct timespec64 lhs,
+ struct timespec64 rhs)
+{
+ struct timespec64 ts_delta;
+ set_normalized_timespec64(&ts_delta, lhs.tv_sec - rhs.tv_sec,
+ lhs.tv_nsec - rhs.tv_nsec);
+ return ts_delta;
+}
+
+/*
+ * Returns true if the timespec64 is norm, false if denorm:
+ */
+static inline bool timespec64_valid(const struct timespec64 *ts)
+{
+ /* Dates before 1970 are bogus */
+ if (ts->tv_sec < 0)
+ return false;
+ /* Can't have more nanoseconds then a second */
+ if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
+ return false;
+ return true;
+}
+
+static inline bool timespec64_valid_strict(const struct timespec64 *ts)
+{
+ if (!timespec64_valid(ts))
+ return false;
+ /* Disallow values that could overflow ktime_t */
+ if ((unsigned long long)ts->tv_sec >= KTIME_SEC_MAX)
+ return false;
+ return true;
+}
+
+/**
+ * timespec64_to_ns - Convert timespec64 to nanoseconds
+ * @ts: pointer to the timespec64 variable to be converted
+ *
+ * Returns the scalar nanosecond representation of the timespec64
+ * parameter.
+ */
+static inline s64 timespec64_to_ns(const struct timespec64 *ts)
+{
+ return ((s64) ts->tv_sec * NSEC_PER_SEC) + ts->tv_nsec;
+}
+
+/**
+ * ns_to_timespec64 - Convert nanoseconds to timespec64
+ * @nsec: the nanoseconds value to be converted
+ *
+ * Returns the timespec64 representation of the nsec parameter.
+ */
+extern struct timespec64 ns_to_timespec64(const s64 nsec);
+
+/**
+ * timespec64_add_ns - Adds nanoseconds to a timespec64
+ * @a: pointer to timespec64 to be incremented
+ * @ns: unsigned nanoseconds value to be added
+ *
+ * This must always be inlined because its used from the x86-64 vdso,
+ * which cannot call other kernel functions.
+ */
+static __always_inline void timespec64_add_ns(struct timespec64 *a, u64 ns)
+{
+ a->tv_sec += __iter_div_u64_rem(a->tv_nsec + ns, NSEC_PER_SEC, &ns);
+ a->tv_nsec = ns;
+}
+
+#endif
+
+#endif /* _LINUX_TIME64_H */
+
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 08/64] time: More core infrastructure for timespec64
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (6 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 07/64] time64: Add time64.h header and define struct timespec64 Thomas Gleixner
@ 2014-07-16 21:03 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 09/64] timekeeping: Convert timekeeping core to use timespec64s Thomas Gleixner
` (56 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:03 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-99c585d.patch --]
[-- Type: text/plain, Size: 5114 bytes --]
From: John Stultz <john.stultz@linaro.org>
Helper and conversion functions for timespec64.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/ktime.h | 28 ++++++++++++++++++++++
include/linux/time64.h | 28 ++++++++++++++++++++++
kernel/time/time.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 118 insertions(+)
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -83,6 +83,12 @@ static inline ktime_t timespec_to_ktime(
return ktime_set(ts.tv_sec, ts.tv_nsec);
}
+/* convert a timespec64 to ktime_t format: */
+static inline ktime_t timespec64_to_ktime(struct timespec64 ts)
+{
+ return ktime_set(ts.tv_sec, ts.tv_nsec);
+}
+
/* convert a timeval to ktime_t format: */
static inline ktime_t timeval_to_ktime(struct timeval tv)
{
@@ -92,6 +98,9 @@ static inline ktime_t timeval_to_ktime(s
/* Map the ktime_t to timespec conversion to ns_to_timespec function */
#define ktime_to_timespec(kt) ns_to_timespec((kt).tv64)
+/* Map the ktime_t to timespec conversion to ns_to_timespec function */
+#define ktime_to_timespec64(kt) ns_to_timespec64((kt).tv64)
+
/* Map the ktime_t to timeval conversion to ns_to_timeval function */
#define ktime_to_timeval(kt) ns_to_timeval((kt).tv64)
@@ -186,6 +195,25 @@ static inline __must_check bool ktime_to
return true;
} else {
return false;
+ }
+}
+
+/**
+ * ktime_to_timespec64_cond - convert a ktime_t variable to timespec64
+ * format only if the variable contains data
+ * @kt: the ktime_t variable to convert
+ * @ts: the timespec variable to store the result in
+ *
+ * Return: %true if there was a successful conversion, %false if kt was 0.
+ */
+static inline __must_check bool ktime_to_timespec64_cond(const ktime_t kt,
+ struct timespec64 *ts)
+{
+ if (kt.tv64) {
+ *ts = ktime_to_timespec64(kt);
+ return true;
+ } else {
+ return false;
}
}
Index: tip/include/linux/time64.h
===================================================================
--- tip.orig/include/linux/time64.h
+++ tip/include/linux/time64.h
@@ -33,6 +33,16 @@ struct timespec64 {
#if __BITS_PER_LONG == 64
+static inline struct timespec timespec64_to_timespec(const struct timespec64 ts64)
+{
+ return ts64;
+}
+
+static inline struct timespec64 timespec_to_timespec64(const struct timespec ts)
+{
+ return ts;
+}
+
# define timespec64_equal timespec_equal
# define timespec64_compare timespec_compare
# define set_normalized_timespec64 set_normalized_timespec
@@ -47,6 +57,24 @@ struct timespec64 {
#else
+static inline struct timespec timespec64_to_timespec(const struct timespec64 ts64)
+{
+ struct timespec ret;
+
+ ret.tv_sec = (time_t)ts64.tv_sec;
+ ret.tv_nsec = ts64.tv_nsec;
+ return ret;
+}
+
+static inline struct timespec64 timespec_to_timespec64(const struct timespec ts)
+{
+ struct timespec64 ret;
+
+ ret.tv_sec = ts.tv_sec;
+ ret.tv_nsec = ts.tv_nsec;
+ return ret;
+}
+
static inline int timespec64_equal(const struct timespec64 *a,
const struct timespec64 *b)
{
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -420,6 +420,68 @@ struct timeval ns_to_timeval(const s64 n
}
EXPORT_SYMBOL(ns_to_timeval);
+#if BITS_PER_LONG == 32
+/**
+ * set_normalized_timespec - set timespec sec and nsec parts and normalize
+ *
+ * @ts: pointer to timespec variable to be set
+ * @sec: seconds to set
+ * @nsec: nanoseconds to set
+ *
+ * Set seconds and nanoseconds field of a timespec variable and
+ * normalize to the timespec storage format
+ *
+ * Note: The tv_nsec part is always in the range of
+ * 0 <= tv_nsec < NSEC_PER_SEC
+ * For negative values only the tv_sec field is negative !
+ */
+void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec)
+{
+ while (nsec >= NSEC_PER_SEC) {
+ /*
+ * The following asm() prevents the compiler from
+ * optimising this loop into a modulo operation. See
+ * also __iter_div_u64_rem() in include/linux/time.h
+ */
+ asm("" : "+rm"(nsec));
+ nsec -= NSEC_PER_SEC;
+ ++sec;
+ }
+ while (nsec < 0) {
+ asm("" : "+rm"(nsec));
+ nsec += NSEC_PER_SEC;
+ --sec;
+ }
+ ts->tv_sec = sec;
+ ts->tv_nsec = nsec;
+}
+EXPORT_SYMBOL(set_normalized_timespec64);
+
+/**
+ * ns_to_timespec64 - Convert nanoseconds to timespec64
+ * @nsec: the nanoseconds value to be converted
+ *
+ * Returns the timespec64 representation of the nsec parameter.
+ */
+struct timespec64 ns_to_timespec64(const s64 nsec)
+{
+ struct timespec64 ts;
+ s32 rem;
+
+ if (!nsec)
+ return (struct timespec64) {0, 0};
+
+ ts.tv_sec = div_s64_rem(nsec, NSEC_PER_SEC, &rem);
+ if (unlikely(rem < 0)) {
+ ts.tv_sec--;
+ rem += NSEC_PER_SEC;
+ }
+ ts.tv_nsec = rem;
+
+ return ts;
+}
+EXPORT_SYMBOL(ns_to_timespec64);
+#endif
/*
* When we convert to jiffies then we interpret incoming values
* the following way:
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 09/64] timekeeping: Convert timekeeping core to use timespec64s
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (7 preceding siblings ...)
2014-07-16 21:03 ` [patch V2 08/64] time: More core infrastructure for timespec64 Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 10/64] time: Consolidate the time accessor prototypes Thomas Gleixner
` (55 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: jstultz-tk-480e2d6.patch --]
[-- Type: text/plain, Size: 21065 bytes --]
From: John Stultz <john.stultz@linaro.org>
Convert the core timekeeping logic to use timespec64s. This moves the
2038 issues out of the core logic and into all of the accessor
functions.
Future changes will need to push the timespec64s out to all
timekeeping users, but that can be done interface by interface.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 10 +-
kernel/time/ntp.c | 8 -
kernel/time/ntp_internal.h | 2
kernel/time/timekeeping.c | 172 ++++++++++++++++++++----------------
kernel/time/timekeeping_debug.c | 2
kernel/time/timekeeping_internal.h | 2
6 files changed, 109 insertions(+), 87 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -55,15 +55,15 @@ struct timekeeper {
* - wall_to_monotonic is no longer the boot time, getboottime must be
* used instead.
*/
- struct timespec wall_to_monotonic;
+ struct timespec64 wall_to_monotonic;
/* Offset clock monotonic -> clock realtime */
ktime_t offs_real;
/* time spent in suspend */
- struct timespec total_sleep_time;
+ struct timespec64 total_sleep_time;
/* Offset clock monotonic -> clock boottime */
ktime_t offs_boot;
/* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
- struct timespec raw_time;
+ struct timespec64 raw_time;
/* The current UTC to TAI offset in seconds */
s32 tai_offset;
/* Offset clock monotonic -> clock tai */
@@ -71,9 +71,9 @@ struct timekeeper {
};
-static inline struct timespec tk_xtime(struct timekeeper *tk)
+static inline struct timespec64 tk_xtime(struct timekeeper *tk)
{
- struct timespec ts;
+ struct timespec64 ts;
ts.tv_sec = tk->xtime_sec;
ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
Index: tip/kernel/time/ntp.c
===================================================================
--- tip.orig/kernel/time/ntp.c
+++ tip/kernel/time/ntp.c
@@ -531,7 +531,7 @@ void ntp_notify_cmos_timer(void) { }
/*
* Propagate a new txc->status value into the NTP state:
*/
-static inline void process_adj_status(struct timex *txc, struct timespec *ts)
+static inline void process_adj_status(struct timex *txc, struct timespec64 *ts)
{
if ((time_status & STA_PLL) && !(txc->status & STA_PLL)) {
time_state = TIME_OK;
@@ -554,7 +554,7 @@ static inline void process_adj_status(st
static inline void process_adjtimex_modes(struct timex *txc,
- struct timespec *ts,
+ struct timespec64 *ts,
s32 *time_tai)
{
if (txc->modes & ADJ_STATUS)
@@ -640,7 +640,7 @@ int ntp_validate_timex(struct timex *txc
* adjtimex mainly allows reading (and writing, if superuser) of
* kernel time-keeping variables. used by xntpd.
*/
-int __do_adjtimex(struct timex *txc, struct timespec *ts, s32 *time_tai)
+int __do_adjtimex(struct timex *txc, struct timespec64 *ts, s32 *time_tai)
{
int result;
@@ -684,7 +684,7 @@ int __do_adjtimex(struct timex *txc, str
/* fill PPS status fields */
pps_fill_timex(txc);
- txc->time.tv_sec = ts->tv_sec;
+ txc->time.tv_sec = (time_t)ts->tv_sec;
txc->time.tv_usec = ts->tv_nsec;
if (!(time_status & STA_NANO))
txc->time.tv_usec /= NSEC_PER_USEC;
Index: tip/kernel/time/ntp_internal.h
===================================================================
--- tip.orig/kernel/time/ntp_internal.h
+++ tip/kernel/time/ntp_internal.h
@@ -7,6 +7,6 @@ extern void ntp_clear(void);
extern u64 ntp_tick_length(void);
extern int second_overflow(unsigned long secs);
extern int ntp_validate_timex(struct timex *);
-extern int __do_adjtimex(struct timex *, struct timespec *, s32 *);
+extern int __do_adjtimex(struct timex *, struct timespec64 *, s32 *);
extern void __hardpps(const struct timespec *, const struct timespec *);
#endif /* _LINUX_NTP_INTERNAL_H */
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -51,43 +51,43 @@ static inline void tk_normalize_xtime(st
}
}
-static void tk_set_xtime(struct timekeeper *tk, const struct timespec *ts)
+static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
tk->xtime_nsec = (u64)ts->tv_nsec << tk->shift;
}
-static void tk_xtime_add(struct timekeeper *tk, const struct timespec *ts)
+static void tk_xtime_add(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec += ts->tv_sec;
tk->xtime_nsec += (u64)ts->tv_nsec << tk->shift;
tk_normalize_xtime(tk);
}
-static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec wtm)
+static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec64 wtm)
{
- struct timespec tmp;
+ struct timespec64 tmp;
/*
* Verify consistency of: offset_real = -wall_to_monotonic
* before modifying anything
*/
- set_normalized_timespec(&tmp, -tk->wall_to_monotonic.tv_sec,
+ set_normalized_timespec64(&tmp, -tk->wall_to_monotonic.tv_sec,
-tk->wall_to_monotonic.tv_nsec);
- WARN_ON_ONCE(tk->offs_real.tv64 != timespec_to_ktime(tmp).tv64);
+ WARN_ON_ONCE(tk->offs_real.tv64 != timespec64_to_ktime(tmp).tv64);
tk->wall_to_monotonic = wtm;
- set_normalized_timespec(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
- tk->offs_real = timespec_to_ktime(tmp);
+ set_normalized_timespec64(&tmp, -wtm.tv_sec, -wtm.tv_nsec);
+ tk->offs_real = timespec64_to_ktime(tmp);
tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tk->tai_offset, 0));
}
-static void tk_set_sleep_time(struct timekeeper *tk, struct timespec t)
+static void tk_set_sleep_time(struct timekeeper *tk, struct timespec64 t)
{
/* Verify consistency before modifying */
- WARN_ON_ONCE(tk->offs_boot.tv64 != timespec_to_ktime(tk->total_sleep_time).tv64);
+ WARN_ON_ONCE(tk->offs_boot.tv64 != timespec64_to_ktime(tk->total_sleep_time).tv64);
tk->total_sleep_time = t;
- tk->offs_boot = timespec_to_ktime(t);
+ tk->offs_boot = timespec64_to_ktime(t);
}
/**
@@ -281,7 +281,7 @@ static void timekeeping_forward_now(stru
tk_normalize_xtime(tk);
nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
- timespec_add_ns(&tk->raw_time, nsec);
+ timespec64_add_ns(&tk->raw_time, nsec);
}
/**
@@ -360,7 +360,7 @@ EXPORT_SYMBOL_GPL(ktime_get);
void ktime_get_ts(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec tomono;
+ struct timespec64 ts64, tomono;
s64 nsec;
unsigned int seq;
@@ -368,15 +368,16 @@ void ktime_get_ts(struct timespec *ts)
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec;
+ ts64.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsec + tomono.tv_nsec);
+ ts64.tv_sec += tomono.tv_sec;
+ ts64.tv_nsec = 0;
+ timespec64_add_ns(&ts64, nsec + tomono.tv_nsec);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL_GPL(ktime_get_ts);
@@ -390,6 +391,7 @@ EXPORT_SYMBOL_GPL(ktime_get_ts);
void timekeeping_clocktai(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
unsigned long seq;
u64 nsecs;
@@ -398,13 +400,14 @@ void timekeeping_clocktai(struct timespe
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec + tk->tai_offset;
+ ts64.tv_sec = tk->xtime_sec + tk->tai_offset;
nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
+ ts64.tv_nsec = 0;
+ timespec64_add_ns(&ts64, nsecs);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL(timekeeping_clocktai);
@@ -446,7 +449,7 @@ void getnstime_raw_and_real(struct times
do {
seq = read_seqcount_begin(&timekeeper_seq);
- *ts_raw = tk->raw_time;
+ *ts_raw = timespec64_to_timespec(tk->raw_time);
ts_real->tv_sec = tk->xtime_sec;
ts_real->tv_nsec = 0;
@@ -487,7 +490,7 @@ EXPORT_SYMBOL(do_gettimeofday);
int do_settimeofday(const struct timespec *tv)
{
struct timekeeper *tk = &timekeeper;
- struct timespec ts_delta, xt;
+ struct timespec64 ts_delta, xt, tmp;
unsigned long flags;
if (!timespec_valid_strict(tv))
@@ -502,9 +505,10 @@ int do_settimeofday(const struct timespe
ts_delta.tv_sec = tv->tv_sec - xt.tv_sec;
ts_delta.tv_nsec = tv->tv_nsec - xt.tv_nsec;
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, ts_delta));
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts_delta));
- tk_set_xtime(tk, tv);
+ tmp = timespec_to_timespec64(*tv);
+ tk_set_xtime(tk, &tmp);
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -528,26 +532,28 @@ int timekeeping_inject_offset(struct tim
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec tmp;
+ struct timespec64 ts64, tmp;
int ret = 0;
if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
return -EINVAL;
+ ts64 = timespec_to_timespec64(*ts);
+
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
timekeeping_forward_now(tk);
/* Make sure the proposed value is valid */
- tmp = timespec_add(tk_xtime(tk), *ts);
- if (!timespec_valid_strict(&tmp)) {
+ tmp = timespec64_add(tk_xtime(tk), ts64);
+ if (!timespec64_valid_strict(&tmp)) {
ret = -EINVAL;
goto error;
}
- tk_xtime_add(tk, ts);
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *ts));
+ tk_xtime_add(tk, &ts64);
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts64));
error: /* even if we error out, we forwarded the time, so call update */
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -691,17 +697,19 @@ EXPORT_SYMBOL_GPL(ktime_get_real);
void getrawmonotonic(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
unsigned long seq;
s64 nsecs;
do {
seq = read_seqcount_begin(&timekeeper_seq);
nsecs = timekeeping_get_ns_raw(tk);
- *ts = tk->raw_time;
+ ts64 = tk->raw_time;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- timespec_add_ns(ts, nsecs);
+ timespec64_add_ns(&ts64, nsecs);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL(getrawmonotonic);
@@ -781,11 +789,12 @@ void __init timekeeping_init(void)
struct timekeeper *tk = &timekeeper;
struct clocksource *clock;
unsigned long flags;
- struct timespec now, boot, tmp;
-
- read_persistent_clock(&now);
+ struct timespec64 now, boot, tmp;
+ struct timespec ts;
- if (!timespec_valid_strict(&now)) {
+ read_persistent_clock(&ts);
+ now = timespec_to_timespec64(ts);
+ if (!timespec64_valid_strict(&now)) {
pr_warn("WARNING: Persistent clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
now.tv_sec = 0;
@@ -793,8 +802,9 @@ void __init timekeeping_init(void)
} else if (now.tv_sec || now.tv_nsec)
persistent_clock_exist = true;
- read_boot_clock(&boot);
- if (!timespec_valid_strict(&boot)) {
+ read_boot_clock(&ts);
+ boot = timespec_to_timespec64(ts);
+ if (!timespec64_valid_strict(&boot)) {
pr_warn("WARNING: Boot clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
boot.tv_sec = 0;
@@ -816,7 +826,7 @@ void __init timekeeping_init(void)
if (boot.tv_sec == 0 && boot.tv_nsec == 0)
boot = tk_xtime(tk);
- set_normalized_timespec(&tmp, -boot.tv_sec, -boot.tv_nsec);
+ set_normalized_timespec64(&tmp, -boot.tv_sec, -boot.tv_nsec);
tk_set_wall_to_mono(tk, tmp);
tmp.tv_sec = 0;
@@ -830,7 +840,7 @@ void __init timekeeping_init(void)
}
/* time in seconds when suspend began */
-static struct timespec timekeeping_suspend_time;
+static struct timespec64 timekeeping_suspend_time;
/**
* __timekeeping_inject_sleeptime - Internal function to add sleep interval
@@ -840,17 +850,17 @@ static struct timespec timekeeping_suspe
* adds the sleep offset to the timekeeping variables.
*/
static void __timekeeping_inject_sleeptime(struct timekeeper *tk,
- struct timespec *delta)
+ struct timespec64 *delta)
{
- if (!timespec_valid_strict(delta)) {
+ if (!timespec64_valid_strict(delta)) {
printk_deferred(KERN_WARNING
"__timekeeping_inject_sleeptime: Invalid "
"sleep delta value!\n");
return;
}
tk_xtime_add(tk, delta);
- tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *delta));
- tk_set_sleep_time(tk, timespec_add(tk->total_sleep_time, *delta));
+ tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *delta));
+ tk_set_sleep_time(tk, timespec64_add(tk->total_sleep_time, *delta));
tk_debug_account_sleep_time(delta);
}
@@ -867,6 +877,7 @@ static void __timekeeping_inject_sleepti
void timekeeping_inject_sleeptime(struct timespec *delta)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 tmp;
unsigned long flags;
/*
@@ -881,7 +892,8 @@ void timekeeping_inject_sleeptime(struct
timekeeping_forward_now(tk);
- __timekeeping_inject_sleeptime(tk, delta);
+ tmp = timespec_to_timespec64(*delta);
+ __timekeeping_inject_sleeptime(tk, &tmp);
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -904,11 +916,13 @@ static void timekeeping_resume(void)
struct timekeeper *tk = &timekeeper;
struct clocksource *clock = tk->clock;
unsigned long flags;
- struct timespec ts_new, ts_delta;
+ struct timespec64 ts_new, ts_delta;
+ struct timespec tmp;
cycle_t cycle_now, cycle_delta;
bool suspendtime_found = false;
- read_persistent_clock(&ts_new);
+ read_persistent_clock(&tmp);
+ ts_new = timespec_to_timespec64(tmp);
clockevents_resume();
clocksource_resume();
@@ -951,10 +965,10 @@ static void timekeeping_resume(void)
}
nsec += ((u64) cycle_delta * mult) >> shift;
- ts_delta = ns_to_timespec(nsec);
+ ts_delta = ns_to_timespec64(nsec);
suspendtime_found = true;
- } else if (timespec_compare(&ts_new, &timekeeping_suspend_time) > 0) {
- ts_delta = timespec_sub(ts_new, timekeeping_suspend_time);
+ } else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) > 0) {
+ ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
suspendtime_found = true;
}
@@ -981,10 +995,12 @@ static int timekeeping_suspend(void)
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec delta, delta_delta;
- static struct timespec old_delta;
+ struct timespec64 delta, delta_delta;
+ static struct timespec64 old_delta;
+ struct timespec tmp;
- read_persistent_clock(&timekeeping_suspend_time);
+ read_persistent_clock(&tmp);
+ timekeeping_suspend_time = timespec_to_timespec64(tmp);
/*
* On some systems the persistent_clock can not be detected at
@@ -1005,8 +1021,8 @@ static int timekeeping_suspend(void)
* try to compensate so the difference in system time
* and persistent_clock time stays close to constant.
*/
- delta = timespec_sub(tk_xtime(tk), timekeeping_suspend_time);
- delta_delta = timespec_sub(delta, old_delta);
+ delta = timespec64_sub(tk_xtime(tk), timekeeping_suspend_time);
+ delta_delta = timespec64_sub(delta, old_delta);
if (abs(delta_delta.tv_sec) >= 2) {
/*
* if delta_delta is too large, assume time correction
@@ -1016,7 +1032,7 @@ static int timekeeping_suspend(void)
} else {
/* Otherwise try to adjust old_system to compensate */
timekeeping_suspend_time =
- timespec_add(timekeeping_suspend_time, delta_delta);
+ timespec64_add(timekeeping_suspend_time, delta_delta);
}
timekeeping_update(tk, TK_MIRROR);
@@ -1253,14 +1269,14 @@ static inline unsigned int accumulate_ns
/* Figure out if its a leap sec and apply if needed */
leap = second_overflow(tk->xtime_sec);
if (unlikely(leap)) {
- struct timespec ts;
+ struct timespec64 ts;
tk->xtime_sec += leap;
ts.tv_sec = leap;
ts.tv_nsec = 0;
tk_set_wall_to_mono(tk,
- timespec_sub(tk->wall_to_monotonic, ts));
+ timespec64_sub(tk->wall_to_monotonic, ts));
__timekeeping_set_tai_offset(tk, tk->tai_offset - leap);
@@ -1469,7 +1485,7 @@ EXPORT_SYMBOL_GPL(getboottime);
void get_monotonic_boottime(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec tomono, sleep;
+ struct timespec64 tomono, sleep, ret;
s64 nsec;
unsigned int seq;
@@ -1477,16 +1493,17 @@ void get_monotonic_boottime(struct times
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts->tv_sec = tk->xtime_sec;
+ ret.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
sleep = tk->total_sleep_time;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts->tv_sec += tomono.tv_sec + sleep.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsec + tomono.tv_nsec + sleep.tv_nsec);
+ ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
+ ret.tv_nsec = 0;
+ timespec64_add_ns(&ret, nsec + tomono.tv_nsec + sleep.tv_nsec);
+ *ts = timespec64_to_timespec(ret);
}
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
@@ -1514,8 +1531,11 @@ EXPORT_SYMBOL_GPL(ktime_get_boottime);
void monotonic_to_bootbased(struct timespec *ts)
{
struct timekeeper *tk = &timekeeper;
+ struct timespec64 ts64;
- *ts = timespec_add(*ts, tk->total_sleep_time);
+ ts64 = timespec_to_timespec64(*ts);
+ ts64 = timespec64_add(ts64, tk->total_sleep_time);
+ *ts = timespec64_to_timespec(ts64);
}
EXPORT_SYMBOL_GPL(monotonic_to_bootbased);
@@ -1531,13 +1551,13 @@ struct timespec __current_kernel_time(vo
{
struct timekeeper *tk = &timekeeper;
- return tk_xtime(tk);
+ return timespec64_to_timespec(tk_xtime(tk));
}
struct timespec current_kernel_time(void)
{
struct timekeeper *tk = &timekeeper;
- struct timespec now;
+ struct timespec64 now;
unsigned long seq;
do {
@@ -1546,14 +1566,14 @@ struct timespec current_kernel_time(void
now = tk_xtime(tk);
} while (read_seqcount_retry(&timekeeper_seq, seq));
- return now;
+ return timespec64_to_timespec(now);
}
EXPORT_SYMBOL(current_kernel_time);
struct timespec get_monotonic_coarse(void)
{
struct timekeeper *tk = &timekeeper;
- struct timespec now, mono;
+ struct timespec64 now, mono;
unsigned long seq;
do {
@@ -1563,9 +1583,10 @@ struct timespec get_monotonic_coarse(voi
mono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- set_normalized_timespec(&now, now.tv_sec + mono.tv_sec,
+ set_normalized_timespec64(&now, now.tv_sec + mono.tv_sec,
now.tv_nsec + mono.tv_nsec);
- return now;
+
+ return timespec64_to_timespec(now);
}
/*
@@ -1589,7 +1610,7 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t *offs_tai)
{
struct timekeeper *tk = &timekeeper;
- struct timespec ts;
+ struct timespec64 ts;
ktime_t now;
unsigned int seq;
@@ -1597,7 +1618,6 @@ ktime_t ktime_get_update_offsets_tick(kt
seq = read_seqcount_begin(&timekeeper_seq);
ts = tk_xtime(tk);
-
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
@@ -1650,14 +1670,14 @@ ktime_t ktime_get_monotonic_offset(void)
{
struct timekeeper *tk = &timekeeper;
unsigned long seq;
- struct timespec wtom;
+ struct timespec64 wtom;
do {
seq = read_seqcount_begin(&timekeeper_seq);
wtom = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- return timespec_to_ktime(wtom);
+ return timespec64_to_ktime(wtom);
}
EXPORT_SYMBOL_GPL(ktime_get_monotonic_offset);
@@ -1668,7 +1688,8 @@ int do_adjtimex(struct timex *txc)
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
- struct timespec ts;
+ struct timespec64 ts;
+ struct timespec tmp;
s32 orig_tai, tai;
int ret;
@@ -1688,7 +1709,8 @@ int do_adjtimex(struct timex *txc)
return ret;
}
- getnstimeofday(&ts);
+ getnstimeofday(&tmp);
+ ts = timespec_to_timespec64(tmp);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
Index: tip/kernel/time/timekeeping_debug.c
===================================================================
--- tip.orig/kernel/time/timekeeping_debug.c
+++ tip/kernel/time/timekeeping_debug.c
@@ -67,7 +67,7 @@ static int __init tk_debug_sleep_time_in
}
late_initcall(tk_debug_sleep_time_init);
-void tk_debug_account_sleep_time(struct timespec *t)
+void tk_debug_account_sleep_time(struct timespec64 *t)
{
sleep_time_bin[fls(t->tv_sec)]++;
}
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -6,7 +6,7 @@
#include <linux/time.h>
#ifdef CONFIG_DEBUG_FS
-extern void tk_debug_account_sleep_time(struct timespec *t);
+extern void tk_debug_account_sleep_time(struct timespec64 *t);
#else
#define tk_debug_account_sleep_time(x)
#endif
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 10/64] time: Consolidate the time accessor prototypes
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (8 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 09/64] timekeeping: Convert timekeeping core to use timespec64s Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 11/64] timekeeping: Provide timespec64 based interfaces Thomas Gleixner
` (54 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: time-move-all-accessor-prototypes-into-separate-header.patch --]
[-- Type: text/plain, Size: 9480 bytes --]
Right now we have time related prototypes in 3 different header
files. Move it to a single timekeeping header file and move the core
internal stuff into a core private header.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/hrtimer.h | 11 ------
include/linux/ktime.h | 8 +---
include/linux/time.h | 45 ++-----------------------
include/linux/timekeeping.h | 78 ++++++++++++++++++++++++++++++++++++++++++++
kernel/time/hrtimer.c | 2 +
kernel/time/posix-timers.c | 2 +
kernel/time/tick-internal.h | 2 +
kernel/time/time.c | 1
kernel/time/timekeeping.h | 20 +++++++++++
9 files changed, 111 insertions(+), 58 deletions(-)
Index: tip/include/linux/hrtimer.h
===================================================================
--- tip.orig/include/linux/hrtimer.h
+++ tip/include/linux/hrtimer.h
@@ -326,17 +326,6 @@ static inline void timerfd_clock_was_set
#endif
extern void hrtimers_resume(void);
-extern ktime_t ktime_get(void);
-extern ktime_t ktime_get_real(void);
-extern ktime_t ktime_get_boottime(void);
-extern ktime_t ktime_get_monotonic_offset(void);
-extern ktime_t ktime_get_clocktai(void);
-extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
- ktime_t *offs_boot,
- ktime_t *offs_tai);
-extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
- ktime_t *offs_boot,
- ktime_t *offs_tai);
DECLARE_PER_CPU(struct tick_device, tick_cpu_device);
Index: tip/include/linux/ktime.h
===================================================================
--- tip.orig/include/linux/ktime.h
+++ tip/include/linux/ktime.h
@@ -250,12 +250,6 @@ static inline __must_check bool ktime_to
#define LOW_RES_NSEC TICK_NSEC
#define KTIME_LOW_RES (ktime_t){ .tv64 = LOW_RES_NSEC }
-/* Get the monotonic time in timespec format: */
-extern void ktime_get_ts(struct timespec *ts);
-
-/* Get the real (wall-) time in timespec format: */
-#define ktime_get_real_ts(ts) getnstimeofday(ts)
-
static inline ktime_t ns_to_ktime(u64 ns)
{
static const ktime_t ktime_zero = { .tv64 = 0 };
@@ -270,4 +264,6 @@ static inline ktime_t ms_to_ktime(u64 ms
return ktime_add_ms(ktime_zero, ms);
}
+# include <linux/timekeeping.h>
+
#endif
Index: tip/include/linux/time.h
===================================================================
--- tip.orig/include/linux/time.h
+++ tip/include/linux/time.h
@@ -99,25 +99,7 @@ static inline bool timespec_valid_strict
return true;
}
-extern bool persistent_clock_exist;
-
-static inline bool has_persistent_clock(void)
-{
- return persistent_clock_exist;
-}
-
-extern void read_persistent_clock(struct timespec *ts);
-extern void read_boot_clock(struct timespec *ts);
-extern int persistent_clock_is_local;
-extern int update_persistent_clock(struct timespec now);
-void timekeeping_init(void);
-extern int timekeeping_suspended;
-
-unsigned long get_seconds(void);
-struct timespec current_kernel_time(void);
-struct timespec __current_kernel_time(void); /* does not take xtime_lock */
-struct timespec get_monotonic_coarse(void);
-void timekeeping_inject_sleeptime(struct timespec *delta);
+extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
#define CURRENT_TIME (current_kernel_time())
#define CURRENT_TIME_SEC ((struct timespec) { get_seconds(), 0 })
@@ -135,33 +117,14 @@ void timekeeping_inject_sleeptime(struct
extern u32 (*arch_gettimeoffset)(void);
#endif
-extern void do_gettimeofday(struct timeval *tv);
-extern int do_settimeofday(const struct timespec *tv);
-extern int do_sys_settimeofday(const struct timespec *tv,
- const struct timezone *tz);
-#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
-extern long do_utimes(int dfd, const char __user *filename, struct timespec *times, int flags);
struct itimerval;
extern int do_setitimer(int which, struct itimerval *value,
struct itimerval *ovalue);
-extern unsigned int alarm_setitimer(unsigned int seconds);
extern int do_getitimer(int which, struct itimerval *value);
-extern int __getnstimeofday(struct timespec *tv);
-extern void getnstimeofday(struct timespec *tv);
-extern void getrawmonotonic(struct timespec *ts);
-extern void getnstime_raw_and_real(struct timespec *ts_raw,
- struct timespec *ts_real);
-extern void getboottime(struct timespec *ts);
-extern void monotonic_to_bootbased(struct timespec *ts);
-extern void get_monotonic_boottime(struct timespec *ts);
-extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
-extern int timekeeping_valid_for_hres(void);
-extern u64 timekeeping_max_deferment(void);
-extern int timekeeping_inject_offset(struct timespec *ts);
-extern s32 timekeeping_get_tai_offset(void);
-extern void timekeeping_set_tai_offset(s32 tai_offset);
-extern void timekeeping_clocktai(struct timespec *ts);
+extern unsigned int alarm_setitimer(unsigned int seconds);
+
+extern long do_utimes(int dfd, const char __user *filename, struct timespec *times, int flags);
struct tms;
extern void do_sys_times(struct tms *);
Index: tip/include/linux/timekeeping.h
===================================================================
--- /dev/null
+++ tip/include/linux/timekeeping.h
@@ -0,0 +1,78 @@
+#ifndef _LINUX_TIMEKEEPING_H
+#define _LINUX_TIMEKEEPING_H
+
+/* Included from linux/ktime.h */
+
+void timekeeping_init(void);
+extern int timekeeping_suspended;
+
+/*
+ * Get and set timeofday
+ */
+extern void do_gettimeofday(struct timeval *tv);
+extern int do_settimeofday(const struct timespec *tv);
+extern int do_sys_settimeofday(const struct timespec *tv,
+ const struct timezone *tz);
+
+/*
+ * Kernel time accessors
+ */
+unsigned long get_seconds(void);
+struct timespec current_kernel_time(void);
+/* does not take xtime_lock */
+struct timespec __current_kernel_time(void);
+
+/*
+ * timespec based interfaces
+ */
+struct timespec get_monotonic_coarse(void);
+extern void getrawmonotonic(struct timespec *ts);
+extern void monotonic_to_bootbased(struct timespec *ts);
+extern void get_monotonic_boottime(struct timespec *ts);
+extern void ktime_get_ts(struct timespec *ts);
+
+extern int __getnstimeofday(struct timespec *tv);
+extern void getnstimeofday(struct timespec *tv);
+extern void getboottime(struct timespec *ts);
+
+#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
+#define ktime_get_real_ts(ts) getnstimeofday(ts)
+
+
+/*
+ * ktime_t based interfaces
+ */
+extern ktime_t ktime_get(void);
+extern ktime_t ktime_get_real(void);
+extern ktime_t ktime_get_boottime(void);
+extern ktime_t ktime_get_monotonic_offset(void);
+extern ktime_t ktime_get_clocktai(void);
+
+/*
+ * RTC specific
+ */
+extern void timekeeping_inject_sleeptime(struct timespec *delta);
+
+/*
+ * PPS accessor
+ */
+extern void getnstime_raw_and_real(struct timespec *ts_raw,
+ struct timespec *ts_real);
+
+/*
+ * Persistent clock related interfaces
+ */
+extern bool persistent_clock_exist;
+extern int persistent_clock_is_local;
+
+static inline bool has_persistent_clock(void)
+{
+ return persistent_clock_exist;
+}
+
+extern void read_persistent_clock(struct timespec *ts);
+extern void read_boot_clock(struct timespec *ts);
+extern int update_persistent_clock(struct timespec now);
+
+
+#endif
Index: tip/kernel/time/hrtimer.c
===================================================================
--- tip.orig/kernel/time/hrtimer.c
+++ tip/kernel/time/hrtimer.c
@@ -54,6 +54,8 @@
#include <trace/events/timer.h>
+#include "timekeeping.h"
+
/*
* The timer bases:
*
Index: tip/kernel/time/posix-timers.c
===================================================================
--- tip.orig/kernel/time/posix-timers.c
+++ tip/kernel/time/posix-timers.c
@@ -49,6 +49,8 @@
#include <linux/export.h>
#include <linux/hashtable.h>
+#include "timekeeping.h"
+
/*
* Management arrays for POSIX timers. Timers are now kept in static hash table
* with 512 entries.
Index: tip/kernel/time/tick-internal.h
===================================================================
--- tip.orig/kernel/time/tick-internal.h
+++ tip/kernel/time/tick-internal.h
@@ -4,6 +4,8 @@
#include <linux/hrtimer.h>
#include <linux/tick.h>
+#include "timekeeping.h"
+
extern seqlock_t jiffies_lock;
#define CS_NAME_LEN 32
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -42,6 +42,7 @@
#include <asm/unistd.h>
#include "timeconst.h"
+#include "timekeeping.h"
/*
* The timezone where the local system is located. Used as a default by some
Index: tip/kernel/time/timekeeping.h
===================================================================
--- /dev/null
+++ tip/kernel/time/timekeeping.h
@@ -0,0 +1,20 @@
+#ifndef _KERNEL_TIME_TIMEKEEPING_H
+#define _KERNEL_TIME_TIMEKEEPING_H
+/*
+ * Internal interfaces for kernel/time/
+ */
+extern ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+extern ktime_t ktime_get_update_offsets_now(ktime_t *offs_real,
+ ktime_t *offs_boot,
+ ktime_t *offs_tai);
+
+extern int timekeeping_valid_for_hres(void);
+extern u64 timekeeping_max_deferment(void);
+extern int timekeeping_inject_offset(struct timespec *ts);
+extern s32 timekeeping_get_tai_offset(void);
+extern void timekeeping_set_tai_offset(s32 tai_offset);
+extern void timekeeping_clocktai(struct timespec *ts);
+
+#endif
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 11/64] timekeeping: Provide timespec64 based interfaces
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (9 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 10/64] time: Consolidate the time accessor prototypes Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 12/64] timekeeper: Move tk_xtime to core code Thomas Gleixner
` (53 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ts64-interfaces.patch --]
[-- Type: text/plain, Size: 7147 bytes --]
To convert callers of the core code to timespec64 we need to provide
the proper interfaces.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 66 ++++++++++++++++++++++++++++++++++++++++----
kernel/time/ntp.c | 7 ++--
kernel/time/timekeeping.c | 47 ++++++++++++++-----------------
3 files changed, 87 insertions(+), 33 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -29,15 +29,71 @@ struct timespec get_monotonic_coarse(voi
extern void getrawmonotonic(struct timespec *ts);
extern void monotonic_to_bootbased(struct timespec *ts);
extern void get_monotonic_boottime(struct timespec *ts);
-extern void ktime_get_ts(struct timespec *ts);
+extern void ktime_get_ts64(struct timespec64 *ts);
+
+extern int __getnstimeofday64(struct timespec64 *tv);
+extern void getnstimeofday64(struct timespec64 *tv);
+
+#if BITS_PER_LONG == 64
+static inline int __getnstimeofday(struct timespec *ts)
+{
+ return __getnstimeofday64(ts);
+}
+
+static inline void getnstimeofday(struct timespec *ts)
+{
+ getnstimeofday64(ts);
+}
+
+static inline void ktime_get_ts(struct timespec *ts)
+{
+ ktime_get_ts64(ts);
+}
+
+static inline void ktime_get_real_ts(struct timespec *ts)
+{
+ getnstimeofday64(ts);
+}
+
+#else
+static inline int __getnstimeofday(struct timespec *ts)
+{
+ struct timespec64 ts64;
+ int ret = __getnstimeofday64(&ts64);
+
+ *ts = timespec64_to_timespec(ts64);
+ return ret;
+}
+
+static inline void getnstimeofday(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ getnstimeofday64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+
+static inline void ktime_get_ts(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ ktime_get_ts64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+
+static inline void ktime_get_real_ts(struct timespec *ts)
+{
+ struct timespec64 ts64;
+
+ getnstimeofday64(&ts64);
+ *ts = timespec64_to_timespec(ts64);
+}
+#endif
-extern int __getnstimeofday(struct timespec *tv);
-extern void getnstimeofday(struct timespec *tv);
extern void getboottime(struct timespec *ts);
#define do_posix_clock_monotonic_gettime(ts) ktime_get_ts(ts)
-#define ktime_get_real_ts(ts) getnstimeofday(ts)
-
+#define ktime_get_real_ts64(ts) getnstimeofday64(ts)
/*
* ktime_t based interfaces
Index: tip/kernel/time/ntp.c
===================================================================
--- tip.orig/kernel/time/ntp.c
+++ tip/kernel/time/ntp.c
@@ -466,7 +466,8 @@ static DECLARE_DELAYED_WORK(sync_cmos_wo
static void sync_cmos_clock(struct work_struct *work)
{
- struct timespec now, next;
+ struct timespec64 now;
+ struct timespec next;
int fail = 1;
/*
@@ -485,9 +486,9 @@ static void sync_cmos_clock(struct work_
return;
}
- getnstimeofday(&now);
+ getnstimeofday64(&now);
if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec * 5) {
- struct timespec adjust = now;
+ struct timespec adjust = timespec64_to_timespec(now);
fail = -ENODEV;
if (persistent_clock_is_local)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -285,13 +285,13 @@ static void timekeeping_forward_now(stru
}
/**
- * __getnstimeofday - Returns the time of day in a timespec.
+ * __getnstimeofday64 - Returns the time of day in a timespec64.
* @ts: pointer to the timespec to be set
*
* Updates the time of day in the timespec.
* Returns 0 on success, or -ve when suspended (timespec will be undefined).
*/
-int __getnstimeofday(struct timespec *ts)
+int __getnstimeofday64(struct timespec64 *ts)
{
struct timekeeper *tk = &timekeeper;
unsigned long seq;
@@ -306,7 +306,7 @@ int __getnstimeofday(struct timespec *ts
} while (read_seqcount_retry(&timekeeper_seq, seq));
ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
+ timespec64_add_ns(ts, nsecs);
/*
* Do not bail out early, in case there were callers still using
@@ -316,19 +316,19 @@ int __getnstimeofday(struct timespec *ts
return -EAGAIN;
return 0;
}
-EXPORT_SYMBOL(__getnstimeofday);
+EXPORT_SYMBOL(__getnstimeofday64);
/**
- * getnstimeofday - Returns the time of day in a timespec.
+ * getnstimeofday64 - Returns the time of day in a timespec64.
* @ts: pointer to the timespec to be set
*
* Returns the time of day in a timespec (WARN if suspended).
*/
-void getnstimeofday(struct timespec *ts)
+void getnstimeofday64(struct timespec64 *ts)
{
- WARN_ON(__getnstimeofday(ts));
+ WARN_ON(__getnstimeofday64(ts));
}
-EXPORT_SYMBOL(getnstimeofday);
+EXPORT_SYMBOL(getnstimeofday64);
ktime_t ktime_get(void)
{
@@ -350,17 +350,17 @@ ktime_t ktime_get(void)
EXPORT_SYMBOL_GPL(ktime_get);
/**
- * ktime_get_ts - get the monotonic clock in timespec format
+ * ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
* The function calculates the monotonic clock from the realtime
* clock and the wall_to_monotonic offset and stores the result
* in normalized timespec format in the variable pointed to by @ts.
*/
-void ktime_get_ts(struct timespec *ts)
+void ktime_get_ts64(struct timespec64 *ts)
{
struct timekeeper *tk = &timekeeper;
- struct timespec64 ts64, tomono;
+ struct timespec64 tomono;
s64 nsec;
unsigned int seq;
@@ -368,18 +368,17 @@ void ktime_get_ts(struct timespec *ts)
do {
seq = read_seqcount_begin(&timekeeper_seq);
- ts64.tv_sec = tk->xtime_sec;
+ ts->tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&timekeeper_seq, seq));
- ts64.tv_sec += tomono.tv_sec;
- ts64.tv_nsec = 0;
- timespec64_add_ns(&ts64, nsec + tomono.tv_nsec);
- *ts = timespec64_to_timespec(ts64);
+ ts->tv_sec += tomono.tv_sec;
+ ts->tv_nsec = 0;
+ timespec64_add_ns(ts, nsec + tomono.tv_nsec);
}
-EXPORT_SYMBOL_GPL(ktime_get_ts);
+EXPORT_SYMBOL_GPL(ktime_get_ts64);
/**
@@ -473,9 +472,9 @@ EXPORT_SYMBOL(getnstime_raw_and_real);
*/
void do_gettimeofday(struct timeval *tv)
{
- struct timespec now;
+ struct timespec64 now;
- getnstimeofday(&now);
+ getnstimeofday64(&now);
tv->tv_sec = now.tv_sec;
tv->tv_usec = now.tv_nsec/1000;
}
@@ -680,11 +679,11 @@ int timekeeping_notify(struct clocksourc
*/
ktime_t ktime_get_real(void)
{
- struct timespec now;
+ struct timespec64 now;
- getnstimeofday(&now);
+ getnstimeofday64(&now);
- return timespec_to_ktime(now);
+ return timespec64_to_ktime(now);
}
EXPORT_SYMBOL_GPL(ktime_get_real);
@@ -1689,7 +1688,6 @@ int do_adjtimex(struct timex *txc)
struct timekeeper *tk = &timekeeper;
unsigned long flags;
struct timespec64 ts;
- struct timespec tmp;
s32 orig_tai, tai;
int ret;
@@ -1709,8 +1707,7 @@ int do_adjtimex(struct timex *txc)
return ret;
}
- getnstimeofday(&tmp);
- ts = timespec_to_timespec64(tmp);
+ getnstimeofday64(&ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&timekeeper_seq);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 12/64] timekeeper: Move tk_xtime to core code
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (10 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 11/64] timekeeping: Provide timespec64 based interfaces Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-23 21:15 ` John Stultz
2014-07-16 21:04 ` [patch V2 13/64] timekeeping: Cache optimize struct timekeeper Thomas Gleixner
` (52 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeper-move-tk-xtime-to-core.patch --]
[-- Type: text/plain, Size: 4190 bytes --]
No users outside of the core.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 18 ---------
kernel/time/timekeeping.c | 70 ++++++++++++++++++++++--------------
2 files changed, 43 insertions(+), 45 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -71,16 +71,6 @@ struct timekeeper {
};
-static inline struct timespec64 tk_xtime(struct timekeeper *tk)
-{
- struct timespec64 ts;
-
- ts.tv_sec = tk->xtime_sec;
- ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
- return ts;
-}
-
-
#ifdef CONFIG_GENERIC_TIME_VSYSCALL
extern void update_vsyscall(struct timekeeper *tk);
@@ -92,14 +82,6 @@ extern void update_vsyscall_old(struct t
struct clocksource *c, u32 mult);
extern void update_vsyscall_tz(void);
-static inline void update_vsyscall(struct timekeeper *tk)
-{
- struct timespec xt;
-
- xt = tk_xtime(tk);
- update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
-}
-
#else
static inline void update_vsyscall(struct timekeeper *tk)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -51,6 +51,15 @@ static inline void tk_normalize_xtime(st
}
}
+static inline struct timespec64 tk_xtime(struct timekeeper *tk)
+{
+ struct timespec64 ts;
+
+ ts.tv_sec = tk->xtime_sec;
+ ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
+ return ts;
+}
+
static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
@@ -199,6 +208,40 @@ static inline s64 timekeeping_get_ns_raw
return nsec + arch_gettimeoffset();
}
+#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
+
+static inline void update_vsyscall(struct timekeeper *tk)
+{
+ struct timespec xt;
+
+ xt = tk_xtime(tk);
+ update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
+}
+
+static inline void old_vsyscall_fixup(struct timekeeper *tk)
+{
+ s64 remainder;
+
+ /*
+ * Store only full nanoseconds into xtime_nsec after rounding
+ * it up and add the remainder to the error difference.
+ * XXX - This is necessary to avoid small 1ns inconsistnecies caused
+ * by truncating the remainder in vsyscalls. However, it causes
+ * additional work to be done in timekeeping_adjust(). Once
+ * the vsyscall implementations are converted to use xtime_nsec
+ * (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
+ * users are removed, this can be killed.
+ */
+ remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
+ tk->xtime_nsec -= remainder;
+ tk->xtime_nsec += 1ULL << tk->shift;
+ tk->ntp_error += remainder << tk->ntp_error_shift;
+ tk->ntp_error -= (1ULL << tk->shift) << tk->ntp_error_shift;
+}
+#else
+#define old_vsyscall_fixup(tk)
+#endif
+
static RAW_NOTIFIER_HEAD(pvclock_gtod_chain);
static void update_pvclock_gtod(struct timekeeper *tk, bool was_set)
@@ -1330,33 +1373,6 @@ static cycle_t logarithmic_accumulation(
return offset;
}
-#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
-static inline void old_vsyscall_fixup(struct timekeeper *tk)
-{
- s64 remainder;
-
- /*
- * Store only full nanoseconds into xtime_nsec after rounding
- * it up and add the remainder to the error difference.
- * XXX - This is necessary to avoid small 1ns inconsistnecies caused
- * by truncating the remainder in vsyscalls. However, it causes
- * additional work to be done in timekeeping_adjust(). Once
- * the vsyscall implementations are converted to use xtime_nsec
- * (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
- * users are removed, this can be killed.
- */
- remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
- tk->xtime_nsec -= remainder;
- tk->xtime_nsec += 1ULL << tk->shift;
- tk->ntp_error += remainder << tk->ntp_error_shift;
- tk->ntp_error -= (1ULL << tk->shift) << tk->ntp_error_shift;
-}
-#else
-#define old_vsyscall_fixup(tk)
-#endif
-
-
-
/**
* update_wall_time - Uses the current clocksource to increment the wall time
*
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 12/64] timekeeper: Move tk_xtime to core code
2014-07-16 21:04 ` [patch V2 12/64] timekeeper: Move tk_xtime to core code Thomas Gleixner
@ 2014-07-23 21:15 ` John Stultz
2014-07-23 21:59 ` Thomas Gleixner
0 siblings, 1 reply; 76+ messages in thread
From: John Stultz @ 2014-07-23 21:15 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, Peter Zijlstra
On Wed, Jul 16, 2014 at 2:04 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Index: tip/kernel/time/timekeeping.c
> ===================================================================
> --- tip.orig/kernel/time/timekeeping.c
> +++ tip/kernel/time/timekeeping.c
> @@ -51,6 +51,15 @@ static inline void tk_normalize_xtime(st
> }
> }
>
> +static inline struct timespec64 tk_xtime(struct timekeeper *tk)
> +{
> + struct timespec64 ts;
> +
> + ts.tv_sec = tk->xtime_sec;
> + ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
> + return ts;
> +}
> +
> static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
> {
> tk->xtime_sec = ts->tv_sec;
> @@ -199,6 +208,40 @@ static inline s64 timekeeping_get_ns_raw
> return nsec + arch_gettimeoffset();
> }
>
> +#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
> +
> +static inline void update_vsyscall(struct timekeeper *tk)
> +{
> + struct timespec xt;
> +
> + xt = tk_xtime(tk);
> + update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
> +}
So one gotcha here, and I realized I missed this in my timekeeping
timespec64 conversion patch, is that here we're calling tk_xtime()
which returns a timespec64, and we stuff it into a timespec.
Now, the reason we didn't see the compiler gripe on this, was that the
only VSYSCALL_OLD implementations left are ia64 and ppc64.
I'll add a fixup patch to the queue to address this.
thanks
-john
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 12/64] timekeeper: Move tk_xtime to core code
2014-07-23 21:15 ` John Stultz
@ 2014-07-23 21:59 ` Thomas Gleixner
0 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-23 21:59 UTC (permalink / raw)
To: John Stultz; +Cc: LKML, Peter Zijlstra
On Wed, 23 Jul 2014, John Stultz wrote:
> On Wed, Jul 16, 2014 at 2:04 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > Index: tip/kernel/time/timekeeping.c
> > ===================================================================
> > --- tip.orig/kernel/time/timekeeping.c
> > +++ tip/kernel/time/timekeeping.c
> > @@ -51,6 +51,15 @@ static inline void tk_normalize_xtime(st
> > }
> > }
> >
> > +static inline struct timespec64 tk_xtime(struct timekeeper *tk)
> > +{
> > + struct timespec64 ts;
> > +
> > + ts.tv_sec = tk->xtime_sec;
> > + ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
> > + return ts;
> > +}
> > +
> > static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
> > {
> > tk->xtime_sec = ts->tv_sec;
> > @@ -199,6 +208,40 @@ static inline s64 timekeeping_get_ns_raw
> > return nsec + arch_gettimeoffset();
> > }
> >
> > +#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
> > +
> > +static inline void update_vsyscall(struct timekeeper *tk)
> > +{
> > + struct timespec xt;
> > +
> > + xt = tk_xtime(tk);
> > + update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
> > +}
>
> So one gotcha here, and I realized I missed this in my timekeeping
> timespec64 conversion patch, is that here we're calling tk_xtime()
> which returns a timespec64, and we stuff it into a timespec.
>
> Now, the reason we didn't see the compiler gripe on this, was that the
> only VSYSCALL_OLD implementations left are ia64 and ppc64.
>
> I'll add a fixup patch to the queue to address this.
Nice catch!
Thanks,
tglx
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 13/64] timekeeping: Cache optimize struct timekeeper
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (11 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 12/64] timekeeper: Move tk_xtime to core code Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 14/64] timekeeping: Use timekeeping_update() instead of memcpy() Thomas Gleixner
` (51 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-optimize-cache-layout.patch --]
[-- Type: text/plain, Size: 25341 bytes --]
struct timekeeper is quite badly sorted for the hot readout path. Most
time access functions need to load two cache lines.
Rearrange it so ktime_get() and getnstimeofday() are happy with a
single cache line.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 84 ++++++++--------
kernel/time/timekeeping.c | 185 ++++++++++++++++++------------------
2 files changed, 143 insertions(+), 126 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -10,7 +10,22 @@
#include <linux/jiffies.h>
#include <linux/time.h>
-/* Structure holding internal timekeeping values. */
+/*
+ * Structure holding internal timekeeping values.
+ *
+ * Note: wall_to_monotonic is what we need to add to xtime (or xtime
+ * corrected for sub jiffie times) to get to monotonic time.
+ * Monotonic is pegged at zero at system boot time, so
+ * wall_to_monotonic will be negative, however, we will ALWAYS keep
+ * the tv_nsec part positive so we can use the usual normalization.
+ *
+ * wall_to_monotonic is moved after resume from suspend for the
+ * monotonic time not to jump. We need to add total_sleep_time to
+ * wall_to_monotonic to get the real boot based time offset.
+ *
+ * - wall_to_monotonic is no longer the boot time, getboottime must be
+ * used instead.
+ */
struct timekeeper {
/* Current clocksource used for timekeeping. */
struct clocksource *clock;
@@ -18,6 +33,29 @@ struct timekeeper {
u32 mult;
/* The shift value of the current clocksource. */
u32 shift;
+ /* Clock shifted nano seconds */
+ u64 xtime_nsec;
+
+ /* Current CLOCK_REALTIME time in seconds */
+ u64 xtime_sec;
+ /* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
+ struct timespec64 wall_to_monotonic;
+
+ /* Offset clock monotonic -> clock realtime */
+ ktime_t offs_real;
+ /* Offset clock monotonic -> clock boottime */
+ ktime_t offs_boot;
+ /* Offset clock monotonic -> clock tai */
+ ktime_t offs_tai;
+
+ /* time spent in suspend */
+ struct timespec64 total_sleep_time;
+ /* The current UTC to TAI offset in seconds */
+ s32 tai_offset;
+
+ /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
+ struct timespec64 raw_time;
+
/* Number of clock cycles in one NTP interval. */
cycle_t cycle_interval;
/* Last cycle value (also stored in clock->cycle_last) */
@@ -29,46 +67,16 @@ struct timekeeper {
/* Raw nano seconds accumulated per NTP interval. */
u32 raw_interval;
- /* Current CLOCK_REALTIME time in seconds */
- u64 xtime_sec;
- /* Clock shifted nano seconds */
- u64 xtime_nsec;
-
- /* Difference between accumulated time and NTP time in ntp
- * shifted nano seconds. */
+ /*
+ * Difference between accumulated time and NTP time in ntp
+ * shifted nano seconds.
+ */
s64 ntp_error;
- /* Shift conversion between clock shifted nano seconds and
- * ntp shifted nano seconds. */
- u32 ntp_error_shift;
-
/*
- * wall_to_monotonic is what we need to add to xtime (or xtime corrected
- * for sub jiffie times) to get to monotonic time. Monotonic is pegged
- * at zero at system boot time, so wall_to_monotonic will be negative,
- * however, we will ALWAYS keep the tv_nsec part positive so we can use
- * the usual normalization.
- *
- * wall_to_monotonic is moved after resume from suspend for the
- * monotonic time not to jump. We need to add total_sleep_time to
- * wall_to_monotonic to get the real boot based time offset.
- *
- * - wall_to_monotonic is no longer the boot time, getboottime must be
- * used instead.
+ * Shift conversion between clock shifted nano seconds and
+ * ntp shifted nano seconds.
*/
- struct timespec64 wall_to_monotonic;
- /* Offset clock monotonic -> clock realtime */
- ktime_t offs_real;
- /* time spent in suspend */
- struct timespec64 total_sleep_time;
- /* Offset clock monotonic -> clock boottime */
- ktime_t offs_boot;
- /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
- struct timespec64 raw_time;
- /* The current UTC to TAI offset in seconds */
- s32 tai_offset;
- /* Offset clock monotonic -> clock tai */
- ktime_t offs_tai;
-
+ u32 ntp_error_shift;
};
#ifdef CONFIG_GENERIC_TIME_VSYSCALL
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -32,9 +32,16 @@
#define TK_MIRROR (1 << 1)
#define TK_CLOCK_WAS_SET (1 << 2)
-static struct timekeeper timekeeper;
+/*
+ * The most important data for readout fits into a single 64 byte
+ * cache line.
+ */
+static struct {
+ seqcount_t seq;
+ struct timekeeper timekeeper;
+} tk_core ____cacheline_aligned;
+
static DEFINE_RAW_SPINLOCK(timekeeper_lock);
-static seqcount_t timekeeper_seq;
static struct timekeeper shadow_timekeeper;
/* flag for if timekeeping is suspended */
@@ -254,7 +261,7 @@ static void update_pvclock_gtod(struct t
*/
int pvclock_gtod_register_notifier(struct notifier_block *nb)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
int ret;
@@ -295,7 +302,8 @@ static void timekeeping_update(struct ti
update_pvclock_gtod(tk, action & TK_CLOCK_WAS_SET);
if (action & TK_MIRROR)
- memcpy(&shadow_timekeeper, &timekeeper, sizeof(timekeeper));
+ memcpy(&shadow_timekeeper, &tk_core.timekeeper,
+ sizeof(tk_core.timekeeper));
}
/**
@@ -336,17 +344,17 @@ static void timekeeping_forward_now(stru
*/
int __getnstimeofday64(struct timespec64 *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
s64 nsecs = 0;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
nsecs = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts->tv_nsec = 0;
timespec64_add_ns(ts, nsecs);
@@ -375,18 +383,18 @@ EXPORT_SYMBOL(getnstimeofday64);
ktime_t ktime_get(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
s64 secs, nsecs;
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ktime_set(secs, nsecs);
}
@@ -402,7 +410,7 @@ EXPORT_SYMBOL_GPL(ktime_get);
*/
void ktime_get_ts64(struct timespec64 *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tomono;
s64 nsec;
unsigned int seq;
@@ -410,12 +418,12 @@ void ktime_get_ts64(struct timespec64 *t
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts->tv_sec += tomono.tv_sec;
ts->tv_nsec = 0;
@@ -432,7 +440,7 @@ EXPORT_SYMBOL_GPL(ktime_get_ts64);
*/
void timekeeping_clocktai(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
unsigned long seq;
u64 nsecs;
@@ -440,12 +448,12 @@ void timekeeping_clocktai(struct timespe
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts64.tv_sec = tk->xtime_sec + tk->tai_offset;
nsecs = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ts64.tv_nsec = 0;
timespec64_add_ns(&ts64, nsecs);
@@ -482,14 +490,14 @@ EXPORT_SYMBOL(ktime_get_clocktai);
*/
void getnstime_raw_and_real(struct timespec *ts_raw, struct timespec *ts_real)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
s64 nsecs_raw, nsecs_real;
WARN_ON_ONCE(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
*ts_raw = timespec64_to_timespec(tk->raw_time);
ts_real->tv_sec = tk->xtime_sec;
@@ -498,7 +506,7 @@ void getnstime_raw_and_real(struct times
nsecs_raw = timekeeping_get_ns_raw(tk);
nsecs_real = timekeeping_get_ns(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
timespec_add_ns(ts_raw, nsecs_raw);
timespec_add_ns(ts_real, nsecs_real);
@@ -531,7 +539,7 @@ EXPORT_SYMBOL(do_gettimeofday);
*/
int do_settimeofday(const struct timespec *tv)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts_delta, xt, tmp;
unsigned long flags;
@@ -539,7 +547,7 @@ int do_settimeofday(const struct timespe
return -EINVAL;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -554,7 +562,7 @@ int do_settimeofday(const struct timespe
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -572,7 +580,7 @@ EXPORT_SYMBOL(do_settimeofday);
*/
int timekeeping_inject_offset(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 ts64, tmp;
int ret = 0;
@@ -583,7 +591,7 @@ int timekeeping_inject_offset(struct tim
ts64 = timespec_to_timespec64(*ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -600,7 +608,7 @@ int timekeeping_inject_offset(struct tim
error: /* even if we error out, we forwarded the time, so call update */
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -617,14 +625,14 @@ EXPORT_SYMBOL(timekeeping_inject_offset)
*/
s32 timekeeping_get_tai_offset(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
s32 ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->tai_offset;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -645,14 +653,14 @@ static void __timekeeping_set_tai_offset
*/
void timekeeping_set_tai_offset(s32 tai_offset)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
__timekeeping_set_tai_offset(tk, tai_offset);
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
clock_was_set();
}
@@ -664,14 +672,14 @@ void timekeeping_set_tai_offset(s32 tai_
*/
static int change_clocksource(void *data)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *new, *old;
unsigned long flags;
new = (struct clocksource *) data;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
/*
@@ -691,7 +699,7 @@ static int change_clocksource(void *data
}
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
return 0;
@@ -706,7 +714,7 @@ static int change_clocksource(void *data
*/
int timekeeping_notify(struct clocksource *clock)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
if (tk->clock == clock)
return 0;
@@ -738,17 +746,17 @@ EXPORT_SYMBOL_GPL(ktime_get_real);
*/
void getrawmonotonic(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
unsigned long seq;
s64 nsecs;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
nsecs = timekeeping_get_ns_raw(tk);
ts64 = tk->raw_time;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
timespec64_add_ns(&ts64, nsecs);
*ts = timespec64_to_timespec(ts64);
@@ -760,16 +768,16 @@ EXPORT_SYMBOL(getrawmonotonic);
*/
int timekeeping_valid_for_hres(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
int ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -779,16 +787,16 @@ int timekeeping_valid_for_hres(void)
*/
u64 timekeeping_max_deferment(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
u64 ret;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret = tk->clock->max_idle_ns;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return ret;
}
@@ -828,7 +836,7 @@ void __weak read_boot_clock(struct times
*/
void __init timekeeping_init(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *clock;
unsigned long flags;
struct timespec64 now, boot, tmp;
@@ -854,7 +862,7 @@ void __init timekeeping_init(void)
}
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
ntp_init();
clock = clocksource_default_clock();
@@ -875,9 +883,10 @@ void __init timekeeping_init(void)
tmp.tv_nsec = 0;
tk_set_sleep_time(tk, tmp);
- memcpy(&shadow_timekeeper, &timekeeper, sizeof(timekeeper));
+ memcpy(&shadow_timekeeper, &tk_core.timekeeper,
+ sizeof(tk_core.timekeeper));
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
}
@@ -918,7 +927,7 @@ static void __timekeeping_inject_sleepti
*/
void timekeeping_inject_sleeptime(struct timespec *delta)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tmp;
unsigned long flags;
@@ -930,7 +939,7 @@ void timekeeping_inject_sleeptime(struct
return;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
@@ -939,7 +948,7 @@ void timekeeping_inject_sleeptime(struct
timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
/* signal hrtimers about time change */
@@ -955,7 +964,7 @@ void timekeeping_inject_sleeptime(struct
*/
static void timekeeping_resume(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct clocksource *clock = tk->clock;
unsigned long flags;
struct timespec64 ts_new, ts_delta;
@@ -970,7 +979,7 @@ static void timekeeping_resume(void)
clocksource_resume();
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
/*
* After system resumes, we need to calculate the suspended time and
@@ -1022,7 +1031,7 @@ static void timekeeping_resume(void)
tk->ntp_error = 0;
timekeeping_suspended = 0;
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
touch_softlockup_watchdog();
@@ -1035,7 +1044,7 @@ static void timekeeping_resume(void)
static int timekeeping_suspend(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 delta, delta_delta;
static struct timespec64 old_delta;
@@ -1053,7 +1062,7 @@ static int timekeeping_suspend(void)
persistent_clock_exist = true;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
timekeeping_suspended = 1;
@@ -1078,7 +1087,7 @@ static int timekeeping_suspend(void)
}
timekeeping_update(tk, TK_MIRROR);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
clockevents_notify(CLOCK_EVT_NOTIFY_SUSPEND, NULL);
@@ -1380,7 +1389,7 @@ static cycle_t logarithmic_accumulation(
void update_wall_time(void)
{
struct clocksource *clock;
- struct timekeeper *real_tk = &timekeeper;
+ struct timekeeper *real_tk = &tk_core.timekeeper;
struct timekeeper *tk = &shadow_timekeeper;
cycle_t offset;
int shift = 0, maxshift;
@@ -1440,7 +1449,7 @@ void update_wall_time(void)
*/
clock_set |= accumulate_nsecs_to_secs(tk);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
/* Update clock->cycle_last with the new value */
clock->cycle_last = tk->cycle_last;
/*
@@ -1450,12 +1459,12 @@ void update_wall_time(void)
* requires changes to all other timekeeper usage sites as
* well, i.e. move the timekeeper pointer getter into the
* spinlocked/seqcount protected sections. And we trade this
- * memcpy under the timekeeper_seq against one before we start
+ * memcpy under the tk_core.seq against one before we start
* updating.
*/
memcpy(real_tk, tk, sizeof(*tk));
timekeeping_update(real_tk, clock_set);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
out:
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
if (clock_set)
@@ -1476,7 +1485,7 @@ out:
*/
void getboottime(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec boottime = {
.tv_sec = tk->wall_to_monotonic.tv_sec +
tk->total_sleep_time.tv_sec,
@@ -1499,7 +1508,7 @@ EXPORT_SYMBOL_GPL(getboottime);
*/
void get_monotonic_boottime(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 tomono, sleep, ret;
s64 nsec;
unsigned int seq;
@@ -1507,13 +1516,13 @@ void get_monotonic_boottime(struct times
WARN_ON(timekeeping_suspended);
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ret.tv_sec = tk->xtime_sec;
nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
sleep = tk->total_sleep_time;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
ret.tv_nsec = 0;
@@ -1545,7 +1554,7 @@ EXPORT_SYMBOL_GPL(ktime_get_boottime);
*/
void monotonic_to_bootbased(struct timespec *ts)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts64;
ts64 = timespec_to_timespec64(*ts);
@@ -1556,7 +1565,7 @@ EXPORT_SYMBOL_GPL(monotonic_to_bootbased
unsigned long get_seconds(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
return tk->xtime_sec;
}
@@ -1564,22 +1573,22 @@ EXPORT_SYMBOL(get_seconds);
struct timespec __current_kernel_time(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
return timespec64_to_timespec(tk_xtime(tk));
}
struct timespec current_kernel_time(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 now;
unsigned long seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
now = tk_xtime(tk);
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return timespec64_to_timespec(now);
}
@@ -1587,16 +1596,16 @@ EXPORT_SYMBOL(current_kernel_time);
struct timespec get_monotonic_coarse(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 now, mono;
unsigned long seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
now = tk_xtime(tk);
mono = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
set_normalized_timespec64(&now, now.tv_sec + mono.tv_sec,
now.tv_nsec + mono.tv_nsec);
@@ -1624,19 +1633,19 @@ void do_timer(unsigned long ticks)
ktime_t ktime_get_update_offsets_tick(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
struct timespec64 ts;
ktime_t now;
unsigned int seq;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
ts = tk_xtime(tk);
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
now = ktime_set(ts.tv_sec, ts.tv_nsec);
now = ktime_sub(now, *offs_real);
@@ -1656,13 +1665,13 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t ktime_get_update_offsets_now(ktime_t *offs_real, ktime_t *offs_boot,
ktime_t *offs_tai)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
ktime_t now;
unsigned int seq;
u64 secs, nsecs;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
secs = tk->xtime_sec;
nsecs = timekeeping_get_ns(tk);
@@ -1670,7 +1679,7 @@ ktime_t ktime_get_update_offsets_now(kti
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
now = ktime_add_ns(ktime_set(secs, 0), nsecs);
now = ktime_sub(now, *offs_real);
@@ -1683,14 +1692,14 @@ ktime_t ktime_get_update_offsets_now(kti
*/
ktime_t ktime_get_monotonic_offset(void)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long seq;
struct timespec64 wtom;
do {
- seq = read_seqcount_begin(&timekeeper_seq);
+ seq = read_seqcount_begin(&tk_core.seq);
wtom = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&timekeeper_seq, seq));
+ } while (read_seqcount_retry(&tk_core.seq, seq));
return timespec64_to_ktime(wtom);
}
@@ -1701,7 +1710,7 @@ EXPORT_SYMBOL_GPL(ktime_get_monotonic_of
*/
int do_adjtimex(struct timex *txc)
{
- struct timekeeper *tk = &timekeeper;
+ struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
struct timespec64 ts;
s32 orig_tai, tai;
@@ -1726,7 +1735,7 @@ int do_adjtimex(struct timex *txc)
getnstimeofday64(&ts);
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
orig_tai = tai = tk->tai_offset;
ret = __do_adjtimex(txc, &ts, &tai);
@@ -1735,7 +1744,7 @@ int do_adjtimex(struct timex *txc)
__timekeeping_set_tai_offset(tk, tai);
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
}
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
if (tai != orig_tai)
@@ -1755,11 +1764,11 @@ void hardpps(const struct timespec *phas
unsigned long flags;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
- write_seqcount_begin(&timekeeper_seq);
+ write_seqcount_begin(&tk_core.seq);
__hardpps(phase_ts, raw_ts);
- write_seqcount_end(&timekeeper_seq);
+ write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
}
EXPORT_SYMBOL(hardpps);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 14/64] timekeeping: Use timekeeping_update() instead of memcpy()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (12 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 13/64] timekeeping: Cache optimize struct timekeeper Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 15/64] timekeeping: Provide internal ktime_t based data Thomas Gleixner
` (50 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-update-instead-of-memcpy.patch --]
[-- Type: text/plain, Size: 789 bytes --]
We already have a function which does the right thing, that also makes
sure that the coming ktime_t based cached values are getting updated.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -883,8 +883,7 @@ void __init timekeeping_init(void)
tmp.tv_nsec = 0;
tk_set_sleep_time(tk, tmp);
- memcpy(&shadow_timekeeper, &tk_core.timekeeper,
- sizeof(tk_core.timekeeper));
+ timekeeping_update(tk, TK_MIRROR);
write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 15/64] timekeeping: Provide internal ktime_t based data
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (13 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 14/64] timekeeping: Use timekeeping_update() instead of memcpy() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 16/64] timekeeping: Use ktime_t based data for ktime_get() Thomas Gleixner
` (49 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-internal-ktime-based-data.patch --]
[-- Type: text/plain, Size: 2132 bytes --]
The ktime_t based interfaces are used a lot in performance critical
code pathes. Add ktime_t based data so the interfaces don't have to
convert from the xtime/timespec based data.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 3 +++
kernel/time/timekeeping.c | 22 ++++++++++++++++++++++
2 files changed, 25 insertions(+)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -36,6 +36,9 @@ struct timekeeper {
/* Clock shifted nano seconds */
u64 xtime_nsec;
+ /* Monotonic base time */
+ ktime_t base_mono;
+
/* Current CLOCK_REALTIME time in seconds */
u64 xtime_sec;
/* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -291,6 +291,26 @@ int pvclock_gtod_unregister_notifier(str
}
EXPORT_SYMBOL_GPL(pvclock_gtod_unregister_notifier);
+/*
+ * Update the ktime_t based scalar nsec members of the timekeeper
+ */
+static inline void tk_update_ktime_data(struct timekeeper *tk)
+{
+ s64 nsec;
+
+ /*
+ * The xtime based monotonic readout is:
+ * nsec = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec + now();
+ * The ktime based monotonic readout is:
+ * nsec = base_mono + now();
+ * ==> base_mono = (xtime_sec + wtm_sec) * 1e9 + wtm_nsec
+ */
+ nsec = (s64)(tk->xtime_sec + tk->wall_to_monotonic.tv_sec);
+ nsec *= NSEC_PER_SEC;
+ nsec += tk->wall_to_monotonic.tv_nsec;
+ tk->base_mono = ns_to_ktime(nsec);
+}
+
/* must hold timekeeper_lock */
static void timekeeping_update(struct timekeeper *tk, unsigned int action)
{
@@ -301,6 +321,8 @@ static void timekeeping_update(struct ti
update_vsyscall(tk);
update_pvclock_gtod(tk, action & TK_CLOCK_WAS_SET);
+ tk_update_ktime_data(tk);
+
if (action & TK_MIRROR)
memcpy(&shadow_timekeeper, &tk_core.timekeeper,
sizeof(tk_core.timekeeper));
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 16/64] timekeeping: Use ktime_t based data for ktime_get()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (14 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 15/64] timekeeping: Provide internal ktime_t based data Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 17/64] timekeeping: Provide ktime_get_with_offset() Thomas Gleixner
` (48 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get.patch --]
[-- Type: text/plain, Size: 1018 bytes --]
Speed up ktime_get() by using ktime_t based data. Text size shrinks by
64 bytes on x8664.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -407,18 +407,19 @@ ktime_t ktime_get(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
unsigned int seq;
- s64 secs, nsecs;
+ ktime_t base;
+ s64 nsecs;
WARN_ON(timekeeping_suspended);
do {
seq = read_seqcount_begin(&tk_core.seq);
- secs = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsecs = timekeeping_get_ns(tk) + tk->wall_to_monotonic.tv_nsec;
+ base = tk->base_mono;
+ nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&tk_core.seq, seq));
- return ktime_set(secs, nsecs);
+ return ktime_add_ns(base, nsecs);
}
EXPORT_SYMBOL_GPL(ktime_get);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 17/64] timekeeping: Provide ktime_get_with_offset()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (15 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 16/64] timekeeping: Use ktime_t based data for ktime_get() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 18/64] timekeeping: Use ktime_t based data for ktime_get_real() Thomas Gleixner
` (47 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-__ktime_get.patch --]
[-- Type: text/plain, Size: 1958 bytes --]
Provide a helper function which lets us implement ktime_t based
interfaces for real, boot and tai clocks.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 +++++++++
kernel/time/timekeeping.c | 27 +++++++++++++++++++++++++++
2 files changed, 36 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -98,7 +98,16 @@ extern void getboottime(struct timespec
/*
* ktime_t based interfaces
*/
+
+enum tk_offsets {
+ TK_OFFS_REAL,
+ TK_OFFS_BOOT,
+ TK_OFFS_TAI,
+ TK_OFFS_MAX,
+};
+
extern ktime_t ktime_get(void);
+extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -423,6 +423,33 @@ ktime_t ktime_get(void)
}
EXPORT_SYMBOL_GPL(ktime_get);
+static ktime_t *offsets[TK_OFFS_MAX] = {
+ [TK_OFFS_REAL] = &tk_core.timekeeper.offs_real,
+ [TK_OFFS_BOOT] = &tk_core.timekeeper.offs_boot,
+ [TK_OFFS_TAI] = &tk_core.timekeeper.offs_tai,
+};
+
+ktime_t ktime_get_with_offset(enum tk_offsets offs)
+{
+ struct timekeeper *tk = &tk_core.timekeeper;
+ unsigned int seq;
+ ktime_t base, *offset = offsets[offs];
+ s64 nsecs;
+
+ WARN_ON(timekeeping_suspended);
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ base = ktime_add(tk->base_mono, *offset);
+ nsecs = timekeeping_get_ns(tk);
+
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return ktime_add_ns(base, nsecs);
+
+}
+EXPORT_SYMBOL_GPL(ktime_get_with_offset);
+
/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 18/64] timekeeping: Use ktime_t based data for ktime_get_real()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (16 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 17/64] timekeeping: Provide ktime_get_with_offset() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 19/64] timekeeping; Use ktime_t based data for ktime_get_boottime() Thomas Gleixner
` (46 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get-real.patch --]
[-- Type: text/plain, Size: 1571 bytes --]
Speed up the readout.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 8 insertions(+), 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,11 +108,18 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_real(void);
extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
+/**
+ * ktime_get_real - get the real (wall-) time in ktime_t format
+ */
+static inline ktime_t ktime_get_real(void)
+{
+ return ktime_get_with_offset(TK_OFFS_REAL);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -774,21 +774,6 @@ int timekeeping_notify(struct clocksourc
}
/**
- * ktime_get_real - get the real (wall-) time in ktime_t format
- *
- * returns the time in ktime_t format
- */
-ktime_t ktime_get_real(void)
-{
- struct timespec64 now;
-
- getnstimeofday64(&now);
-
- return timespec64_to_ktime(now);
-}
-EXPORT_SYMBOL_GPL(ktime_get_real);
-
-/**
* getrawmonotonic - Returns the raw monotonic time in a timespec
* @ts: pointer to the timespec to be set
*
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 19/64] timekeeping; Use ktime_t based data for ktime_get_boottime()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (17 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 18/64] timekeeping: Use ktime_t based data for ktime_get_real() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 20/64] timekeeping: Use ktime_t based data for ktime_get_clocktai() Thomas Gleixner
` (45 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-get-boottime.patch --]
[-- Type: text/plain, Size: 1925 bytes --]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 12 +++++++++++-
kernel/time/timekeeping.c | 17 -----------------
2 files changed, 11 insertions(+), 18 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,7 +108,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_boottime(void);
extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_get_clocktai(void);
@@ -120,6 +119,17 @@ static inline ktime_t ktime_get_real(voi
return ktime_get_with_offset(TK_OFFS_REAL);
}
+/**
+ * ktime_get_boottime - Returns monotonic time since boot in ktime_t format
+ *
+ * This is similar to CLOCK_MONTONIC/ktime_get, but also includes the
+ * time spent in suspend.
+ */
+static inline ktime_t ktime_get_boottime(void)
+{
+ return ktime_get_with_offset(TK_OFFS_BOOT);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1566,23 +1566,6 @@ void get_monotonic_boottime(struct times
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
/**
- * ktime_get_boottime - Returns monotonic time since boot in a ktime
- *
- * Returns the monotonic time since boot in a ktime
- *
- * This is similar to CLOCK_MONTONIC/ktime_get, but also
- * includes the time spent in suspend.
- */
-ktime_t ktime_get_boottime(void)
-{
- struct timespec ts;
-
- get_monotonic_boottime(&ts);
- return timespec_to_ktime(ts);
-}
-EXPORT_SYMBOL_GPL(ktime_get_boottime);
-
-/**
* monotonic_to_bootbased - Convert the monotonic time to boot based.
* @ts: pointer to the timespec to be converted
*/
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 20/64] timekeeping: Use ktime_t based data for ktime_get_clocktai()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (18 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 19/64] timekeeping; Use ktime_t based data for ktime_get_boottime() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 21/64] timekeeping: Use ktime_t data for ktime_get_update_offsets_now() Thomas Gleixner
` (44 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-based-for-ktime-clocktai.patch --]
[-- Type: text/plain, Size: 1605 bytes --]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 8 insertions(+), 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -109,7 +109,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_monotonic_offset(void);
-extern ktime_t ktime_get_clocktai(void);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -130,6 +129,14 @@ static inline ktime_t ktime_get_boottime
return ktime_get_with_offset(TK_OFFS_BOOT);
}
+/**
+ * ktime_get_clocktai - Returns the TAI time of day in ktime_t format
+ */
+static inline ktime_t ktime_get_clocktai(void)
+{
+ return ktime_get_with_offset(TK_OFFS_TAI);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -512,21 +512,6 @@ void timekeeping_clocktai(struct timespe
}
EXPORT_SYMBOL(timekeeping_clocktai);
-
-/**
- * ktime_get_clocktai - Returns the TAI time of day in a ktime
- *
- * Returns the time of day in a ktime.
- */
-ktime_t ktime_get_clocktai(void)
-{
- struct timespec ts;
-
- timekeeping_clocktai(&ts);
- return timespec_to_ktime(ts);
-}
-EXPORT_SYMBOL(ktime_get_clocktai);
-
#ifdef CONFIG_NTP_PPS
/**
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 21/64] timekeeping: Use ktime_t data for ktime_get_update_offsets_now()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (19 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 20/64] timekeeping: Use ktime_t based data for ktime_get_clocktai() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 22/64] timekeeping; Use ktime based data for ktime_get_update_offsets_tick() Thomas Gleixner
` (43 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-for-hrtimer-if.patch --]
[-- Type: text/plain, Size: 1058 bytes --]
No need to juggle with timespecs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1668,14 +1668,14 @@ ktime_t ktime_get_update_offsets_now(kti
ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
- ktime_t now;
unsigned int seq;
- u64 secs, nsecs;
+ ktime_t base;
+ u64 nsecs;
do {
seq = read_seqcount_begin(&tk_core.seq);
- secs = tk->xtime_sec;
+ base = tk->base_mono;
nsecs = timekeeping_get_ns(tk);
*offs_real = tk->offs_real;
@@ -1683,9 +1683,7 @@ ktime_t ktime_get_update_offsets_now(kti
*offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&tk_core.seq, seq));
- now = ktime_add_ns(ktime_set(secs, 0), nsecs);
- now = ktime_sub(now, *offs_real);
- return now;
+ return ktime_add_ns(base, nsecs);
}
#endif
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 22/64] timekeeping; Use ktime based data for ktime_get_update_offsets_tick()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (20 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 21/64] timekeeping: Use ktime_t data for ktime_get_update_offsets_now() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 23/64] timekeeping: Provide ktime_mono_to_any() Thomas Gleixner
` (42 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-use-ktime-for-hrtimer-if-2.patch --]
[-- Type: text/plain, Size: 1051 bytes --]
No need to juggle with timespecs.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1636,22 +1636,22 @@ ktime_t ktime_get_update_offsets_tick(kt
ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 ts;
- ktime_t now;
unsigned int seq;
+ ktime_t base;
+ u64 nsecs;
do {
seq = read_seqcount_begin(&tk_core.seq);
- ts = tk_xtime(tk);
+ base = tk->base_mono;
+ nsecs = tk->xtime_nsec >> tk->shift;
+
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
*offs_tai = tk->offs_tai;
} while (read_seqcount_retry(&tk_core.seq, seq));
- now = ktime_set(ts.tv_sec, ts.tv_nsec);
- now = ktime_sub(now, *offs_real);
- return now;
+ return ktime_add_ns(base, nsecs);
}
#ifdef CONFIG_HIGH_RES_TIMERS
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 23/64] timekeeping: Provide ktime_mono_to_any()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (21 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 22/64] timekeeping; Use ktime based data for ktime_get_update_offsets_tick() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 24/64] timerfd: Use ktime_mono_to_real() Thomas Gleixner
` (41 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-mono_to-wall.patch --]
[-- Type: text/plain, Size: 1999 bytes --]
ktime based conversion function to map a monotonic time stamp to a
different CLOCK.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 +++++++++
kernel/time/timekeeping.c | 20 ++++++++++++++++++++
2 files changed, 29 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -109,6 +109,7 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_get_monotonic_offset(void);
+extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -137,6 +138,14 @@ static inline ktime_t ktime_get_clocktai
return ktime_get_with_offset(TK_OFFS_TAI);
}
+/**
+ * ktime_mono_to_real - Convert monotonic time to clock realtime
+ */
+static inline ktime_t ktime_mono_to_real(ktime_t mono)
+{
+ return ktime_mono_to_any(mono, TK_OFFS_REAL);
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -451,6 +451,26 @@ ktime_t ktime_get_with_offset(enum tk_of
EXPORT_SYMBOL_GPL(ktime_get_with_offset);
/**
+ * ktime_mono_to_any() - convert mononotic time to any other time
+ * @tmono: time to convert.
+ * @offs: which offset to use
+ */
+ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs)
+{
+ ktime_t *offset = offsets[offs];
+ unsigned long seq;
+ ktime_t tconv;
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ tconv = ktime_add(tmono, *offset);
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return tconv;
+}
+EXPORT_SYMBOL_GPL(ktime_mono_to_any);
+
+/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 24/64] timerfd: Use ktime_mono_to_real()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (22 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 23/64] timekeeping: Provide ktime_mono_to_any() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 25/64] input: evdev: " Thomas Gleixner
` (40 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: fs-timerfd-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 1430 bytes --]
We have a few other use cases of ktime_get_monotonic_offset() which
can be optimized with ktime_mono_to_real(). The timerfd code uses the
offset only for comparison, so we can use ktime_mono_to_real(0) for
this as well.
Funny enough text size shrinks with that on ARM and x8664 !?
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
fs/timerfd.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
Index: tip/fs/timerfd.c
===================================================================
--- tip.orig/fs/timerfd.c
+++ tip/fs/timerfd.c
@@ -92,7 +92,7 @@ static enum alarmtimer_restart timerfd_a
*/
void timerfd_clock_was_set(void)
{
- ktime_t moffs = ktime_get_monotonic_offset();
+ ktime_t moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
struct timerfd_ctx *ctx;
unsigned long flags;
@@ -125,7 +125,7 @@ static bool timerfd_canceled(struct time
{
if (!ctx->might_cancel || ctx->moffs.tv64 != KTIME_MAX)
return false;
- ctx->moffs = ktime_get_monotonic_offset();
+ ctx->moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
return true;
}
@@ -336,7 +336,7 @@ SYSCALL_DEFINE2(timerfd_create, int, clo
else
hrtimer_init(&ctx->t.tmr, clockid, HRTIMER_MODE_ABS);
- ctx->moffs = ktime_get_monotonic_offset();
+ ctx->moffs = ktime_mono_to_real((ktime_t){ .tv64 = 0 });
ufd = anon_inode_getfd("[timerfd]", &timerfd_fops, ctx,
O_RDWR | (flags & TFD_SHARED_FCNTL_FLAGS));
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 25/64] input: evdev: Use ktime_mono_to_real()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (23 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 24/64] timerfd: Use ktime_mono_to_real() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 26/64] drm: " Thomas Gleixner
` (39 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: input-evdev-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 1162 bytes --]
Convert the monotonic timestamp with ktime_mono_to_real() in
evdev_events().
In evdev_queue_syn_dropped() we can call either ktime_get() or
ktime_get_real() depending on the clkid. No point in having two calls
for CLOCK_REALTIME.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
drivers/input/evdev.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Index: tip/drivers/input/evdev.c
===================================================================
--- tip.orig/drivers/input/evdev.c
+++ tip/drivers/input/evdev.c
@@ -108,9 +108,8 @@ static void evdev_queue_syn_dropped(stru
struct input_event ev;
ktime_t time;
- time = ktime_get();
- if (client->clkid != CLOCK_MONOTONIC)
- time = ktime_sub(time, ktime_get_monotonic_offset());
+ time = (client->clkid == CLOCK_MONOTONIC) ?
+ ktime_get() : ktime_get_real();
ev.time = ktime_to_timeval(time);
ev.type = EV_SYN;
@@ -202,7 +201,7 @@ static void evdev_events(struct input_ha
ktime_t time_mono, time_real;
time_mono = ktime_get();
- time_real = ktime_sub(time_mono, ktime_get_monotonic_offset());
+ time_real = ktime_mono_to_real(time_mono);
rcu_read_lock();
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 26/64] drm: Use ktime_mono_to_real()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (24 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 25/64] input: evdev: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 27/64] timekeeping: Remove ktime_get_monotonic_offset() Thomas Gleixner
` (38 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: drm-use-ktime-mono-to-real.patch --]
[-- Type: text/plain, Size: 2123 bytes --]
Convert the monotonic timestamp with ktime_mono_to_real() in
drm_calc_vbltimestamp_from_scanoutpos().
In get_drm_timestamp we can call either ktime_get() or
ktime_get_real() depending on drm_timestamp_monotonic. No point in
having two calls into the core for CLOCK_REALTIME.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
drivers/gpu/drm/drm_irq.c | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
Index: tip/drivers/gpu/drm/drm_irq.c
===================================================================
--- tip.orig/drivers/gpu/drm/drm_irq.c
+++ tip/drivers/gpu/drm/drm_irq.c
@@ -542,8 +542,8 @@ int drm_calc_vbltimestamp_from_scanoutpo
const struct drm_crtc *refcrtc,
const struct drm_display_mode *mode)
{
- ktime_t stime, etime, mono_time_offset;
struct timeval tv_etime;
+ ktime_t stime, etime;
int vbl_status;
int vpos, hpos, i;
int framedur_ns, linedur_ns, pixeldur_ns, delta_ns, duration_ns;
@@ -588,13 +588,6 @@ int drm_calc_vbltimestamp_from_scanoutpo
vbl_status = dev->driver->get_scanout_position(dev, crtc, flags, &vpos,
&hpos, &stime, &etime);
- /*
- * Get correction for CLOCK_MONOTONIC -> CLOCK_REALTIME if
- * CLOCK_REALTIME is requested.
- */
- if (!drm_timestamp_monotonic)
- mono_time_offset = ktime_get_monotonic_offset();
-
/* Return as no-op if scanout query unsupported or failed. */
if (!(vbl_status & DRM_SCANOUTPOS_VALID)) {
DRM_DEBUG("crtc %d : scanoutpos query failed [%d].\n",
@@ -633,7 +626,7 @@ int drm_calc_vbltimestamp_from_scanoutpo
delta_ns = vpos * linedur_ns + hpos * pixeldur_ns;
if (!drm_timestamp_monotonic)
- etime = ktime_sub(etime, mono_time_offset);
+ etime = ktime_mono_to_real(etime);
/* save this only for debugging purposes */
tv_etime = ktime_to_timeval(etime);
@@ -664,10 +657,7 @@ static struct timeval get_drm_timestamp(
{
ktime_t now;
- now = ktime_get();
- if (!drm_timestamp_monotonic)
- now = ktime_sub(now, ktime_get_monotonic_offset());
-
+ now = drm_timestamp_monotonic ? ktime_get() : ktime_get_real();
return ktime_to_timeval(now);
}
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 27/64] timekeeping: Remove ktime_get_monotonic_offset()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (25 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 26/64] drm: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 28/64] timekeeping: Provide ktime_get[*]_ns() helpers Thomas Gleixner
` (37 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-remove-ktime_get_monotonic_offset.patch --]
[-- Type: text/plain, Size: 1468 bytes --]
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 1 -
kernel/time/timekeeping.c | 18 ------------------
2 files changed, 19 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -108,7 +108,6 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
-extern ktime_t ktime_get_monotonic_offset(void);
extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
/**
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1708,24 +1708,6 @@ ktime_t ktime_get_update_offsets_now(kti
#endif
/**
- * ktime_get_monotonic_offset() - get wall_to_monotonic in ktime_t format
- */
-ktime_t ktime_get_monotonic_offset(void)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- unsigned long seq;
- struct timespec64 wtom;
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
- wtom = tk->wall_to_monotonic;
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- return timespec64_to_ktime(wtom);
-}
-EXPORT_SYMBOL_GPL(ktime_get_monotonic_offset);
-
-/**
* do_adjtimex() - Accessor function to NTP __do_adjtimex function
*/
int do_adjtimex(struct timex *txc)
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 28/64] timekeeping: Provide ktime_get[*]_ns() helpers
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (26 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 27/64] timekeeping: Remove ktime_get_monotonic_offset() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 29/64] time: Export nsecs_to_jiffies() Thomas Gleixner
` (36 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-get-ns.patch --]
[-- Type: text/plain, Size: 831 bytes --]
A lot of code converts either timespecs or ktime_t to
nanoseconds. Provide helper functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -145,6 +145,21 @@ static inline ktime_t ktime_mono_to_real
return ktime_mono_to_any(mono, TK_OFFS_REAL);
}
+static inline u64 ktime_get_ns(void)
+{
+ return ktime_to_ns(ktime_get());
+}
+
+static inline u64 ktime_get_real_ns(void)
+{
+ return ktime_to_ns(ktime_get_real());
+}
+
+static inline u64 ktime_get_boot_ns(void)
+{
+ return ktime_to_ns(ktime_get_boottime());
+}
+
/*
* RTC specific
*/
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 29/64] time: Export nsecs_to_jiffies()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (27 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 28/64] timekeeping: Provide ktime_get[*]_ns() helpers Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 30/64] sched: Make task->real_start_time nanoseconds based Thomas Gleixner
` (35 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: time-export-nsecs-to-jiffies.patch --]
[-- Type: text/plain, Size: 561 bytes --]
Required for moving drivers to the nanosecond based interfaces.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/time.c | 1 +
1 file changed, 1 insertion(+)
Index: tip/kernel/time/time.c
===================================================================
--- tip.orig/kernel/time/time.c
+++ tip/kernel/time/time.c
@@ -757,6 +757,7 @@ unsigned long nsecs_to_jiffies(u64 n)
{
return (unsigned long)nsecs_to_jiffies64(n);
}
+EXPORT_SYMBOL_GPL(nsecs_to_jiffies);
/*
* Add two timespec values and do a safety check for overflow.
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 30/64] sched: Make task->real_start_time nanoseconds based
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (28 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 29/64] time: Export nsecs_to_jiffies() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 31/64] sched: Make task->start_time " Thomas Gleixner
` (34 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: sched-make-real-start-time-ktime-t.patch --]
[-- Type: text/plain, Size: 2040 bytes --]
Simplify the only user of this data by removing the timespec
conversion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
fs/proc/array.c | 7 +------
include/linux/sched.h | 2 +-
kernel/fork.c | 3 +--
3 files changed, 3 insertions(+), 9 deletions(-)
Index: tip/fs/proc/array.c
===================================================================
--- tip.orig/fs/proc/array.c
+++ tip/fs/proc/array.c
@@ -473,13 +473,8 @@ static int do_task_stat(struct seq_file
priority = task_prio(task);
nice = task_nice(task);
- /* Temporary variable needed for gcc-2.96 */
- /* convert timespec -> nsec*/
- start_time =
- (unsigned long long)task->real_start_time.tv_sec * NSEC_PER_SEC
- + task->real_start_time.tv_nsec;
/* convert nsec -> ticks */
- start_time = nsec_to_clock_t(start_time);
+ start_time = nsec_to_clock_t(task->real_start_time);
seq_printf(m, "%d (%s) %c", pid_nr_ns(pid, ns), tcomm, state);
seq_put_decimal_ll(m, ' ', ppid);
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -1368,7 +1368,7 @@ struct task_struct {
#endif
unsigned long nvcsw, nivcsw; /* context switch counts */
struct timespec start_time; /* monotonic time */
- struct timespec real_start_time; /* boot based time */
+ u64 real_start_time; /* boot based time in nsec */
/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */
unsigned long min_flt, maj_flt;
Index: tip/kernel/fork.c
===================================================================
--- tip.orig/kernel/fork.c
+++ tip/kernel/fork.c
@@ -1263,8 +1263,7 @@ static struct task_struct *copy_process(
posix_cpu_timers_init(p);
ktime_get_ts(&p->start_time);
- p->real_start_time = p->start_time;
- monotonic_to_bootbased(&p->real_start_time);
+ p->real_start_time = ktime_get_boot_ns();
p->io_context = NULL;
p->audit_context = NULL;
if (clone_flags & CLONE_THREAD)
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 31/64] sched: Make task->start_time nanoseconds based
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (29 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 30/64] sched: Make task->real_start_time nanoseconds based Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 32/64] delayacct: Make accounting nanosecond based Thomas Gleixner
` (33 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: sched-make-start-time-nsec.patch --]
[-- Type: text/plain, Size: 3419 bytes --]
Simplify the timespec to nsec/usec conversions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/sched.h | 2 +-
kernel/acct.c | 10 +++-------
kernel/fork.c | 2 +-
kernel/tsacct.c | 19 +++++++++----------
4 files changed, 14 insertions(+), 19 deletions(-)
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -1367,7 +1367,7 @@ struct task_struct {
} vtime_snap_whence;
#endif
unsigned long nvcsw, nivcsw; /* context switch counts */
- struct timespec start_time; /* monotonic time */
+ u64 start_time; /* monotonic time in nsec */
u64 real_start_time; /* boot based time in nsec */
/* mm fault and swap info: this can arguably be seen as either mm-specific or thread-specific */
unsigned long min_flt, maj_flt;
Index: tip/kernel/acct.c
===================================================================
--- tip.orig/kernel/acct.c
+++ tip/kernel/acct.c
@@ -458,9 +458,7 @@ static void do_acct_process(struct bsd_a
acct_t ac;
mm_segment_t fs;
unsigned long flim;
- u64 elapsed;
- u64 run_time;
- struct timespec uptime;
+ u64 elapsed, run_time;
struct tty_struct *tty;
const struct cred *orig_cred;
@@ -484,10 +482,8 @@ static void do_acct_process(struct bsd_a
strlcpy(ac.ac_comm, current->comm, sizeof(ac.ac_comm));
/* calculate run_time in nsec*/
- ktime_get_ts(&uptime);
- run_time = (u64)uptime.tv_sec*NSEC_PER_SEC + uptime.tv_nsec;
- run_time -= (u64)current->group_leader->start_time.tv_sec * NSEC_PER_SEC
- + current->group_leader->start_time.tv_nsec;
+ run_time = ktime_get_ns();
+ run_time -= current->group_leader->start_time;
/* convert nsec -> AHZ */
elapsed = nsec_to_AHZ(run_time);
#if ACCT_VERSION==3
Index: tip/kernel/fork.c
===================================================================
--- tip.orig/kernel/fork.c
+++ tip/kernel/fork.c
@@ -1262,7 +1262,7 @@ static struct task_struct *copy_process(
posix_cpu_timers_init(p);
- ktime_get_ts(&p->start_time);
+ p->start_time = ktime_get_ns();
p->real_start_time = ktime_get_boot_ns();
p->io_context = NULL;
p->audit_context = NULL;
Index: tip/kernel/tsacct.c
===================================================================
--- tip.orig/kernel/tsacct.c
+++ tip/kernel/tsacct.c
@@ -31,20 +31,19 @@ void bacct_add_tsk(struct user_namespace
struct taskstats *stats, struct task_struct *tsk)
{
const struct cred *tcred;
- struct timespec uptime, ts;
cputime_t utime, stime, utimescaled, stimescaled;
- u64 ac_etime;
+ u64 delta;
BUILD_BUG_ON(TS_COMM_LEN < TASK_COMM_LEN);
- /* calculate task elapsed time in timespec */
- ktime_get_ts(&uptime);
- ts = timespec_sub(uptime, tsk->start_time);
- /* rebase elapsed time to usec (should never be negative) */
- ac_etime = timespec_to_ns(&ts);
- do_div(ac_etime, NSEC_PER_USEC);
- stats->ac_etime = ac_etime;
- stats->ac_btime = get_seconds() - ts.tv_sec;
+ /* calculate task elapsed time in nsec */
+ delta = ktime_get_ns() - tsk->start_time;
+ /* Convert to micro seconds */
+ do_div(delta, NSEC_PER_USEC);
+ stats->ac_etime = delta;
+ /* Convert to seconds for btime */
+ do_div(delta, USEC_PER_SEC);
+ stats->ac_btime = get_seconds() - delta;
if (thread_group_leader(tsk)) {
stats->ac_exitcode = tsk->exit_code;
if (tsk->flags & PF_FORKNOEXEC)
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 32/64] delayacct: Make accounting nanosecond based
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (30 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 31/64] sched: Make task->start_time " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 33/64] delayacct: Remove braindamaged type conversions Thomas Gleixner
` (32 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: delayacct-use-ktimet.patch --]
[-- Type: text/plain, Size: 3477 bytes --]
Kill the timespec juggling and calculate with plain nanoseconds.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/sched.h | 4 ++--
kernel/delayacct.c | 34 ++++++++++++----------------------
2 files changed, 14 insertions(+), 24 deletions(-)
Index: tip/include/linux/sched.h
===================================================================
--- tip.orig/include/linux/sched.h
+++ tip/include/linux/sched.h
@@ -813,7 +813,7 @@ struct task_delay_info {
* associated with the operation is added to XXX_delay.
* XXX_delay contains the accumulated delay time in nanoseconds.
*/
- struct timespec blkio_start, blkio_end; /* Shared by blkio, swapin */
+ u64 blkio_start; /* Shared by blkio, swapin */
u64 blkio_delay; /* wait for sync block io completion */
u64 swapin_delay; /* wait for swapin block io completion */
u32 blkio_count; /* total count of the number of sync block */
@@ -821,7 +821,7 @@ struct task_delay_info {
u32 swapin_count; /* total count of the number of swapin block */
/* io operations performed */
- struct timespec freepages_start, freepages_end;
+ u64 freepages_start;
u64 freepages_delay; /* wait for memory reclaim */
u32 freepages_count; /* total count of memory reclaim */
};
Index: tip/kernel/delayacct.c
===================================================================
--- tip.orig/kernel/delayacct.c
+++ tip/kernel/delayacct.c
@@ -46,32 +46,25 @@ void __delayacct_tsk_init(struct task_st
}
/*
- * Finish delay accounting for a statistic using
- * its timestamps (@start, @end), accumalator (@total) and @count
+ * Finish delay accounting for a statistic using its timestamps (@start),
+ * accumalator (@total) and @count
*/
-
-static void delayacct_end(struct timespec *start, struct timespec *end,
- u64 *total, u32 *count)
+static void delayacct_end(u64 *start, u64 *total, u32 *count)
{
- struct timespec ts;
- s64 ns;
+ s64 ns = ktime_get_ns() - *start;
unsigned long flags;
- ktime_get_ts(end);
- ts = timespec_sub(*end, *start);
- ns = timespec_to_ns(&ts);
- if (ns < 0)
- return;
-
- spin_lock_irqsave(¤t->delays->lock, flags);
- *total += ns;
- (*count)++;
- spin_unlock_irqrestore(¤t->delays->lock, flags);
+ if (ns > 0) {
+ spin_lock_irqsave(¤t->delays->lock, flags);
+ *total += ns;
+ (*count)++;
+ spin_unlock_irqrestore(¤t->delays->lock, flags);
+ }
}
void __delayacct_blkio_start(void)
{
- ktime_get_ts(¤t->delays->blkio_start);
+ current->delays->blkio_start = ktime_get_ns();
}
void __delayacct_blkio_end(void)
@@ -79,12 +72,10 @@ void __delayacct_blkio_end(void)
if (current->delays->flags & DELAYACCT_PF_SWAPIN)
/* Swapin block I/O */
delayacct_end(¤t->delays->blkio_start,
- ¤t->delays->blkio_end,
¤t->delays->swapin_delay,
¤t->delays->swapin_count);
else /* Other block I/O */
delayacct_end(¤t->delays->blkio_start,
- ¤t->delays->blkio_end,
¤t->delays->blkio_delay,
¤t->delays->blkio_count);
}
@@ -159,13 +150,12 @@ __u64 __delayacct_blkio_ticks(struct tas
void __delayacct_freepages_start(void)
{
- ktime_get_ts(¤t->delays->freepages_start);
+ current->delays->freepages_start = ktime_get_ns();
}
void __delayacct_freepages_end(void)
{
delayacct_end(¤t->delays->freepages_start,
- ¤t->delays->freepages_end,
¤t->delays->freepages_delay,
¤t->delays->freepages_count);
}
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 33/64] delayacct: Remove braindamaged type conversions
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (31 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 32/64] delayacct: Make accounting nanosecond based Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 34/64] powerpc: cell: Use ktime_get_ns() Thomas Gleixner
` (31 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: delayacct-simplify.patch --]
[-- Type: text/plain, Size: 1454 bytes --]
Converting cputime to timespec and timespec to nanoseconds makes no
sense. Use cputime_to_ns() and be done with it.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/delayacct.c | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
Index: tip/kernel/delayacct.c
===================================================================
--- tip.orig/kernel/delayacct.c
+++ tip/kernel/delayacct.c
@@ -82,23 +82,19 @@ void __delayacct_blkio_end(void)
int __delayacct_add_tsk(struct taskstats *d, struct task_struct *tsk)
{
- s64 tmp;
- unsigned long t1;
- unsigned long long t2, t3;
- unsigned long flags;
- struct timespec ts;
cputime_t utime, stime, stimescaled, utimescaled;
+ unsigned long long t2, t3;
+ unsigned long flags, t1;
+ s64 tmp;
- tmp = (s64)d->cpu_run_real_total;
task_cputime(tsk, &utime, &stime);
- cputime_to_timespec(utime + stime, &ts);
- tmp += timespec_to_ns(&ts);
+ tmp = (s64)d->cpu_run_real_total;
+ tmp += cputime_to_nsecs(utime + stime);
d->cpu_run_real_total = (tmp < (s64)d->cpu_run_real_total) ? 0 : tmp;
- tmp = (s64)d->cpu_scaled_run_real_total;
task_cputime_scaled(tsk, &utimescaled, &stimescaled);
- cputime_to_timespec(utimescaled + stimescaled, &ts);
- tmp += timespec_to_ns(&ts);
+ tmp = (s64)d->cpu_scaled_run_real_total;
+ tmp += cputime_to_nsecs(utimescaled + stimescaled);
d->cpu_scaled_run_real_total =
(tmp < (s64)d->cpu_scaled_run_real_total) ? 0 : tmp;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 34/64] powerpc: cell: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (32 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 33/64] delayacct: Remove braindamaged type conversions Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 35/64] connector: " Thomas Gleixner
` (30 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Arnd Bergmann
[-- Attachment #1: powerpc-cell-remove-silly-timespec-dance.patch --]
[-- Type: text/plain, Size: 4016 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
---
arch/powerpc/platforms/cell/spu_base.c | 11 +++--------
arch/powerpc/platforms/cell/spufs/context.c | 4 +---
arch/powerpc/platforms/cell/spufs/file.c | 4 +---
arch/powerpc/platforms/cell/spufs/sched.c | 4 +---
4 files changed, 6 insertions(+), 17 deletions(-)
Index: tip/arch/powerpc/platforms/cell/spu_base.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spu_base.c
+++ tip/arch/powerpc/platforms/cell/spu_base.c
@@ -611,7 +611,6 @@ static int __init create_spu(void *data)
int ret;
static int number;
unsigned long flags;
- struct timespec ts;
ret = -ENOMEM;
spu = kzalloc(sizeof (*spu), GFP_KERNEL);
@@ -652,8 +651,7 @@ static int __init create_spu(void *data)
mutex_unlock(&spu_full_list_mutex);
spu->stats.util_state = SPU_UTIL_IDLE_LOADED;
- ktime_get_ts(&ts);
- spu->stats.tstamp = timespec_to_ns(&ts);
+ spu->stats.tstamp = ktime_get_ns();
INIT_LIST_HEAD(&spu->aff_list);
@@ -676,7 +674,6 @@ static const char *spu_state_names[] = {
static unsigned long long spu_acct_time(struct spu *spu,
enum spu_utilization_state state)
{
- struct timespec ts;
unsigned long long time = spu->stats.times[state];
/*
@@ -684,10 +681,8 @@ static unsigned long long spu_acct_time(
* statistics are not updated. Apply the time delta from the
* last recorded state of the spu.
*/
- if (spu->stats.util_state == state) {
- ktime_get_ts(&ts);
- time += timespec_to_ns(&ts) - spu->stats.tstamp;
- }
+ if (spu->stats.util_state == state)
+ time += ktime_get_ns() - spu->stats.tstamp;
return time / NSEC_PER_MSEC;
}
Index: tip/arch/powerpc/platforms/cell/spufs/context.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/context.c
+++ tip/arch/powerpc/platforms/cell/spufs/context.c
@@ -36,7 +36,6 @@ atomic_t nr_spu_contexts = ATOMIC_INIT(0
struct spu_context *alloc_spu_context(struct spu_gang *gang)
{
struct spu_context *ctx;
- struct timespec ts;
ctx = kzalloc(sizeof *ctx, GFP_KERNEL);
if (!ctx)
@@ -67,8 +66,7 @@ struct spu_context *alloc_spu_context(st
__spu_update_sched_info(ctx);
spu_set_timeslice(ctx);
ctx->stats.util_state = SPU_UTIL_IDLE_LOADED;
- ktime_get_ts(&ts);
- ctx->stats.tstamp = timespec_to_ns(&ts);
+ ctx->stats.tstamp = ktime_get_ns();
atomic_inc(&nr_spu_contexts);
goto out;
Index: tip/arch/powerpc/platforms/cell/spufs/file.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/file.c
+++ tip/arch/powerpc/platforms/cell/spufs/file.c
@@ -2338,7 +2338,6 @@ static const char *ctx_state_names[] = {
static unsigned long long spufs_acct_time(struct spu_context *ctx,
enum spu_utilization_state state)
{
- struct timespec ts;
unsigned long long time = ctx->stats.times[state];
/*
@@ -2351,8 +2350,7 @@ static unsigned long long spufs_acct_tim
* of the spu context.
*/
if (ctx->spu && ctx->stats.util_state == state) {
- ktime_get_ts(&ts);
- time += timespec_to_ns(&ts) - ctx->stats.tstamp;
+ time += ktime_get_ns() - ctx->stats.tstamp;
}
return time / NSEC_PER_MSEC;
Index: tip/arch/powerpc/platforms/cell/spufs/sched.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/spufs/sched.c
+++ tip/arch/powerpc/platforms/cell/spufs/sched.c
@@ -1039,13 +1039,11 @@ void spuctx_switch_state(struct spu_cont
{
unsigned long long curtime;
signed long long delta;
- struct timespec ts;
struct spu *spu;
enum spu_utilization_state old_state;
int node;
- ktime_get_ts(&ts);
- curtime = timespec_to_ns(&ts);
+ curtime = ktime_get_ns();
delta = curtime - ctx->stats.tstamp;
WARN_ON(!mutex_is_locked(&ctx->state_mutex));
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 35/64] connector: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (33 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 34/64] powerpc: cell: Use ktime_get_ns() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:06 ` Evgeniy Polyakov
2014-07-16 21:04 ` [patch V2 36/64] mfd: cros_ec_spi: " Thomas Gleixner
` (29 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Evgeniy Polyakov
[-- Attachment #1: connector-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 6565 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Evgeniy Polyakov <zbr@ioremap.net>
---
drivers/connector/cn_proc.c | 36 +++++++++---------------------------
1 file changed, 9 insertions(+), 27 deletions(-)
Index: tip/drivers/connector/cn_proc.c
===================================================================
--- tip.orig/drivers/connector/cn_proc.c
+++ tip/drivers/connector/cn_proc.c
@@ -69,7 +69,6 @@ void proc_fork_connector(struct task_str
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
struct task_struct *parent;
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -79,8 +78,7 @@ void proc_fork_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_FORK;
rcu_read_lock();
parent = rcu_dereference(task->real_parent);
@@ -102,7 +100,6 @@ void proc_exec_connector(struct task_str
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -112,8 +109,7 @@ void proc_exec_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_EXEC;
ev->event_data.exec.process_pid = task->pid;
ev->event_data.exec.process_tgid = task->tgid;
@@ -130,7 +126,6 @@ void proc_id_connector(struct task_struc
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
const struct cred *cred;
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -156,8 +151,7 @@ void proc_id_connector(struct task_struc
}
rcu_read_unlock();
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
memcpy(&msg->id, &cn_proc_event_id, sizeof(msg->id));
msg->ack = 0; /* not used */
@@ -170,7 +164,6 @@ void proc_sid_connector(struct task_stru
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -180,8 +173,7 @@ void proc_sid_connector(struct task_stru
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_SID;
ev->event_data.sid.process_pid = task->pid;
ev->event_data.sid.process_tgid = task->tgid;
@@ -197,7 +189,6 @@ void proc_ptrace_connector(struct task_s
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -207,8 +198,7 @@ void proc_ptrace_connector(struct task_s
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_PTRACE;
ev->event_data.ptrace.process_pid = task->pid;
ev->event_data.ptrace.process_tgid = task->tgid;
@@ -232,7 +222,6 @@ void proc_comm_connector(struct task_str
{
struct cn_msg *msg;
struct proc_event *ev;
- struct timespec ts;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
if (atomic_read(&proc_event_num_listeners) < 1)
@@ -242,8 +231,7 @@ void proc_comm_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_COMM;
ev->event_data.comm.process_pid = task->pid;
ev->event_data.comm.process_tgid = task->tgid;
@@ -261,7 +249,6 @@ void proc_coredump_connector(struct task
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -270,8 +257,7 @@ void proc_coredump_connector(struct task
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_COREDUMP;
ev->event_data.coredump.process_pid = task->pid;
ev->event_data.coredump.process_tgid = task->tgid;
@@ -288,7 +274,6 @@ void proc_exit_connector(struct task_str
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -297,8 +282,7 @@ void proc_exit_connector(struct task_str
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
get_seq(&msg->seq, &ev->cpu);
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->what = PROC_EVENT_EXIT;
ev->event_data.exit.process_pid = task->pid;
ev->event_data.exit.process_tgid = task->tgid;
@@ -325,7 +309,6 @@ static void cn_proc_ack(int err, int rcv
struct cn_msg *msg;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE] __aligned(8);
- struct timespec ts;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
@@ -334,8 +317,7 @@ static void cn_proc_ack(int err, int rcv
ev = (struct proc_event *)msg->data;
memset(&ev->event_data, 0, sizeof(ev->event_data));
msg->seq = rcvd_seq;
- ktime_get_ts(&ts); /* get high res monotonic timestamp */
- ev->timestamp_ns = timespec_to_ns(&ts);
+ ev->timestamp_ns = ktime_get_ns();
ev->cpu = -1;
ev->what = PROC_EVENT_NONE;
ev->event_data.ack.err = err;
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 35/64] connector: Use ktime_get_ns()
2014-07-16 21:04 ` [patch V2 35/64] connector: " Thomas Gleixner
@ 2014-07-16 21:06 ` Evgeniy Polyakov
0 siblings, 0 replies; 76+ messages in thread
From: Evgeniy Polyakov @ 2014-07-16 21:06 UTC (permalink / raw)
To: Thomas Gleixner, LKML; +Cc: John Stultz, Peter Zijlstra
Hi
17.07.2014, 01:04, "Thomas Gleixner" <tglx@linutronix.de>:
> Replace the ever recurring:
> ts = ktime_get_ts();
> ns = timespec_to_ns(&ts);
> with
> ns = ktime_get_ns();
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Evgeniy Polyakov <zbr@ioremap.net>
Looks good to me, thank you.
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 36/64] mfd: cros_ec_spi: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (34 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 35/64] connector: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 37/64] misc: ioc4: " Thomas Gleixner
` (28 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Lee Jones
[-- Attachment #1: mfd-cros-ec-spi-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 1426 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Lee Jones <lee.jones@linaro.org>
---
drivers/mfd/cros_ec_spi.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
Index: tip/drivers/mfd/cros_ec_spi.c
===================================================================
--- tip.orig/drivers/mfd/cros_ec_spi.c
+++ tip/drivers/mfd/cros_ec_spi.c
@@ -225,7 +225,6 @@ static int cros_ec_command_spi_xfer(stru
u8 *ptr;
int sum;
int ret = 0, final_ret;
- struct timespec ts;
/*
* We have the shared ec_dev buffer plus we do lots of separate spi_sync
@@ -239,11 +238,9 @@ static int cros_ec_command_spi_xfer(stru
/* If it's too soon to do another transaction, wait */
if (ec_spi->last_transfer_ns) {
- struct timespec ts;
unsigned long delay; /* The delay completed so far */
- ktime_get_ts(&ts);
- delay = timespec_to_ns(&ts) - ec_spi->last_transfer_ns;
+ delay = ktime_get_ns() - ec_spi->last_transfer_ns;
if (delay < EC_SPI_RECOVERY_TIME_NS)
ndelay(EC_SPI_RECOVERY_TIME_NS - delay);
}
@@ -280,8 +277,7 @@ static int cros_ec_command_spi_xfer(stru
}
final_ret = spi_sync(ec_spi->spi, &msg);
- ktime_get_ts(&ts);
- ec_spi->last_transfer_ns = timespec_to_ns(&ts);
+ ec_spi->last_transfer_ns = ktime_get_ns();
if (!ret)
ret = final_ret;
if (ret < 0) {
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 37/64] misc: ioc4: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (35 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 36/64] mfd: cros_ec_spi: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 38/64] net: mlx5: " Thomas Gleixner
` (27 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Arnd Bergmann, Greg Kroah-Hartman
[-- Attachment #1: misc-ioc4-remove-silly-time-stamp-dance.patch --]
[-- Type: text/plain, Size: 1517 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/misc/ioc4.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Index: tip/drivers/misc/ioc4.c
===================================================================
--- tip.orig/drivers/misc/ioc4.c
+++ tip/drivers/misc/ioc4.c
@@ -145,7 +145,6 @@ ioc4_clock_calibrate(struct ioc4_driver_
union ioc4_int_out int_out;
union ioc4_gpcr gpcr;
unsigned int state, last_state = 1;
- struct timespec start_ts, end_ts;
uint64_t start, end, period;
unsigned int count = 0;
@@ -174,10 +173,10 @@ ioc4_clock_calibrate(struct ioc4_driver_
if (!last_state && state) {
count++;
if (count == IOC4_CALIBRATE_END) {
- ktime_get_ts(&end_ts);
+ end = ktime_get_ns();
break;
} else if (count == IOC4_CALIBRATE_DISCARD)
- ktime_get_ts(&start_ts);
+ start = ktime_get_ns();
}
last_state = state;
} while (1);
@@ -192,8 +191,6 @@ ioc4_clock_calibrate(struct ioc4_driver_
* by which the IOC4 generates the square wave, to get the
* period of an IOC4 INT_OUT count.
*/
- end = end_ts.tv_sec * NSEC_PER_SEC + end_ts.tv_nsec;
- start = start_ts.tv_sec * NSEC_PER_SEC + start_ts.tv_nsec;
period = (end - start) /
(IOC4_CALIBRATE_CYCLES * 2 * (IOC4_CALIBRATE_COUNT + 1));
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 38/64] net: mlx5: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (36 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 37/64] misc: ioc4: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 39/64] fs: lockd: " Thomas Gleixner
` (26 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Eli Cohen
[-- Attachment #1: net-mlx5-remove-silly-timespec-dance.patch --]
[-- Type: text/plain, Size: 2977 bytes --]
This code is beyond silly:
struct timespec ts = ktime_get_ts();
ktime_t ktime = timespec_to_ktime(ts);
Further down the code builds the delta of two ktime_t values and
converts the result to nanoseconds.
Use ktime_get_ns() and replace all the nonsense.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Eli Cohen <eli@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 16 ++++------------
include/linux/mlx5/driver.h | 4 ++--
2 files changed, 6 insertions(+), 14 deletions(-)
Index: tip/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
===================================================================
--- tip.orig/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ tip/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -548,7 +548,7 @@ static void cmd_work_handler(struct work
lay->status_own = CMD_OWNER_HW;
set_signature(ent, !cmd->checksum_disabled);
dump_command(dev, ent, 1);
- ktime_get_ts(&ent->ts1);
+ ent->ts1 = ktime_get_ns();
/* ring doorbell after the descriptor is valid */
wmb();
@@ -637,7 +637,6 @@ static int mlx5_cmd_invoke(struct mlx5_c
{
struct mlx5_cmd *cmd = &dev->cmd;
struct mlx5_cmd_work_ent *ent;
- ktime_t t1, t2, delta;
struct mlx5_cmd_stats *stats;
int err = 0;
s64 ds;
@@ -668,10 +667,7 @@ static int mlx5_cmd_invoke(struct mlx5_c
if (err == -ETIMEDOUT)
goto out;
- t1 = timespec_to_ktime(ent->ts1);
- t2 = timespec_to_ktime(ent->ts2);
- delta = ktime_sub(t2, t1);
- ds = ktime_to_ns(delta);
+ ds = ent->ts2 - ent->ts1;
op = be16_to_cpu(((struct mlx5_inbox_hdr *)in->first.data)->opcode);
if (op < ARRAY_SIZE(cmd->stats)) {
stats = &cmd->stats[op];
@@ -1135,7 +1131,6 @@ void mlx5_cmd_comp_handler(struct mlx5_c
void *context;
int err;
int i;
- ktime_t t1, t2, delta;
s64 ds;
struct mlx5_cmd_stats *stats;
unsigned long flags;
@@ -1149,7 +1144,7 @@ void mlx5_cmd_comp_handler(struct mlx5_c
sem = &cmd->pages_sem;
else
sem = &cmd->sem;
- ktime_get_ts(&ent->ts2);
+ ent->ts2 = ktime_get_ns();
memcpy(ent->out->first.data, ent->lay->out, sizeof(ent->lay->out));
dump_command(dev, ent, 0);
if (!ent->ret) {
@@ -1163,10 +1158,7 @@ void mlx5_cmd_comp_handler(struct mlx5_c
}
free_ent(cmd, ent->idx);
if (ent->callback) {
- t1 = timespec_to_ktime(ent->ts1);
- t2 = timespec_to_ktime(ent->ts2);
- delta = ktime_sub(t2, t1);
- ds = ktime_to_ns(delta);
+ ds = ent->ts2 - ent->ts1;
if (ent->op < ARRAY_SIZE(cmd->stats)) {
stats = &cmd->stats[ent->op];
spin_lock_irqsave(&stats->lock, flags);
Index: tip/include/linux/mlx5/driver.h
===================================================================
--- tip.orig/include/linux/mlx5/driver.h
+++ tip/include/linux/mlx5/driver.h
@@ -604,8 +604,8 @@ struct mlx5_cmd_work_ent {
int page_queue;
u8 status;
u8 token;
- struct timespec ts1;
- struct timespec ts2;
+ u64 ts1;
+ u64 ts2;
u16 op;
};
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 39/64] fs: lockd: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (37 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 38/64] net: mlx5: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 40/64] hwmon: ibmaem: " Thomas Gleixner
` (25 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Trond Myklebust, J. Bruce Fields
[-- Attachment #1: fs-lockd-remove-silly-timestamp-dance.patch --]
[-- Type: text/plain, Size: 845 bytes --]
Replace the ever recurring:
ts = ktime_get_ts();
ns = timespec_to_ns(&ts);
with
ns = ktime_get_ns();
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Trond Myklebust <trond.myklebust@primarydata.com>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
---
fs/lockd/mon.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
Index: tip/fs/lockd/mon.c
===================================================================
--- tip.orig/fs/lockd/mon.c
+++ tip/fs/lockd/mon.c
@@ -306,11 +306,9 @@ static struct nsm_handle *nsm_lookup_pri
static void nsm_init_private(struct nsm_handle *nsm)
{
u64 *p = (u64 *)&nsm->sm_priv.data;
- struct timespec ts;
s64 ns;
- ktime_get_ts(&ts);
- ns = timespec_to_ns(&ts);
+ ns = ktime_get_ns();
put_unaligned(ns, p);
put_unaligned((unsigned long)nsm, p + 1);
}
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 40/64] hwmon: ibmaem: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (38 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 39/64] fs: lockd: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-21 8:37 ` Jean Delvare
2014-07-16 21:04 ` [patch V2 41/64] iio: Use ktime_get_real_ns() Thomas Gleixner
` (24 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Jean Delvare
[-- Attachment #1: hwmon-ibmaem-use-ktime.patch --]
[-- Type: text/plain, Size: 1395 bytes --]
Using the wall clock time for delta time calculations is wrong to
begin with because wall clock time can be set from userspace and NTP.
Such data wants to be based on clock monotonic.
The calcuations also are done on a nanosecond basis. Use the
nanoseconds based interface right away.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jean Delvare <jdelvare@suse.de>
---
drivers/hwmon/ibmaem.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
Index: tip/drivers/hwmon/ibmaem.c
===================================================================
--- tip.orig/drivers/hwmon/ibmaem.c
+++ tip/drivers/hwmon/ibmaem.c
@@ -842,11 +842,10 @@ static ssize_t aem_show_power(struct dev
struct aem_data *data = dev_get_drvdata(dev);
u64 before, after, delta, time;
signed long leftover;
- struct timespec b, a;
mutex_lock(&data->lock);
update_aem_energy_one(data, attr->index);
- getnstimeofday(&b);
+ time = ktime_get_ns();
before = data->energy[attr->index];
leftover = schedule_timeout_interruptible(
@@ -858,11 +857,10 @@ static ssize_t aem_show_power(struct dev
}
update_aem_energy_one(data, attr->index);
- getnstimeofday(&a);
+ time = ktime_get_ns() - time;
after = data->energy[attr->index];
mutex_unlock(&data->lock);
- time = timespec_to_ns(&a) - timespec_to_ns(&b);
delta = (after - before) * UJ_PER_MJ;
return sprintf(buf, "%llu\n",
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 40/64] hwmon: ibmaem: Use ktime_get_ns()
2014-07-16 21:04 ` [patch V2 40/64] hwmon: ibmaem: " Thomas Gleixner
@ 2014-07-21 8:37 ` Jean Delvare
2014-07-21 21:38 ` Darrick J. Wong
0 siblings, 1 reply; 76+ messages in thread
From: Jean Delvare @ 2014-07-21 8:37 UTC (permalink / raw)
To: Thomas Gleixner, Darrick J. Wong; +Cc: LKML, John Stultz, Peter Zijlstra
Hi Thomas,
Le Wednesday 16 July 2014 à 21:04 +0000, Thomas Gleixner a écrit :
> Using the wall clock time for delta time calculations is wrong to
> begin with because wall clock time can be set from userspace and NTP.
> Such data wants to be based on clock monotonic.
>
> The calcuations also are done on a nanosecond basis. Use the
Typo: calculations
> nanoseconds based interface right away.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Jean Delvare <jdelvare@suse.de>
> ---
> drivers/hwmon/ibmaem.c | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> Index: tip/drivers/hwmon/ibmaem.c
> ===================================================================
> --- tip.orig/drivers/hwmon/ibmaem.c
> +++ tip/drivers/hwmon/ibmaem.c
> @@ -842,11 +842,10 @@ static ssize_t aem_show_power(struct dev
> struct aem_data *data = dev_get_drvdata(dev);
> u64 before, after, delta, time;
> signed long leftover;
> - struct timespec b, a;
>
> mutex_lock(&data->lock);
> update_aem_energy_one(data, attr->index);
> - getnstimeofday(&b);
> + time = ktime_get_ns();
> before = data->energy[attr->index];
>
> leftover = schedule_timeout_interruptible(
> @@ -858,11 +857,10 @@ static ssize_t aem_show_power(struct dev
> }
>
> update_aem_energy_one(data, attr->index);
> - getnstimeofday(&a);
> + time = ktime_get_ns() - time;
> after = data->energy[attr->index];
> mutex_unlock(&data->lock);
>
> - time = timespec_to_ns(&a) - timespec_to_ns(&b);
> delta = (after - before) * UJ_PER_MJ;
>
> return sprintf(buf, "%llu\n",
I'm not familiar with the driver and I can't test it, and I'm not
familiar with kernel timekeeping either. All I can say is that the
changes look reasonable and good.
Acked-by: Jean Delvare <jdelvare@suse.de>
Darrick, maybe you want to comment or test?
--
Jean Delvare
SUSE L3 Support
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [patch V2 40/64] hwmon: ibmaem: Use ktime_get_ns()
2014-07-21 8:37 ` Jean Delvare
@ 2014-07-21 21:38 ` Darrick J. Wong
0 siblings, 0 replies; 76+ messages in thread
From: Darrick J. Wong @ 2014-07-21 21:38 UTC (permalink / raw)
To: Jean Delvare; +Cc: Thomas Gleixner, LKML, John Stultz, Peter Zijlstra
On Mon, Jul 21, 2014 at 10:37:38AM +0200, Jean Delvare wrote:
> Hi Thomas,
>
> Le Wednesday 16 July 2014 à 21:04 +0000, Thomas Gleixner a écrit :
> > Using the wall clock time for delta time calculations is wrong to
> > begin with because wall clock time can be set from userspace and NTP.
> > Such data wants to be based on clock monotonic.
> >
> > The calcuations also are done on a nanosecond basis. Use the
>
> Typo: calculations
>
> > nanoseconds based interface right away.
> >
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Jean Delvare <jdelvare@suse.de>
> > ---
> > drivers/hwmon/ibmaem.c | 6 ++----
> > 1 file changed, 2 insertions(+), 4 deletions(-)
> >
> > Index: tip/drivers/hwmon/ibmaem.c
> > ===================================================================
> > --- tip.orig/drivers/hwmon/ibmaem.c
> > +++ tip/drivers/hwmon/ibmaem.c
> > @@ -842,11 +842,10 @@ static ssize_t aem_show_power(struct dev
> > struct aem_data *data = dev_get_drvdata(dev);
> > u64 before, after, delta, time;
> > signed long leftover;
> > - struct timespec b, a;
> >
> > mutex_lock(&data->lock);
> > update_aem_energy_one(data, attr->index);
> > - getnstimeofday(&b);
> > + time = ktime_get_ns();
> > before = data->energy[attr->index];
> >
> > leftover = schedule_timeout_interruptible(
> > @@ -858,11 +857,10 @@ static ssize_t aem_show_power(struct dev
> > }
> >
> > update_aem_energy_one(data, attr->index);
> > - getnstimeofday(&a);
> > + time = ktime_get_ns() - time;
> > after = data->energy[attr->index];
> > mutex_unlock(&data->lock);
> >
> > - time = timespec_to_ns(&a) - timespec_to_ns(&b);
> > delta = (after - before) * UJ_PER_MJ;
> >
> > return sprintf(buf, "%llu\n",
>
> I'm not familiar with the driver and I can't test it, and I'm not
> familiar with kernel timekeeping either. All I can say is that the
> changes look reasonable and good.
>
> Acked-by: Jean Delvare <jdelvare@suse.de>
>
> Darrick, maybe you want to comment or test?
Looks ok to me from a theoretical point of view, but since I no longer work at
IBM, I don't have hardware for a real test. :(
--D
>
> --
> Jean Delvare
> SUSE L3 Support
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 41/64] iio: Use ktime_get_real_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (39 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 40/64] hwmon: ibmaem: " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 42/64] arm: bL_switcher:k " Thomas Gleixner
` (23 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Jonathan Cameron
[-- Attachment #1: iio-use-ktime-get-ns.patch --]
[-- Type: text/plain, Size: 852 bytes --]
No idea why iio needs wall clock based time stamps, but we can avoid
the timespec conversion dance by using the new interfaces.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Jonathan Cameron <jic23@kernel.org>
---
include/linux/iio/iio.h | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
Index: tip/include/linux/iio/iio.h
===================================================================
--- tip.orig/include/linux/iio/iio.h
+++ tip/include/linux/iio/iio.h
@@ -277,14 +277,7 @@ static inline bool iio_channel_has_info(
**/
static inline s64 iio_get_time_ns(void)
{
- struct timespec ts;
- /*
- * calls getnstimeofday.
- * If hrtimers then up to ns accurate, if not microsecond.
- */
- ktime_get_real_ts(&ts);
-
- return timespec_to_ns(&ts);
+ return ktime_get_real_ns();
}
/* Device operating modes */
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 42/64] arm: bL_switcher:k Use ktime_get_real_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (40 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 41/64] iio: Use ktime_get_real_ns() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 43/64] x86: kvm: Use ktime_get_boot_ns() Thomas Gleixner
` (22 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Russell King, linux-arm-kernel
[-- Attachment #1: arm-bL_switcher-use-ktime_get_ns.patch --]
[-- Type: text/plain, Size: 1646 bytes --]
Use the nanoseconds based interface instead of converting from a
timespec.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm/common/bL_switcher.c | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
Index: tip/arch/arm/common/bL_switcher.c
===================================================================
--- tip.orig/arch/arm/common/bL_switcher.c
+++ tip/arch/arm/common/bL_switcher.c
@@ -58,16 +58,6 @@ static int read_mpidr(void)
}
/*
- * Get a global nanosecond time stamp for tracing.
- */
-static s64 get_ns(void)
-{
- struct timespec ts;
- getnstimeofday(&ts);
- return timespec_to_ns(&ts);
-}
-
-/*
* bL switcher core code.
*/
@@ -224,7 +214,7 @@ static int bL_switch_to(unsigned int new
*/
local_irq_disable();
local_fiq_disable();
- trace_cpu_migrate_begin(get_ns(), ob_mpidr);
+ trace_cpu_migrate_begin(ktime_get_real_ns(), ob_mpidr);
/* redirect GIC's SGIs to our counterpart */
gic_migrate_target(bL_gic_id[ib_cpu][ib_cluster]);
@@ -267,7 +257,7 @@ static int bL_switch_to(unsigned int new
tdev->evtdev->next_event, 1);
}
- trace_cpu_migrate_finish(get_ns(), ib_mpidr);
+ trace_cpu_migrate_finish(ktime_get_real_ns(), ib_mpidr);
local_fiq_enable();
local_irq_enable();
@@ -558,7 +548,7 @@ int bL_switcher_get_logical_index(u32 mp
static void bL_switcher_trace_trigger_cpu(void *__always_unused info)
{
- trace_cpu_migrate_current(get_ns(), read_mpidr());
+ trace_cpu_migrate_current(ktime_get_real_ns(), read_mpidr());
}
int bL_switcher_trace_trigger(void)
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 43/64] x86: kvm: Use ktime_get_boot_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (41 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 42/64] arm: bL_switcher:k " Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-17 10:58 ` Paolo Bonzini
2014-07-16 21:04 ` [patch V2 44/64] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based Thomas Gleixner
` (21 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
[-- Attachment #1: x86-kvm-use-ktime.patch --]
[-- Type: text/plain, Size: 714 bytes --]
Use the new nanoseconds based interface and get rid of the timespec
conversion dance.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: kvm@vger.kernel.org
---
arch/x86/kvm/x86.c | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -1109,11 +1109,7 @@ static void kvm_get_time_scale(uint32_t
static inline u64 get_kernel_ns(void)
{
- struct timespec ts;
-
- ktime_get_ts(&ts);
- monotonic_to_bootbased(&ts);
- return timespec_to_ns(&ts);
+ return ktime_get_boot_ns();
}
#ifdef CONFIG_X86_64
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 43/64] x86: kvm: Use ktime_get_boot_ns()
2014-07-16 21:04 ` [patch V2 43/64] x86: kvm: Use ktime_get_boot_ns() Thomas Gleixner
@ 2014-07-17 10:58 ` Paolo Bonzini
0 siblings, 0 replies; 76+ messages in thread
From: Paolo Bonzini @ 2014-07-17 10:58 UTC (permalink / raw)
To: Thomas Gleixner, LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
Il 16/07/2014 23:04, Thomas Gleixner ha scritto:
> Use the new nanoseconds based interface and get rid of the timespec
> conversion dance.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Gleb Natapov <gleb@kernel.org>
> Cc: kvm@vger.kernel.org
> ---
> arch/x86/kvm/x86.c | 6 +-----
> 1 file changed, 1 insertion(+), 5 deletions(-)
>
> Index: tip/arch/x86/kvm/x86.c
> ===================================================================
> --- tip.orig/arch/x86/kvm/x86.c
> +++ tip/arch/x86/kvm/x86.c
> @@ -1109,11 +1109,7 @@ static void kvm_get_time_scale(uint32_t
>
> static inline u64 get_kernel_ns(void)
> {
> - struct timespec ts;
> -
> - ktime_get_ts(&ts);
> - monotonic_to_bootbased(&ts);
> - return timespec_to_ns(&ts);
> + return ktime_get_boot_ns();
> }
>
> #ifdef CONFIG_X86_64
>
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
I will remove get_kernel_ns if you don't do that for me...
Paolo
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 44/64] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (42 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 43/64] x86: kvm: Use ktime_get_boot_ns() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-17 10:58 ` Paolo Bonzini
2014-07-16 21:04 ` [patch V2 45/64] timekeeping: Remove monotonic_to_bootbased Thomas Gleixner
` (20 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
[-- Attachment #1: x86-kvm-use-ktime-based.patch --]
[-- Type: text/plain, Size: 3093 bytes --]
Convert the relevant base data right away to nanoseconds instead of
doing the conversion on every readout. Reduces text size by 160 bytes.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: kvm@vger.kernel.org
---
arch/x86/kvm/x86.c | 44 ++++++++++++++------------------------------
1 file changed, 14 insertions(+), 30 deletions(-)
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -984,9 +984,8 @@ struct pvclock_gtod_data {
u32 shift;
} clock;
- /* open coded 'struct timespec' */
- u64 monotonic_time_snsec;
- time_t monotonic_time_sec;
+ u64 boot_ns;
+ u64 nsec_base;
};
static struct pvclock_gtod_data pvclock_gtod_data;
@@ -994,6 +993,9 @@ static struct pvclock_gtod_data pvclock_
static void update_pvclock_gtod(struct timekeeper *tk)
{
struct pvclock_gtod_data *vdata = &pvclock_gtod_data;
+ u64 boot_ns;
+
+ boot_ns = ktime_to_ns(ktime_add(tk->base_mono, tk->offs_boot));
write_seqcount_begin(&vdata->seq);
@@ -1004,17 +1006,8 @@ static void update_pvclock_gtod(struct t
vdata->clock.mult = tk->mult;
vdata->clock.shift = tk->shift;
- vdata->monotonic_time_sec = tk->xtime_sec
- + tk->wall_to_monotonic.tv_sec;
- vdata->monotonic_time_snsec = tk->xtime_nsec
- + (tk->wall_to_monotonic.tv_nsec
- << tk->shift);
- while (vdata->monotonic_time_snsec >=
- (((u64)NSEC_PER_SEC) << tk->shift)) {
- vdata->monotonic_time_snsec -=
- ((u64)NSEC_PER_SEC) << tk->shift;
- vdata->monotonic_time_sec++;
- }
+ vdata->boot_ns = boot_ns;
+ vdata->nsec_base = tk->xtime_nsec;
write_seqcount_end(&vdata->seq);
}
@@ -1371,23 +1364,22 @@ static inline u64 vgettsc(cycle_t *cycle
return v * gtod->clock.mult;
}
-static int do_monotonic(struct timespec *ts, cycle_t *cycle_now)
+static int do_monotonic_boot(s64 *t, cycle_t *cycle_now)
{
+ struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
unsigned long seq;
- u64 ns;
int mode;
- struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
+ u64 ns;
- ts->tv_nsec = 0;
do {
seq = read_seqcount_begin(>od->seq);
mode = gtod->clock.vclock_mode;
- ts->tv_sec = gtod->monotonic_time_sec;
- ns = gtod->monotonic_time_snsec;
+ ns = gtod->nsec_base;
ns += vgettsc(cycle_now);
ns >>= gtod->clock.shift;
+ ns += gtod->boot_ns;
} while (unlikely(read_seqcount_retry(>od->seq, seq)));
- timespec_add_ns(ts, ns);
+ *t = ns;
return mode;
}
@@ -1395,19 +1387,11 @@ static int do_monotonic(struct timespec
/* returns true if host is using tsc clocksource */
static bool kvm_get_time_and_clockread(s64 *kernel_ns, cycle_t *cycle_now)
{
- struct timespec ts;
-
/* checked again under seqlock below */
if (pvclock_gtod_data.clock.vclock_mode != VCLOCK_TSC)
return false;
- if (do_monotonic(&ts, cycle_now) != VCLOCK_TSC)
- return false;
-
- monotonic_to_bootbased(&ts);
- *kernel_ns = timespec_to_ns(&ts);
-
- return true;
+ return do_monotonic_boot(kernel_ns, cycle_now) == VCLOCK_TSC;
}
#endif
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 44/64] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based
2014-07-16 21:04 ` [patch V2 44/64] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based Thomas Gleixner
@ 2014-07-17 10:58 ` Paolo Bonzini
0 siblings, 0 replies; 76+ messages in thread
From: Paolo Bonzini @ 2014-07-17 10:58 UTC (permalink / raw)
To: Thomas Gleixner, LKML; +Cc: John Stultz, Peter Zijlstra, Gleb Natapov, kvm
Il 16/07/2014 23:04, Thomas Gleixner ha scritto:
> Convert the relevant base data right away to nanoseconds instead of
> doing the conversion on every readout. Reduces text size by 160 bytes.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Gleb Natapov <gleb@kernel.org>
> Cc: kvm@vger.kernel.org
> ---
> arch/x86/kvm/x86.c | 44 ++++++++++++++------------------------------
> 1 file changed, 14 insertions(+), 30 deletions(-)
>
> Index: tip/arch/x86/kvm/x86.c
> ===================================================================
> --- tip.orig/arch/x86/kvm/x86.c
> +++ tip/arch/x86/kvm/x86.c
> @@ -984,9 +984,8 @@ struct pvclock_gtod_data {
> u32 shift;
> } clock;
>
> - /* open coded 'struct timespec' */
> - u64 monotonic_time_snsec;
> - time_t monotonic_time_sec;
> + u64 boot_ns;
> + u64 nsec_base;
> };
>
> static struct pvclock_gtod_data pvclock_gtod_data;
> @@ -994,6 +993,9 @@ static struct pvclock_gtod_data pvclock_
> static void update_pvclock_gtod(struct timekeeper *tk)
> {
> struct pvclock_gtod_data *vdata = &pvclock_gtod_data;
> + u64 boot_ns;
> +
> + boot_ns = ktime_to_ns(ktime_add(tk->base_mono, tk->offs_boot));
>
> write_seqcount_begin(&vdata->seq);
>
> @@ -1004,17 +1006,8 @@ static void update_pvclock_gtod(struct t
> vdata->clock.mult = tk->mult;
> vdata->clock.shift = tk->shift;
>
> - vdata->monotonic_time_sec = tk->xtime_sec
> - + tk->wall_to_monotonic.tv_sec;
> - vdata->monotonic_time_snsec = tk->xtime_nsec
> - + (tk->wall_to_monotonic.tv_nsec
> - << tk->shift);
> - while (vdata->monotonic_time_snsec >=
> - (((u64)NSEC_PER_SEC) << tk->shift)) {
> - vdata->monotonic_time_snsec -=
> - ((u64)NSEC_PER_SEC) << tk->shift;
> - vdata->monotonic_time_sec++;
> - }
> + vdata->boot_ns = boot_ns;
> + vdata->nsec_base = tk->xtime_nsec;
>
> write_seqcount_end(&vdata->seq);
> }
> @@ -1371,23 +1364,22 @@ static inline u64 vgettsc(cycle_t *cycle
> return v * gtod->clock.mult;
> }
>
> -static int do_monotonic(struct timespec *ts, cycle_t *cycle_now)
> +static int do_monotonic_boot(s64 *t, cycle_t *cycle_now)
> {
> + struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
> unsigned long seq;
> - u64 ns;
> int mode;
> - struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
> + u64 ns;
>
> - ts->tv_nsec = 0;
> do {
> seq = read_seqcount_begin(>od->seq);
> mode = gtod->clock.vclock_mode;
> - ts->tv_sec = gtod->monotonic_time_sec;
> - ns = gtod->monotonic_time_snsec;
> + ns = gtod->nsec_base;
> ns += vgettsc(cycle_now);
> ns >>= gtod->clock.shift;
> + ns += gtod->boot_ns;
> } while (unlikely(read_seqcount_retry(>od->seq, seq)));
> - timespec_add_ns(ts, ns);
> + *t = ns;
>
> return mode;
> }
> @@ -1395,19 +1387,11 @@ static int do_monotonic(struct timespec
> /* returns true if host is using tsc clocksource */
> static bool kvm_get_time_and_clockread(s64 *kernel_ns, cycle_t *cycle_now)
> {
> - struct timespec ts;
> -
> /* checked again under seqlock below */
> if (pvclock_gtod_data.clock.vclock_mode != VCLOCK_TSC)
> return false;
>
> - if (do_monotonic(&ts, cycle_now) != VCLOCK_TSC)
> - return false;
> -
> - monotonic_to_bootbased(&ts);
> - *kernel_ns = timespec_to_ns(&ts);
> -
> - return true;
> + return do_monotonic_boot(kernel_ns, cycle_now) == VCLOCK_TSC;
> }
> #endif
>
>
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 45/64] timekeeping: Remove monotonic_to_bootbased
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (43 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 44/64] x86: kvm: Make kvm_get_time_and_clockread() nanoseconds based Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 46/64] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime() Thomas Gleixner
` (19 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-remove-monotonic-to-bootbased.patch --]
[-- Type: text/plain, Size: 1524 bytes --]
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 1 -
kernel/time/timekeeping.c | 15 ---------------
2 files changed, 16 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -27,7 +27,6 @@ struct timespec __current_kernel_time(vo
*/
struct timespec get_monotonic_coarse(void);
extern void getrawmonotonic(struct timespec *ts);
-extern void monotonic_to_bootbased(struct timespec *ts);
extern void get_monotonic_boottime(struct timespec *ts);
extern void ktime_get_ts64(struct timespec64 *ts);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1570,21 +1570,6 @@ void get_monotonic_boottime(struct times
}
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
-/**
- * monotonic_to_bootbased - Convert the monotonic time to boot based.
- * @ts: pointer to the timespec to be converted
- */
-void monotonic_to_bootbased(struct timespec *ts)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 ts64;
-
- ts64 = timespec_to_timespec64(*ts);
- ts64 = timespec64_add(ts64, tk->total_sleep_time);
- *ts = timespec64_to_timespec(ts64);
-}
-EXPORT_SYMBOL_GPL(monotonic_to_bootbased);
-
unsigned long get_seconds(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 46/64] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (44 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 45/64] timekeeping: Remove monotonic_to_bootbased Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:04 ` [patch V2 47/64] timekeeping: Simplify getboottime() Thomas Gleixner
` (18 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-simplify-get-monotonic-boottime.patch --]
[-- Type: text/plain, Size: 2500 bytes --]
get_monotonic_boottime() is not used in fast pathes, so the extra
timespec conversion is not problematic.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 9 ++++++++-
kernel/time/timekeeping.c | 34 ----------------------------------
2 files changed, 8 insertions(+), 35 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -27,7 +27,6 @@ struct timespec __current_kernel_time(vo
*/
struct timespec get_monotonic_coarse(void);
extern void getrawmonotonic(struct timespec *ts);
-extern void get_monotonic_boottime(struct timespec *ts);
extern void ktime_get_ts64(struct timespec64 *ts);
extern int __getnstimeofday64(struct timespec64 *tv);
@@ -160,6 +159,14 @@ static inline u64 ktime_get_boot_ns(void
}
/*
+ * Timespec interfaces utilizing the ktime based ones
+ */
+static inline void get_monotonic_boottime(struct timespec *ts)
+{
+ *ts = ktime_to_timespec(ktime_get_boottime());
+}
+
+/*
* RTC specific
*/
extern void timekeeping_inject_sleeptime(struct timespec *delta);
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1536,40 +1536,6 @@ void getboottime(struct timespec *ts)
}
EXPORT_SYMBOL_GPL(getboottime);
-/**
- * get_monotonic_boottime - Returns monotonic time since boot
- * @ts: pointer to the timespec to be set
- *
- * Returns the monotonic time since boot in a timespec.
- *
- * This is similar to CLOCK_MONTONIC/ktime_get_ts, but also
- * includes the time spent in suspend.
- */
-void get_monotonic_boottime(struct timespec *ts)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 tomono, sleep, ret;
- s64 nsec;
- unsigned int seq;
-
- WARN_ON(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
- ret.tv_sec = tk->xtime_sec;
- nsec = timekeeping_get_ns(tk);
- tomono = tk->wall_to_monotonic;
- sleep = tk->total_sleep_time;
-
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- ret.tv_sec += tomono.tv_sec + sleep.tv_sec;
- ret.tv_nsec = 0;
- timespec64_add_ns(&ret, nsec + tomono.tv_nsec + sleep.tv_nsec);
- *ts = timespec64_to_timespec(ret);
-}
-EXPORT_SYMBOL_GPL(get_monotonic_boottime);
-
unsigned long get_seconds(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 47/64] timekeeping: Simplify getboottime()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (45 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 46/64] timekeeping: Use ktime_get_boottime() for get_monotonic_boottime() Thomas Gleixner
@ 2014-07-16 21:04 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 48/64] timekeeping: Remove timekeeper.total_sleep_time Thomas Gleixner
` (17 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:04 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-simplify-getboottime.patch --]
[-- Type: text/plain, Size: 992 bytes --]
Subtracting plain nsec values and converting to timespec is simpler
than the whole timespec math. Not really fastpath code, so the
division is not an issue.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/timekeeping.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -1525,14 +1525,9 @@ out:
void getboottime(struct timespec *ts)
{
struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec boottime = {
- .tv_sec = tk->wall_to_monotonic.tv_sec +
- tk->total_sleep_time.tv_sec,
- .tv_nsec = tk->wall_to_monotonic.tv_nsec +
- tk->total_sleep_time.tv_nsec
- };
+ ktime_t t = ktime_sub(tk->offs_real, tk->offs_boot);
- set_normalized_timespec(ts, -boottime.tv_sec, -boottime.tv_nsec);
+ *ts = ktime_to_timespec(t);
}
EXPORT_SYMBOL_GPL(getboottime);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 48/64] timekeeping: Remove timekeeper.total_sleep_time
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (46 preceding siblings ...)
2014-07-16 21:04 ` [patch V2 47/64] timekeeping: Simplify getboottime() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 49/64] timekeeping: Simplify timekeeping_clocktai() Thomas Gleixner
` (16 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-kill-total-sleep-time.patch --]
[-- Type: text/plain, Size: 2562 bytes --]
No more users. Remove it
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 6 ++----
kernel/time/timekeeping.c | 14 +++-----------
2 files changed, 5 insertions(+), 15 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -20,8 +20,8 @@
* the tv_nsec part positive so we can use the usual normalization.
*
* wall_to_monotonic is moved after resume from suspend for the
- * monotonic time not to jump. We need to add total_sleep_time to
- * wall_to_monotonic to get the real boot based time offset.
+ * monotonic time not to jump. To calculate the real boot time offset
+ * we need to do offs_real - offs_boot.
*
* - wall_to_monotonic is no longer the boot time, getboottime must be
* used instead.
@@ -51,8 +51,6 @@ struct timekeeper {
/* Offset clock monotonic -> clock tai */
ktime_t offs_tai;
- /* time spent in suspend */
- struct timespec64 total_sleep_time;
/* The current UTC to TAI offset in seconds */
s32 tai_offset;
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -97,13 +97,9 @@ static void tk_set_wall_to_mono(struct t
tk->offs_tai = ktime_add(tk->offs_real, ktime_set(tk->tai_offset, 0));
}
-static void tk_set_sleep_time(struct timekeeper *tk, struct timespec64 t)
+static inline void tk_update_sleep_time(struct timekeeper *tk, ktime_t delta)
{
- /* Verify consistency before modifying */
- WARN_ON_ONCE(tk->offs_boot.tv64 != timespec64_to_ktime(tk->total_sleep_time).tv64);
-
- tk->total_sleep_time = t;
- tk->offs_boot = timespec64_to_ktime(t);
+ tk->offs_boot = ktime_add(tk->offs_boot, delta);
}
/**
@@ -919,10 +915,6 @@ void __init timekeeping_init(void)
set_normalized_timespec64(&tmp, -boot.tv_sec, -boot.tv_nsec);
tk_set_wall_to_mono(tk, tmp);
- tmp.tv_sec = 0;
- tmp.tv_nsec = 0;
- tk_set_sleep_time(tk, tmp);
-
timekeeping_update(tk, TK_MIRROR);
write_seqcount_end(&tk_core.seq);
@@ -950,7 +942,7 @@ static void __timekeeping_inject_sleepti
}
tk_xtime_add(tk, delta);
tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *delta));
- tk_set_sleep_time(tk, timespec64_add(tk->total_sleep_time, *delta));
+ tk_update_sleep_time(tk, timespec64_to_ktime(*delta));
tk_debug_account_sleep_time(delta);
}
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 49/64] timekeeping: Simplify timekeeping_clocktai()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (47 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 48/64] timekeeping: Remove timekeeper.total_sleep_time Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 50/64] hangcheck-timer: Use ktime_get_ns() Thomas Gleixner
` (15 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-simplify-clocktai.patch --]
[-- Type: text/plain, Size: 1780 bytes --]
timekeeping_clocktai() is not used in fast pathes, so the extra
timespec conversion is not problematic.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeping.h | 5 +++++
kernel/time/timekeeping.c | 31 -------------------------------
2 files changed, 5 insertions(+), 31 deletions(-)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -166,6 +166,11 @@ static inline void get_monotonic_boottim
*ts = ktime_to_timespec(ktime_get_boottime());
}
+static inline void timekeeping_clocktai(struct timespec *ts)
+{
+ *ts = ktime_to_timespec(ktime_get_clocktai());
+}
+
/*
* RTC specific
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -497,37 +497,6 @@ void ktime_get_ts64(struct timespec64 *t
}
EXPORT_SYMBOL_GPL(ktime_get_ts64);
-
-/**
- * timekeeping_clocktai - Returns the TAI time of day in a timespec
- * @ts: pointer to the timespec to be set
- *
- * Returns the time of day in a timespec.
- */
-void timekeeping_clocktai(struct timespec *ts)
-{
- struct timekeeper *tk = &tk_core.timekeeper;
- struct timespec64 ts64;
- unsigned long seq;
- u64 nsecs;
-
- WARN_ON(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(&tk_core.seq);
-
- ts64.tv_sec = tk->xtime_sec + tk->tai_offset;
- nsecs = timekeeping_get_ns(tk);
-
- } while (read_seqcount_retry(&tk_core.seq, seq));
-
- ts64.tv_nsec = 0;
- timespec64_add_ns(&ts64, nsecs);
- *ts = timespec64_to_timespec(ts64);
-
-}
-EXPORT_SYMBOL(timekeeping_clocktai);
-
#ifdef CONFIG_NTP_PPS
/**
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 50/64] hangcheck-timer: Use ktime_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (48 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 49/64] timekeeping: Simplify timekeeping_clocktai() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 22:08 ` Greg Kroah-Hartman
2014-07-16 21:05 ` [patch V2 51/64] timekeeping: Provide ktime_get_raw() Thomas Gleixner
` (14 subsequent siblings)
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML
Cc: John Stultz, Peter Zijlstra, Arnd Bergmann, Greg Kroah-Hartman,
Heiko Carstens
[-- Attachment #1: hangcheck-timer-get-rid-of-cruft.patch --]
[-- Type: text/plain, Size: 3005 bytes --]
There is no point in having a S390 private implementation and there is
no point in using the raw monotonic time. The NTP freqeuency
adjustment of CLOCK_MONOTONIC is really not doing any harm for the
hang check timer.
Use ktime_get_ns() for everything and get rid of the timespec
conversions.
V2: Drop the raw monotonic and the S390 special case
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
---
drivers/char/hangcheck-timer.c | 33 +++++----------------------------
1 file changed, 5 insertions(+), 28 deletions(-)
Index: tip/drivers/char/hangcheck-timer.c
===================================================================
--- tip.orig/drivers/char/hangcheck-timer.c
+++ tip/drivers/char/hangcheck-timer.c
@@ -49,7 +49,7 @@
#include <asm/uaccess.h>
#include <linux/sysrq.h>
#include <linux/timer.h>
-#include <linux/time.h>
+#include <linux/hrtimer.h>
#define VERSION_STR "0.9.1"
@@ -117,24 +117,7 @@ __setup("hcheck_reboot", hangcheck_parse
__setup("hcheck_dump_tasks", hangcheck_parse_dump_tasks);
#endif /* not MODULE */
-#if defined(CONFIG_S390)
-# define HAVE_MONOTONIC
-# define TIMER_FREQ 1000000000ULL
-#else
-# define TIMER_FREQ 1000000000ULL
-#endif
-
-#ifdef HAVE_MONOTONIC
-extern unsigned long long monotonic_clock(void);
-#else
-static inline unsigned long long monotonic_clock(void)
-{
- struct timespec ts;
- getrawmonotonic(&ts);
- return timespec_to_ns(&ts);
-}
-#endif /* HAVE_MONOTONIC */
-
+#define TIMER_FREQ 1000000000ULL
/* Last time scheduled */
static unsigned long long hangcheck_tsc, hangcheck_tsc_margin;
@@ -143,12 +126,11 @@ static void hangcheck_fire(unsigned long
static DEFINE_TIMER(hangcheck_ticktock, hangcheck_fire, 0, 0);
-
static void hangcheck_fire(unsigned long data)
{
unsigned long long cur_tsc, tsc_diff;
- cur_tsc = monotonic_clock();
+ cur_tsc = ktime_get_ns();
if (cur_tsc > hangcheck_tsc)
tsc_diff = cur_tsc - hangcheck_tsc;
@@ -177,7 +159,7 @@ static void hangcheck_fire(unsigned long
tsc_diff, tsc_diff - hangcheck_tick*TIMER_FREQ);
#endif
mod_timer(&hangcheck_ticktock, jiffies + (hangcheck_tick*HZ));
- hangcheck_tsc = monotonic_clock();
+ hangcheck_tsc = ktime_get_ns();
}
@@ -185,16 +167,11 @@ static int __init hangcheck_init(void)
{
printk("Hangcheck: starting hangcheck timer %s (tick is %d seconds, margin is %d seconds).\n",
VERSION_STR, hangcheck_tick, hangcheck_margin);
-#if defined (HAVE_MONOTONIC)
- printk("Hangcheck: Using monotonic_clock().\n");
-#else
- printk("Hangcheck: Using getrawmonotonic().\n");
-#endif /* HAVE_MONOTONIC */
hangcheck_tsc_margin =
(unsigned long long)(hangcheck_margin + hangcheck_tick);
hangcheck_tsc_margin *= (unsigned long long)TIMER_FREQ;
- hangcheck_tsc = monotonic_clock();
+ hangcheck_tsc = ktime_get_ns();
mod_timer(&hangcheck_ticktock, jiffies + (hangcheck_tick*HZ));
return 0;
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 50/64] hangcheck-timer: Use ktime_get_ns()
2014-07-16 21:05 ` [patch V2 50/64] hangcheck-timer: Use ktime_get_ns() Thomas Gleixner
@ 2014-07-16 22:08 ` Greg Kroah-Hartman
0 siblings, 0 replies; 76+ messages in thread
From: Greg Kroah-Hartman @ 2014-07-16 22:08 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, John Stultz, Peter Zijlstra, Arnd Bergmann, Heiko Carstens
On Wed, Jul 16, 2014 at 09:05:03PM -0000, Thomas Gleixner wrote:
> There is no point in having a S390 private implementation and there is
> no point in using the raw monotonic time. The NTP freqeuency
> adjustment of CLOCK_MONOTONIC is really not doing any harm for the
> hang check timer.
>
> Use ktime_get_ns() for everything and get rid of the timespec
> conversions.
>
> V2: Drop the raw monotonic and the S390 special case
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
^ permalink raw reply [flat|nested] 76+ messages in thread
* [patch V2 51/64] timekeeping: Provide ktime_get_raw()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (49 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 50/64] hangcheck-timer: Use ktime_get_ns() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 52/64] drm: i915: Use nsec based interfaces Thomas Gleixner
` (13 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-provide-ktime-get-raw.patch --]
[-- Type: text/plain, Size: 2935 bytes --]
Provide a ktime_t based interface for raw monotonic time.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 3 +++
include/linux/timekeeping.h | 6 ++++++
kernel/time/timekeeping.c | 25 +++++++++++++++++++++++++
3 files changed, 34 insertions(+)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -54,6 +54,9 @@ struct timekeeper {
/* The current UTC to TAI offset in seconds */
s32 tai_offset;
+ /* Monotonic raw base time */
+ ktime_t base_raw;
+
/* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
struct timespec64 raw_time;
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -107,6 +107,7 @@ enum tk_offsets {
extern ktime_t ktime_get(void);
extern ktime_t ktime_get_with_offset(enum tk_offsets offs);
extern ktime_t ktime_mono_to_any(ktime_t tmono, enum tk_offsets offs);
+extern ktime_t ktime_get_raw(void);
/**
* ktime_get_real - get the real (wall-) time in ktime_t format
@@ -158,6 +159,11 @@ static inline u64 ktime_get_boot_ns(void
return ktime_to_ns(ktime_get_boottime());
}
+static inline u64 ktime_get_raw_ns(void)
+{
+ return ktime_to_ns(ktime_get_raw());
+}
+
/*
* Timespec interfaces utilizing the ktime based ones
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -305,6 +305,9 @@ static inline void tk_update_ktime_data(
nsec *= NSEC_PER_SEC;
nsec += tk->wall_to_monotonic.tv_nsec;
tk->base_mono = ns_to_ktime(nsec);
+
+ /* Update the monotonic raw base */
+ tk->base_raw = timespec64_to_ktime(tk->raw_time);
}
/* must hold timekeeper_lock */
@@ -467,6 +470,27 @@ ktime_t ktime_mono_to_any(ktime_t tmono,
EXPORT_SYMBOL_GPL(ktime_mono_to_any);
/**
+ * ktime_get_raw - Returns the raw monotonic time in ktime_t format
+ */
+ktime_t ktime_get_raw(void)
+{
+ struct timekeeper *tk = &tk_core.timekeeper;
+ unsigned int seq;
+ ktime_t base;
+ s64 nsecs;
+
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ base = tk->base_raw;
+ nsecs = timekeeping_get_ns_raw(tk);
+
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ return ktime_add_ns(base, nsecs);
+}
+EXPORT_SYMBOL_GPL(ktime_get_raw);
+
+/**
* ktime_get_ts64 - get the monotonic clock in timespec64 format
* @ts: pointer to timespec variable
*
@@ -878,6 +902,7 @@ void __init timekeeping_init(void)
tk_set_xtime(tk, &now);
tk->raw_time.tv_sec = 0;
tk->raw_time.tv_nsec = 0;
+ tk->base_raw.tv64 = 0;
if (boot.tv_sec == 0 && boot.tv_nsec == 0)
boot = tk_xtime(tk);
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 52/64] drm: i915: Use nsec based interfaces
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (50 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 51/64] timekeeping: Provide ktime_get_raw() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 53/64] drm: vmwgfx: " Thomas Gleixner
` (12 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Daniel Vetter
[-- Attachment #1: drm-i915-use-nsec-interfaces.patch --]
[-- Type: text/plain, Size: 5456 bytes --]
Use ktime_get_raw_ns() and get rid of the back and forth timespec
conversions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 33 ++++++++++++---------------------
drivers/gpu/drm/i915/intel_pm.c | 12 +++++-------
3 files changed, 18 insertions(+), 29 deletions(-)
Index: tip/drivers/gpu/drm/i915/i915_drv.h
===================================================================
--- tip.orig/drivers/gpu/drm/i915/i915_drv.h
+++ tip/drivers/gpu/drm/i915/i915_drv.h
@@ -931,7 +931,7 @@ struct intel_ilk_power_mgmt {
unsigned long last_time1;
unsigned long chipset_power;
u64 last_count2;
- struct timespec last_time2;
+ u64 last_time2;
unsigned long gfx_power;
u8 corr;
Index: tip/drivers/gpu/drm/i915/i915_gem.c
===================================================================
--- tip.orig/drivers/gpu/drm/i915/i915_gem.c
+++ tip/drivers/gpu/drm/i915/i915_gem.c
@@ -1149,16 +1149,16 @@ static bool can_wait_boost(struct drm_i9
static int __wait_seqno(struct intel_engine_cs *ring, u32 seqno,
unsigned reset_counter,
bool interruptible,
- struct timespec *timeout,
+ s64 *timeout,
struct drm_i915_file_private *file_priv)
{
struct drm_device *dev = ring->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
const bool irq_test_in_progress =
ACCESS_ONCE(dev_priv->gpu_error.test_irq_rings) & intel_ring_flag(ring);
- struct timespec before, now;
DEFINE_WAIT(wait);
unsigned long timeout_expire;
+ s64 before, now;
int ret;
WARN(dev_priv->pm.irqs_disabled, "IRQs disabled\n");
@@ -1166,7 +1166,7 @@ static int __wait_seqno(struct intel_eng
if (i915_seqno_passed(ring->get_seqno(ring, true), seqno))
return 0;
- timeout_expire = timeout ? jiffies + timespec_to_jiffies_timeout(timeout) : 0;
+ timeout_expire = timeout ? jiffies + nsecs_to_jiffies((u64)*timeout) : 0;
if (INTEL_INFO(dev)->gen >= 6 && can_wait_boost(file_priv)) {
gen6_rps_boost(dev_priv);
@@ -1181,7 +1181,7 @@ static int __wait_seqno(struct intel_eng
/* Record current time in case interrupted by signal, or wedged */
trace_i915_gem_request_wait_begin(ring, seqno);
- getrawmonotonic(&before);
+ before = ktime_get_raw_ns();
for (;;) {
struct timer_list timer;
@@ -1230,7 +1230,7 @@ static int __wait_seqno(struct intel_eng
destroy_timer_on_stack(&timer);
}
}
- getrawmonotonic(&now);
+ now = ktime_get_raw_ns();
trace_i915_gem_request_wait_end(ring, seqno);
if (!irq_test_in_progress)
@@ -1239,10 +1239,9 @@ static int __wait_seqno(struct intel_eng
finish_wait(&ring->irq_queue, &wait);
if (timeout) {
- struct timespec sleep_time = timespec_sub(now, before);
- *timeout = timespec_sub(*timeout, sleep_time);
- if (!timespec_valid(timeout)) /* i.e. negative time remains */
- set_normalized_timespec(timeout, 0, 0);
+ s64 tres = *timeout - (now - before);
+
+ *timeout = tres < 0 ? 0 : tres;
}
return ret;
@@ -2753,16 +2752,10 @@ i915_gem_wait_ioctl(struct drm_device *d
struct drm_i915_gem_wait *args = data;
struct drm_i915_gem_object *obj;
struct intel_engine_cs *ring = NULL;
- struct timespec timeout_stack, *timeout = NULL;
unsigned reset_counter;
u32 seqno = 0;
int ret = 0;
- if (args->timeout_ns >= 0) {
- timeout_stack = ns_to_timespec(args->timeout_ns);
- timeout = &timeout_stack;
- }
-
ret = i915_mutex_lock_interruptible(dev);
if (ret)
return ret;
@@ -2787,9 +2780,9 @@ i915_gem_wait_ioctl(struct drm_device *d
goto out;
/* Do this after OLR check to make sure we make forward progress polling
- * on this IOCTL with a 0 timeout (like busy ioctl)
+ * on this IOCTL with a timeout <=0 (like busy ioctl)
*/
- if (!args->timeout_ns) {
+ if (args->timeout_ns <= 0) {
ret = -ETIME;
goto out;
}
@@ -2798,10 +2791,8 @@ i915_gem_wait_ioctl(struct drm_device *d
reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
mutex_unlock(&dev->struct_mutex);
- ret = __wait_seqno(ring, seqno, reset_counter, true, timeout, file->driver_priv);
- if (timeout)
- args->timeout_ns = timespec_to_ns(timeout);
- return ret;
+ return __wait_seqno(ring, seqno, reset_counter, true, &args->timeout_ns,
+ file->driver_priv);
out:
drm_gem_object_unreference(&obj->base);
Index: tip/drivers/gpu/drm/i915/intel_pm.c
===================================================================
--- tip.orig/drivers/gpu/drm/i915/intel_pm.c
+++ tip/drivers/gpu/drm/i915/intel_pm.c
@@ -2993,7 +2993,7 @@ static void ironlake_enable_drps(struct
I915_READ(0x112e0);
dev_priv->ips.last_time1 = jiffies_to_msecs(jiffies);
dev_priv->ips.last_count2 = I915_READ(0x112f4);
- getrawmonotonic(&dev_priv->ips.last_time2);
+ dev_priv->ips.last_time2 = ktime_get_raw_ns();
spin_unlock_irq(&mchdev_lock);
}
@@ -4314,18 +4314,16 @@ static u16 pvid_to_extvid(struct drm_i91
static void __i915_update_gfx_val(struct drm_i915_private *dev_priv)
{
- struct timespec now, diff1;
- u64 diff;
- unsigned long diffms;
+ u64 now, diff, diffms;
u32 count;
assert_spin_locked(&mchdev_lock);
- getrawmonotonic(&now);
- diff1 = timespec_sub(now, dev_priv->ips.last_time2);
+ now = ktime_get_raw_ns();
+ diffms = now - dev_priv->ips.last_time2;
+ do_div(diffms, NSEC_PER_MSEC);
/* Don't divide by 0 */
- diffms = diff1.tv_sec * 1000 + diff1.tv_nsec / 1000000;
if (!diffms)
return;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 53/64] drm: vmwgfx: Use nsec based interfaces
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (51 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 52/64] drm: i915: Use nsec based interfaces Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 54/64] wireless: ath9k: Get rid of timespec conversions Thomas Gleixner
` (11 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Thomas Hellstrom
[-- Attachment #1: drm-vmwgfx-use-nsec-raw.patch --]
[-- Type: text/plain, Size: 3563 bytes --]
No point in converting timespecs back and forth.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
---
drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 4 +--
drivers/gpu/drm/vmwgfx/vmwgfx_marker.c | 44 ++++++++++-----------------------
2 files changed, 16 insertions(+), 32 deletions(-)
Index: tip/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
===================================================================
--- tip.orig/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ tip/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -159,8 +159,8 @@ struct vmw_surface {
struct vmw_marker_queue {
struct list_head head;
- struct timespec lag;
- struct timespec lag_time;
+ u64 lag;
+ u64 lag_time;
spinlock_t lock;
};
Index: tip/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
===================================================================
--- tip.orig/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
+++ tip/drivers/gpu/drm/vmwgfx/vmwgfx_marker.c
@@ -31,14 +31,14 @@
struct vmw_marker {
struct list_head head;
uint32_t seqno;
- struct timespec submitted;
+ u64 submitted;
};
void vmw_marker_queue_init(struct vmw_marker_queue *queue)
{
INIT_LIST_HEAD(&queue->head);
- queue->lag = ns_to_timespec(0);
- getrawmonotonic(&queue->lag_time);
+ queue->lag = 0;
+ queue->lag_time = ktime_get_raw_ns();
spin_lock_init(&queue->lock);
}
@@ -62,7 +62,7 @@ int vmw_marker_push(struct vmw_marker_qu
return -ENOMEM;
marker->seqno = seqno;
- getrawmonotonic(&marker->submitted);
+ marker->submitted = ktime_get_raw_ns();
spin_lock(&queue->lock);
list_add_tail(&marker->head, &queue->head);
spin_unlock(&queue->lock);
@@ -74,14 +74,14 @@ int vmw_marker_pull(struct vmw_marker_qu
uint32_t signaled_seqno)
{
struct vmw_marker *marker, *next;
- struct timespec now;
bool updated = false;
+ u64 now;
spin_lock(&queue->lock);
- getrawmonotonic(&now);
+ now = ktime_get_raw_ns();
if (list_empty(&queue->head)) {
- queue->lag = ns_to_timespec(0);
+ queue->lag = 0;
queue->lag_time = now;
updated = true;
goto out_unlock;
@@ -91,7 +91,7 @@ int vmw_marker_pull(struct vmw_marker_qu
if (signaled_seqno - marker->seqno > (1 << 30))
continue;
- queue->lag = timespec_sub(now, marker->submitted);
+ queue->lag = now - marker->submitted;
queue->lag_time = now;
updated = true;
list_del(&marker->head);
@@ -104,27 +104,13 @@ out_unlock:
return (updated) ? 0 : -EBUSY;
}
-static struct timespec vmw_timespec_add(struct timespec t1,
- struct timespec t2)
+static u64 vmw_fifo_lag(struct vmw_marker_queue *queue)
{
- t1.tv_sec += t2.tv_sec;
- t1.tv_nsec += t2.tv_nsec;
- if (t1.tv_nsec >= 1000000000L) {
- t1.tv_sec += 1;
- t1.tv_nsec -= 1000000000L;
- }
-
- return t1;
-}
-
-static struct timespec vmw_fifo_lag(struct vmw_marker_queue *queue)
-{
- struct timespec now;
+ u64 now;
spin_lock(&queue->lock);
- getrawmonotonic(&now);
- queue->lag = vmw_timespec_add(queue->lag,
- timespec_sub(now, queue->lag_time));
+ now = ktime_get_raw_ns();
+ queue->lag += now - queue->lag_time;
queue->lag_time = now;
spin_unlock(&queue->lock);
return queue->lag;
@@ -134,11 +120,9 @@ static struct timespec vmw_fifo_lag(stru
static bool vmw_lag_lt(struct vmw_marker_queue *queue,
uint32_t us)
{
- struct timespec lag, cond;
+ u64 cond = (u64) us * NSEC_PER_USEC;
- cond = ns_to_timespec((s64) us * 1000);
- lag = vmw_fifo_lag(queue);
- return (timespec_compare(&lag, &cond) < 1);
+ return vmw_fifo_lag(queue) <= cond;
}
int vmw_wait_lag(struct vmw_private *dev_priv,
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 54/64] wireless: ath9k: Get rid of timespec conversions
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (52 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 53/64] drm: vmwgfx: " Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 55/64] clocksource: Make delta calculation a function Thomas Gleixner
` (10 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, QCA ath9k Development, John W. Linville
[-- Attachment #1: wireless-ath9-use-proper-interfaces.patch --]
[-- Type: text/plain, Size: 1472 bytes --]
We have interfaces. Remove the open coded cruft. Reduces text size
along with the code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: QCA ath9k Development <ath9k-devel@qca.qualcomm.com>
Cc: John W. Linville <linville@tuxdriver.com>
---
drivers/net/wireless/ath/ath9k/hw.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
Index: tip/drivers/net/wireless/ath/ath9k/hw.c
===================================================================
--- tip.orig/drivers/net/wireless/ath/ath9k/hw.c
+++ tip/drivers/net/wireless/ath/ath9k/hw.c
@@ -1734,7 +1734,6 @@ int ath9k_hw_reset(struct ath_hw *ah, st
struct ath9k_hw_cal_data *caldata, bool fastcc)
{
struct ath_common *common = ath9k_hw_common(ah);
- struct timespec ts;
u32 saveLedState;
u32 saveDefAntenna;
u32 macStaId1;
@@ -1784,8 +1783,7 @@ int ath9k_hw_reset(struct ath_hw *ah, st
/* Save TSF before chip reset, a cold reset clears it */
tsf = ath9k_hw_gettsf64(ah);
- getrawmonotonic(&ts);
- usec = ts.tv_sec * 1000000ULL + ts.tv_nsec / 1000;
+ usec = ktime_to_us(ktime_get_raw());
saveLedState = REG_READ(ah, AR_CFG_LED) &
(AR_CFG_LED_ASSOC_CTL | AR_CFG_LED_MODE_SEL |
@@ -1818,8 +1816,7 @@ int ath9k_hw_reset(struct ath_hw *ah, st
}
/* Restore TSF */
- getrawmonotonic(&ts);
- usec = ts.tv_sec * 1000000ULL + ts.tv_nsec / 1000 - usec;
+ usec = ktime_to_us(ktime_get_raw()) - usec;
ath9k_hw_settsf64(ah, tsf + usec);
if (AR_SREV_9280_20_OR_LATER(ah))
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 55/64] clocksource: Make delta calculation a function
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (53 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 54/64] wireless: ath9k: Get rid of timespec conversions Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 56/64] clocksource: Move cycle_last validation to core code Thomas Gleixner
` (9 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: clocksource-make-delta-calculation-a-function.patch --]
[-- Type: text/plain, Size: 5651 bytes --]
We want to move the TSC sanity check into core code to make NMI safe
accessors to clock monotonic[_raw] possible. For this we need to
sanity check the delta calculation. Create a helper function and
convert all sites to use it.
[ Build fix from jstultz ]
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/time/clocksource.c | 12 +++++++-----
kernel/time/timekeeping.c | 26 ++++++++++++++------------
kernel/time/timekeeping_internal.h | 6 ++++++
3 files changed, 27 insertions(+), 17 deletions(-)
Index: tip/kernel/time/clocksource.c
===================================================================
--- tip.orig/kernel/time/clocksource.c
+++ tip/kernel/time/clocksource.c
@@ -32,6 +32,7 @@
#include <linux/kthread.h>
#include "tick-internal.h"
+#include "timekeeping_internal.h"
void timecounter_init(struct timecounter *tc,
const struct cyclecounter *cc,
@@ -249,7 +250,7 @@ void clocksource_mark_unstable(struct cl
static void clocksource_watchdog(unsigned long data)
{
struct clocksource *cs;
- cycle_t csnow, wdnow;
+ cycle_t csnow, wdnow, delta;
int64_t wd_nsec, cs_nsec;
int next_cpu, reset_pending;
@@ -282,11 +283,12 @@ static void clocksource_watchdog(unsigne
continue;
}
- wd_nsec = clocksource_cyc2ns((wdnow - cs->wd_last) & watchdog->mask,
- watchdog->mult, watchdog->shift);
+ delta = clocksource_delta(wdnow, cs->wd_last, watchdog->mask);
+ wd_nsec = clocksource_cyc2ns(delta, watchdog->mult,
+ watchdog->shift);
- cs_nsec = clocksource_cyc2ns((csnow - cs->cs_last) &
- cs->mask, cs->mult, cs->shift);
+ delta = clocksource_delta(csnow, cs->cs_last, cs->mask);
+ cs_nsec = clocksource_cyc2ns(delta, cs->mult, cs->shift);
cs->cs_last = csnow;
cs->wd_last = wdnow;
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -173,7 +173,7 @@ static inline u32 arch_gettimeoffset(voi
static inline s64 timekeeping_get_ns(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
@@ -182,9 +182,9 @@ static inline s64 timekeeping_get_ns(str
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
- nsec = cycle_delta * tk->mult + tk->xtime_nsec;
+ nsec = delta * tk->mult + tk->xtime_nsec;
nsec >>= tk->shift;
/* If arch requires, add in get_arch_timeoffset() */
@@ -193,7 +193,7 @@ static inline s64 timekeeping_get_ns(str
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
@@ -202,10 +202,10 @@ static inline s64 timekeeping_get_ns_raw
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
/* convert delta to nanoseconds. */
- nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
+ nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
/* If arch requires, add in get_arch_timeoffset() */
return nsec + arch_gettimeoffset();
@@ -336,23 +336,23 @@ static void timekeeping_update(struct ti
*/
static void timekeeping_forward_now(struct timekeeper *tk)
{
- cycle_t cycle_now, cycle_delta;
+ cycle_t cycle_now, delta;
struct clocksource *clock;
s64 nsec;
clock = tk->clock;
cycle_now = clock->read(clock);
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
tk->cycle_last = clock->cycle_last = cycle_now;
- tk->xtime_nsec += cycle_delta * tk->mult;
+ tk->xtime_nsec += delta * tk->mult;
/* If arch requires, add in get_arch_timeoffset() */
tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
tk_normalize_xtime(tk);
- nsec = clocksource_cyc2ns(cycle_delta, clock->mult, clock->shift);
+ nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
timespec64_add_ns(&tk->raw_time, nsec);
}
@@ -1026,7 +1026,8 @@ static void timekeeping_resume(void)
u32 shift = clock->shift;
s64 nsec = 0;
- cycle_delta = (cycle_now - clock->cycle_last) & clock->mask;
+ cycle_delta = clocksource_delta(cycle_now, clock->cycle_last,
+ clock->mask);
/*
* "cycle_delta * mutl" may cause 64 bits overflow, if the
@@ -1432,7 +1433,8 @@ void update_wall_time(void)
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = (clock->read(clock) - clock->cycle_last) & clock->mask;
+ offset = clocksource_delta(clock->read(clock), clock->cycle_last,
+ clock->mask);
#endif
/* Check if there's really nothing to do */
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -3,6 +3,7 @@
/*
* timekeeping debug functions
*/
+#include <linux/clocksource.h>
#include <linux/time.h>
#ifdef CONFIG_DEBUG_FS
@@ -11,4 +12,9 @@ extern void tk_debug_account_sleep_time(
#define tk_debug_account_sleep_time(x)
#endif
+static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
+{
+ return (now - last) & mask;
+}
+
#endif /* _TIMEKEEPING_INTERNAL_H */
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 56/64] clocksource: Move cycle_last validation to core code
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (54 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 55/64] clocksource: Make delta calculation a function Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 57/64] clocksource: Get rid of cycle_last Thomas Gleixner
` (8 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: clocksource-move-cycle_last-validation-to-core.patch --]
[-- Type: text/plain, Size: 4307 bytes --]
The only user of the cycle_last validation is the x86 TSC. In order to
provide NMI safe accessor functions for clock monotonic and
monotonic_raw we need to do that in the core.
We can't do the TSC specific
if (now < cycle_last)
now = cycle_last;
for the other wrapping around clocksources, but TSC has
CLOCKSOURCE_MASK(64) which actually does not mask out anything so if
now is less than cycle_last the subtraction will give a negative
result. So we can check for that in clocksource_delta() and return 0
for that case.
Implement and enable it for x86
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 1 +
arch/x86/kernel/tsc.c | 21 +++++++++------------
kernel/time/Kconfig | 5 +++++
kernel/time/timekeeping_internal.h | 9 +++++++++
4 files changed, 24 insertions(+), 12 deletions(-)
Index: tip/arch/x86/Kconfig
===================================================================
--- tip.orig/arch/x86/Kconfig
+++ tip/arch/x86/Kconfig
@@ -109,6 +109,7 @@ config X86
select CLOCKSOURCE_WATCHDOG
select GENERIC_CLOCKEVENTS
select ARCH_CLOCKSOURCE_DATA
+ select CLOCKSOURCE_VALIDATE_LAST_CYCLE
select GENERIC_CLOCKEVENTS_BROADCAST if X86_64 || (X86_32 && X86_LOCAL_APIC)
select GENERIC_TIME_VSYSCALL
select GENERIC_STRNCPY_FROM_USER
Index: tip/arch/x86/kernel/tsc.c
===================================================================
--- tip.orig/arch/x86/kernel/tsc.c
+++ tip/arch/x86/kernel/tsc.c
@@ -951,7 +951,7 @@ core_initcall(cpufreq_tsc);
static struct clocksource clocksource_tsc;
/*
- * We compare the TSC to the cycle_last value in the clocksource
+ * We used to compare the TSC to the cycle_last value in the clocksource
* structure to avoid a nasty time-warp. This can be observed in a
* very small window right after one CPU updated cycle_last under
* xtime/vsyscall_gtod lock and the other CPU reads a TSC value which
@@ -961,26 +961,23 @@ static struct clocksource clocksource_ts
* due to the unsigned delta calculation of the time keeping core
* code, which is necessary to support wrapping clocksources like pm
* timer.
+ *
+ * This sanity check is now done in the core timekeeping code.
+ * checking the result of read_tsc() - cycle_last for being negative.
+ * That works because CLOCKSOURCE_MASK(64) does not mask out any bit.
*/
static cycle_t read_tsc(struct clocksource *cs)
{
- cycle_t ret = (cycle_t)get_cycles();
-
- return ret >= clocksource_tsc.cycle_last ?
- ret : clocksource_tsc.cycle_last;
-}
-
-static void resume_tsc(struct clocksource *cs)
-{
- if (!boot_cpu_has(X86_FEATURE_NONSTOP_TSC_S3))
- clocksource_tsc.cycle_last = 0;
+ return (cycle_t)get_cycles();
}
+/*
+ * .mask MUST be CLOCKSOURCE_MASK(64). See comment above read_tsc()
+ */
static struct clocksource clocksource_tsc = {
.name = "tsc",
.rating = 300,
.read = read_tsc,
- .resume = resume_tsc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS |
CLOCK_SOURCE_MUST_VERIFY,
Index: tip/kernel/time/Kconfig
===================================================================
--- tip.orig/kernel/time/Kconfig
+++ tip/kernel/time/Kconfig
@@ -12,6 +12,11 @@ config CLOCKSOURCE_WATCHDOG
config ARCH_CLOCKSOURCE_DATA
bool
+# Clocksources require validation of the clocksource against the last
+# cycle update - x86/TSC misfeature
+config CLOCKSOURCE_VALIDATE_LAST_CYCLE
+ bool
+
# Timekeeping vsyscall support
config GENERIC_TIME_VSYSCALL
bool
Index: tip/kernel/time/timekeeping_internal.h
===================================================================
--- tip.orig/kernel/time/timekeeping_internal.h
+++ tip/kernel/time/timekeeping_internal.h
@@ -12,9 +12,18 @@ extern void tk_debug_account_sleep_time(
#define tk_debug_account_sleep_time(x)
#endif
+#ifdef CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE
+static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
+{
+ cycle_t ret = (now - last) & mask;
+
+ return (s64) ret > 0 ? ret : 0;
+}
+#else
static inline cycle_t clocksource_delta(cycle_t now, cycle_t last, cycle_t mask)
{
return (now - last) & mask;
}
+#endif
#endif /* _TIMEKEEPING_INTERNAL_H */
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 57/64] clocksource: Get rid of cycle_last
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (55 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 56/64] clocksource: Move cycle_last validation to core code Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 58/64] timekeeping: Restructure the timekeeper some more Thomas Gleixner
` (7 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: clocksource-get-rid-of-cycle-last.patch --]
[-- Type: text/plain, Size: 10535 bytes --]
cycle_last was added to the clocksource to support the TSC
validation. We moved that to the core code, so we can get rid of the
extra copy.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/arm64/kernel/vdso.c | 2 +-
arch/ia64/kernel/time.c | 4 ++--
arch/powerpc/kernel/time.c | 4 ++--
arch/s390/kernel/time.c | 2 +-
arch/tile/kernel/time.c | 2 +-
arch/x86/kernel/vsyscall_gtod.c | 2 +-
arch/x86/kvm/x86.c | 2 +-
include/linux/clocksource.h | 2 --
include/linux/timekeeper_internal.h | 7 ++++---
kernel/time/timekeeping.c | 23 +++++++++++------------
10 files changed, 24 insertions(+), 26 deletions(-)
Index: tip/arch/arm64/kernel/vdso.c
===================================================================
--- tip.orig/arch/arm64/kernel/vdso.c
+++ tip/arch/arm64/kernel/vdso.c
@@ -224,7 +224,7 @@ void update_vsyscall(struct timekeeper *
vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
if (!use_syscall) {
- vdso_data->cs_cycle_last = tk->clock->cycle_last;
+ vdso_data->cs_cycle_last = tk->cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_nsec = tk->xtime_nsec;
vdso_data->cs_mult = tk->mult;
Index: tip/arch/ia64/kernel/time.c
===================================================================
--- tip.orig/arch/ia64/kernel/time.c
+++ tip/arch/ia64/kernel/time.c
@@ -441,7 +441,7 @@ void update_vsyscall_tz(void)
}
void update_vsyscall_old(struct timespec *wall, struct timespec *wtm,
- struct clocksource *c, u32 mult)
+ struct clocksource *c, u32 mult, cycles_t cycle_last)
{
write_seqcount_begin(&fsyscall_gtod_data.seq);
@@ -450,7 +450,7 @@ void update_vsyscall_old(struct timespec
fsyscall_gtod_data.clk_mult = mult;
fsyscall_gtod_data.clk_shift = c->shift;
fsyscall_gtod_data.clk_fsys_mmio = c->archdata.fsys_mmio;
- fsyscall_gtod_data.clk_cycle_last = c->cycle_last;
+ fsyscall_gtod_data.clk_cycle_last = cycle_last;
/* copy kernel time structures */
fsyscall_gtod_data.wall_time.tv_sec = wall->tv_sec;
Index: tip/arch/powerpc/kernel/time.c
===================================================================
--- tip.orig/arch/powerpc/kernel/time.c
+++ tip/arch/powerpc/kernel/time.c
@@ -741,7 +741,7 @@ static cycle_t timebase_read(struct cloc
}
void update_vsyscall_old(struct timespec *wall_time, struct timespec *wtm,
- struct clocksource *clock, u32 mult)
+ struct clocksource *clock, u32 mult, cycle_t cycle_last)
{
u64 new_tb_to_xs, new_stamp_xsec;
u32 frac_sec;
@@ -774,7 +774,7 @@ void update_vsyscall_old(struct timespec
* We expect the caller to have done the first increment of
* vdso_data->tb_update_count already.
*/
- vdso_data->tb_orig_stamp = clock->cycle_last;
+ vdso_data->tb_orig_stamp = cycle_last;
vdso_data->stamp_xsec = new_stamp_xsec;
vdso_data->tb_to_xs = new_tb_to_xs;
vdso_data->wtom_clock_sec = wtm->tv_sec;
Index: tip/arch/s390/kernel/time.c
===================================================================
--- tip.orig/arch/s390/kernel/time.c
+++ tip/arch/s390/kernel/time.c
@@ -220,7 +220,7 @@ void update_vsyscall(struct timekeeper *
/* Make userspace gettimeofday spin until we're done. */
++vdso_data->tb_update_count;
smp_wmb();
- vdso_data->xtime_tod_stamp = tk->clock->cycle_last;
+ vdso_data->xtime_tod_stamp = tk->cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_nsec = tk->xtime_nsec;
vdso_data->wtom_clock_sec =
Index: tip/arch/tile/kernel/time.c
===================================================================
--- tip.orig/arch/tile/kernel/time.c
+++ tip/arch/tile/kernel/time.c
@@ -269,7 +269,7 @@ void update_vsyscall(struct timekeeper *
/* Userspace gettimeofday will spin while this value is odd. */
++vdso_data->tb_update_count;
smp_wmb();
- vdso_data->xtime_tod_stamp = clock->cycle_last;
+ vdso_data->xtime_tod_stamp = tk->cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_nsec = tk->xtime_nsec;
vdso_data->wtom_clock_sec = wtm->tv_sec;
Index: tip/arch/x86/kernel/vsyscall_gtod.c
===================================================================
--- tip.orig/arch/x86/kernel/vsyscall_gtod.c
+++ tip/arch/x86/kernel/vsyscall_gtod.c
@@ -32,7 +32,7 @@ void update_vsyscall(struct timekeeper *
/* copy vsyscall data */
vdata->vclock_mode = tk->clock->archdata.vclock_mode;
- vdata->cycle_last = tk->clock->cycle_last;
+ vdata->cycle_last = tk->cycle_last;
vdata->mask = tk->clock->mask;
vdata->mult = tk->mult;
vdata->shift = tk->shift;
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -1001,7 +1001,7 @@ static void update_pvclock_gtod(struct t
/* copy pvclock gtod data */
vdata->clock.vclock_mode = tk->clock->archdata.vclock_mode;
- vdata->clock.cycle_last = tk->clock->cycle_last;
+ vdata->clock.cycle_last = tk->cycle_last;
vdata->clock.mask = tk->clock->mask;
vdata->clock.mult = tk->mult;
vdata->clock.shift = tk->shift;
Index: tip/include/linux/clocksource.h
===================================================================
--- tip.orig/include/linux/clocksource.h
+++ tip/include/linux/clocksource.h
@@ -162,7 +162,6 @@ extern u64 timecounter_cyc2time(struct t
* @archdata: arch-specific data
* @suspend: suspend function for the clocksource, if necessary
* @resume: resume function for the clocksource, if necessary
- * @cycle_last: most recent cycle counter value seen by ::read()
* @owner: module reference, must be set by clocksource in modules
*/
struct clocksource {
@@ -171,7 +170,6 @@ struct clocksource {
* clocksource itself is cacheline aligned.
*/
cycle_t (*read)(struct clocksource *cs);
- cycle_t cycle_last;
cycle_t mask;
u32 mult;
u32 shift;
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -29,6 +29,8 @@
struct timekeeper {
/* Current clocksource used for timekeeping. */
struct clocksource *clock;
+ /* Last cycle value */
+ cycle_t cycle_last;
/* NTP adjusted clock multiplier */
u32 mult;
/* The shift value of the current clocksource. */
@@ -62,8 +64,6 @@ struct timekeeper {
/* Number of clock cycles in one NTP interval. */
cycle_t cycle_interval;
- /* Last cycle value (also stored in clock->cycle_last) */
- cycle_t cycle_last;
/* Number of clock shifted nano seconds in one NTP interval. */
u64 xtime_interval;
/* shifted nano seconds left over when rounding cycle_interval */
@@ -91,7 +91,8 @@ extern void update_vsyscall_tz(void);
#elif defined(CONFIG_GENERIC_TIME_VSYSCALL_OLD)
extern void update_vsyscall_old(struct timespec *ts, struct timespec *wtm,
- struct clocksource *c, u32 mult);
+ struct clocksource *c, u32 mult,
+ cycles_t cycle_last);
extern void update_vsyscall_tz(void);
#else
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -121,7 +121,7 @@ static void tk_setup_internals(struct ti
old_clock = tk->clock;
tk->clock = clock;
- tk->cycle_last = clock->cycle_last = clock->read(clock);
+ tk->cycle_last = clock->read(clock);
/* Do the ns -> cycle conversion first, using original mult */
tmp = NTP_INTERVAL_LENGTH;
@@ -182,7 +182,7 @@ static inline s64 timekeeping_get_ns(str
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
nsec = delta * tk->mult + tk->xtime_nsec;
nsec >>= tk->shift;
@@ -202,7 +202,7 @@ static inline s64 timekeeping_get_ns_raw
cycle_now = clock->read(clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
/* convert delta to nanoseconds. */
nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
@@ -218,7 +218,8 @@ static inline void update_vsyscall(struc
struct timespec xt;
xt = tk_xtime(tk);
- update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult);
+ update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult,
+ tk->cycle_last);
}
static inline void old_vsyscall_fixup(struct timekeeper *tk)
@@ -342,8 +343,8 @@ static void timekeeping_forward_now(stru
clock = tk->clock;
cycle_now = clock->read(clock);
- delta = clocksource_delta(cycle_now, clock->cycle_last, clock->mask);
- tk->cycle_last = clock->cycle_last = cycle_now;
+ delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
+ tk->cycle_last = cycle_now;
tk->xtime_nsec += delta * tk->mult;
@@ -1020,13 +1021,13 @@ static void timekeeping_resume(void)
*/
cycle_now = clock->read(clock);
if ((clock->flags & CLOCK_SOURCE_SUSPEND_NONSTOP) &&
- cycle_now > clock->cycle_last) {
+ cycle_now > tk->cycle_last) {
u64 num, max = ULLONG_MAX;
u32 mult = clock->mult;
u32 shift = clock->shift;
s64 nsec = 0;
- cycle_delta = clocksource_delta(cycle_now, clock->cycle_last,
+ cycle_delta = clocksource_delta(cycle_now, tk->cycle_last,
clock->mask);
/*
@@ -1053,7 +1054,7 @@ static void timekeeping_resume(void)
__timekeeping_inject_sleeptime(tk, &ts_delta);
/* Re-base the last cycle value */
- tk->cycle_last = clock->cycle_last = cycle_now;
+ tk->cycle_last = cycle_now;
tk->ntp_error = 0;
timekeeping_suspended = 0;
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -1433,7 +1434,7 @@ void update_wall_time(void)
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = clocksource_delta(clock->read(clock), clock->cycle_last,
+ offset = clocksource_delta(clock->read(clock), tk->cycle_last,
clock->mask);
#endif
@@ -1477,8 +1478,6 @@ void update_wall_time(void)
clock_set |= accumulate_nsecs_to_secs(tk);
write_seqcount_begin(&tk_core.seq);
- /* Update clock->cycle_last with the new value */
- clock->cycle_last = tk->cycle_last;
/*
* Update the real timekeeper.
*
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 58/64] timekeeping: Restructure the timekeeper some more
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (56 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 57/64] clocksource: Get rid of cycle_last Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 59/64] timekeeping: Create struct tk_read_base and use it in struct timekeeper Thomas Gleixner
` (6 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
[-- Attachment #1: timekeeping-restructure-some-more.patch --]
[-- Type: text/plain, Size: 5357 bytes --]
Access to time requires to touch two cachelines at minimum
1) The timekeeper data structure
2) The clocksource data structure
The access to the clocksource data structure can be avoided as almost
all clocksource implementations ignore the argument to the read
callback, which is a pointer to the clocksource.
But the core needs to touch it to access the members @read and @mask.
So we are better off by copying the @read function pointer and the
@mask from the clocksource to the core data structure itself.
For the most used ktime_get() access all required data including the
@read and @mask copies fits together with the sequence counter into a
single 64 byte cacheline.
For the other time access functions we touch in the current code three
cache lines in the worst case. But with the clocksource data copies we
can reduce that to two adjacent cachelines, which is more efficient
than disjunct cache lines.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/timekeeper_internal.h | 4 ++++
kernel/time/timekeeping.c | 35 +++++++++++++++--------------------
2 files changed, 19 insertions(+), 20 deletions(-)
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -29,6 +29,10 @@
struct timekeeper {
/* Current clocksource used for timekeeping. */
struct clocksource *clock;
+ /* Read function of @clock */
+ cycle_t (*read)(struct clocksource *cs);
+ /* Bitmask for two's complement subtraction of non 64bit counters */
+ cycle_t mask;
/* Last cycle value */
cycle_t cycle_last;
/* NTP adjusted clock multiplier */
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -121,7 +121,9 @@ static void tk_setup_internals(struct ti
old_clock = tk->clock;
tk->clock = clock;
- tk->cycle_last = clock->read(clock);
+ tk->read = clock->read;
+ tk->mask = clock->mask;
+ tk->cycle_last = tk->read(clock);
/* Do the ns -> cycle conversion first, using original mult */
tmp = NTP_INTERVAL_LENGTH;
@@ -174,15 +176,13 @@ static inline u32 arch_gettimeoffset(voi
static inline s64 timekeeping_get_ns(struct timekeeper *tk)
{
cycle_t cycle_now, delta;
- struct clocksource *clock;
s64 nsec;
/* read clocksource: */
- clock = tk->clock;
- cycle_now = clock->read(clock);
+ cycle_now = tk->read(tk->clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
nsec = delta * tk->mult + tk->xtime_nsec;
nsec >>= tk->shift;
@@ -193,16 +193,15 @@ static inline s64 timekeeping_get_ns(str
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
{
+ struct clocksource *clock = tk->clock;
cycle_t cycle_now, delta;
- struct clocksource *clock;
s64 nsec;
/* read clocksource: */
- clock = tk->clock;
- cycle_now = clock->read(clock);
+ cycle_now = tk->read(clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
/* convert delta to nanoseconds. */
nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
@@ -337,13 +336,12 @@ static void timekeeping_update(struct ti
*/
static void timekeeping_forward_now(struct timekeeper *tk)
{
+ struct clocksource *clock = tk->clock;
cycle_t cycle_now, delta;
- struct clocksource *clock;
s64 nsec;
- clock = tk->clock;
- cycle_now = clock->read(clock);
- delta = clocksource_delta(cycle_now, tk->cycle_last, clock->mask);
+ cycle_now = tk->read(clock);
+ delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
tk->cycle_last = cycle_now;
tk->xtime_nsec += delta * tk->mult;
@@ -1019,7 +1017,7 @@ static void timekeeping_resume(void)
* The less preferred source will only be tried if there is no better
* usable source. The rtc part is handled separately in rtc core code.
*/
- cycle_now = clock->read(clock);
+ cycle_now = tk->read(clock);
if ((clock->flags & CLOCK_SOURCE_SUSPEND_NONSTOP) &&
cycle_now > tk->cycle_last) {
u64 num, max = ULLONG_MAX;
@@ -1028,7 +1026,7 @@ static void timekeeping_resume(void)
s64 nsec = 0;
cycle_delta = clocksource_delta(cycle_now, tk->cycle_last,
- clock->mask);
+ tk->mask);
/*
* "cycle_delta * mutl" may cause 64 bits overflow, if the
@@ -1415,7 +1413,6 @@ static cycle_t logarithmic_accumulation(
*/
void update_wall_time(void)
{
- struct clocksource *clock;
struct timekeeper *real_tk = &tk_core.timekeeper;
struct timekeeper *tk = &shadow_timekeeper;
cycle_t offset;
@@ -1429,13 +1426,11 @@ void update_wall_time(void)
if (unlikely(timekeeping_suspended))
goto out;
- clock = real_tk->clock;
-
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = clocksource_delta(clock->read(clock), tk->cycle_last,
- clock->mask);
+ offset = clocksource_delta(tk->read(tk->clock), tk->cycle_last,
+ tk->mask);
#endif
/* Check if there's really nothing to do */
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 59/64] timekeeping: Create struct tk_read_base and use it in struct timekeeper
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (57 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 58/64] timekeeping: Restructure the timekeeper some more Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 60/64] timekeeping: Use tk_read_base as argument for timekeeping_get_ns() Thomas Gleixner
` (5 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Ingo Molnar, Mathieu Desnoyers
[-- Attachment #1: timekeeping-substruct-timekeeper.patch --]
[-- Type: text/plain, Size: 24947 bytes --]
The members of the new struct are the required ones for the new NMI
safe accessor to clcok monotonic. In order to reuse the existing
timekeeping code and to make the update of the fast NMI safe
timekeepers a simple memcpy use the struct for the timekeeper as well
and convert all users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
arch/arm64/kernel/vdso.c | 10 +-
arch/s390/kernel/time.c | 16 ++--
arch/tile/kernel/time.c | 10 +-
arch/x86/kernel/vsyscall_gtod.c | 23 +++---
arch/x86/kvm/x86.c | 14 +--
include/linux/timekeeper_internal.h | 103 +++++++++++++++-------------
kernel/time/timekeeping.c | 132 ++++++++++++++++++------------------
7 files changed, 158 insertions(+), 150 deletions(-)
Index: tip/arch/arm64/kernel/vdso.c
===================================================================
--- tip.orig/arch/arm64/kernel/vdso.c
+++ tip/arch/arm64/kernel/vdso.c
@@ -211,7 +211,7 @@ struct vm_area_struct *get_gate_vma(stru
void update_vsyscall(struct timekeeper *tk)
{
struct timespec xtime_coarse;
- u32 use_syscall = strcmp(tk->clock->name, "arch_sys_counter");
+ u32 use_syscall = strcmp(tk->tkr.clock->name, "arch_sys_counter");
++vdso_data->tb_seq_count;
smp_wmb();
@@ -224,11 +224,11 @@ void update_vsyscall(struct timekeeper *
vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
if (!use_syscall) {
- vdso_data->cs_cycle_last = tk->cycle_last;
+ vdso_data->cs_cycle_last = tk->tkr.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->xtime_nsec;
- vdso_data->cs_mult = tk->mult;
- vdso_data->cs_shift = tk->shift;
+ vdso_data->xtime_clock_nsec = tk->tkr.xtime_nsec;
+ vdso_data->cs_mult = tk->tkr.mult;
+ vdso_data->cs_shift = tk->tkr.shift;
}
smp_wmb();
Index: tip/arch/s390/kernel/time.c
===================================================================
--- tip.orig/arch/s390/kernel/time.c
+++ tip/arch/s390/kernel/time.c
@@ -214,26 +214,26 @@ void update_vsyscall(struct timekeeper *
{
u64 nsecps;
- if (tk->clock != &clocksource_tod)
+ if (tk->tkr.clock != &clocksource_tod)
return;
/* Make userspace gettimeofday spin until we're done. */
++vdso_data->tb_update_count;
smp_wmb();
- vdso_data->xtime_tod_stamp = tk->cycle_last;
+ vdso_data->xtime_tod_stamp = tk->tkr.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->xtime_nsec;
+ vdso_data->xtime_clock_nsec = tk->tkr.xtime_nsec;
vdso_data->wtom_clock_sec =
tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- vdso_data->wtom_clock_nsec = tk->xtime_nsec +
- + ((u64) tk->wall_to_monotonic.tv_nsec << tk->shift);
- nsecps = (u64) NSEC_PER_SEC << tk->shift;
+ vdso_data->wtom_clock_nsec = tk->tkr.xtime_nsec +
+ + ((u64) tk->wall_to_monotonic.tv_nsec << tk->tkr.shift);
+ nsecps = (u64) NSEC_PER_SEC << tk->tkr.shift;
while (vdso_data->wtom_clock_nsec >= nsecps) {
vdso_data->wtom_clock_nsec -= nsecps;
vdso_data->wtom_clock_sec++;
}
- vdso_data->tk_mult = tk->mult;
- vdso_data->tk_shift = tk->shift;
+ vdso_data->tk_mult = tk->tkr.mult;
+ vdso_data->tk_shift = tk->tkr.shift;
smp_wmb();
++vdso_data->tb_update_count;
}
Index: tip/arch/tile/kernel/time.c
===================================================================
--- tip.orig/arch/tile/kernel/time.c
+++ tip/arch/tile/kernel/time.c
@@ -261,7 +261,7 @@ void update_vsyscall_tz(void)
void update_vsyscall(struct timekeeper *tk)
{
struct timespec *wtm = &tk->wall_to_monotonic;
- struct clocksource *clock = tk->clock;
+ struct clocksource *clock = tk->tkr.clock;
if (clock != &cycle_counter_cs)
return;
@@ -269,13 +269,13 @@ void update_vsyscall(struct timekeeper *
/* Userspace gettimeofday will spin while this value is odd. */
++vdso_data->tb_update_count;
smp_wmb();
- vdso_data->xtime_tod_stamp = tk->cycle_last;
+ vdso_data->xtime_tod_stamp = tk->tkr.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->xtime_nsec;
+ vdso_data->xtime_clock_nsec = tk->tkr.xtime_nsec;
vdso_data->wtom_clock_sec = wtm->tv_sec;
vdso_data->wtom_clock_nsec = wtm->tv_nsec;
- vdso_data->mult = tk->mult;
- vdso_data->shift = tk->shift;
+ vdso_data->mult = tk->tkr.mult;
+ vdso_data->shift = tk->tkr.shift;
smp_wmb();
++vdso_data->tb_update_count;
}
Index: tip/arch/x86/kernel/vsyscall_gtod.c
===================================================================
--- tip.orig/arch/x86/kernel/vsyscall_gtod.c
+++ tip/arch/x86/kernel/vsyscall_gtod.c
@@ -31,29 +31,30 @@ void update_vsyscall(struct timekeeper *
gtod_write_begin(vdata);
/* copy vsyscall data */
- vdata->vclock_mode = tk->clock->archdata.vclock_mode;
- vdata->cycle_last = tk->cycle_last;
- vdata->mask = tk->clock->mask;
- vdata->mult = tk->mult;
- vdata->shift = tk->shift;
+ vdata->vclock_mode = tk->tkr.clock->archdata.vclock_mode;
+ vdata->cycle_last = tk->tkr.cycle_last;
+ vdata->mask = tk->tkr.mask;
+ vdata->mult = tk->tkr.mult;
+ vdata->shift = tk->tkr.shift;
vdata->wall_time_sec = tk->xtime_sec;
- vdata->wall_time_snsec = tk->xtime_nsec;
+ vdata->wall_time_snsec = tk->tkr.xtime_nsec;
vdata->monotonic_time_sec = tk->xtime_sec
+ tk->wall_to_monotonic.tv_sec;
- vdata->monotonic_time_snsec = tk->xtime_nsec
+ vdata->monotonic_time_snsec = tk->tkr.xtime_nsec
+ ((u64)tk->wall_to_monotonic.tv_nsec
- << tk->shift);
+ << tk->tkr.shift);
while (vdata->monotonic_time_snsec >=
- (((u64)NSEC_PER_SEC) << tk->shift)) {
+ (((u64)NSEC_PER_SEC) << tk->tkr.shift)) {
vdata->monotonic_time_snsec -=
- ((u64)NSEC_PER_SEC) << tk->shift;
+ ((u64)NSEC_PER_SEC) << tk->tkr.shift;
vdata->monotonic_time_sec++;
}
vdata->wall_time_coarse_sec = tk->xtime_sec;
- vdata->wall_time_coarse_nsec = (long)(tk->xtime_nsec >> tk->shift);
+ vdata->wall_time_coarse_nsec = (long)(tk->tkr.xtime_nsec >>
+ tk->tkr.shift);
vdata->monotonic_time_coarse_sec =
vdata->wall_time_coarse_sec + tk->wall_to_monotonic.tv_sec;
Index: tip/arch/x86/kvm/x86.c
===================================================================
--- tip.orig/arch/x86/kvm/x86.c
+++ tip/arch/x86/kvm/x86.c
@@ -995,19 +995,19 @@ static void update_pvclock_gtod(struct t
struct pvclock_gtod_data *vdata = &pvclock_gtod_data;
u64 boot_ns;
- boot_ns = ktime_to_ns(ktime_add(tk->base_mono, tk->offs_boot));
+ boot_ns = ktime_to_ns(ktime_add(tk->tkr.base_mono, tk->offs_boot));
write_seqcount_begin(&vdata->seq);
/* copy pvclock gtod data */
- vdata->clock.vclock_mode = tk->clock->archdata.vclock_mode;
- vdata->clock.cycle_last = tk->cycle_last;
- vdata->clock.mask = tk->clock->mask;
- vdata->clock.mult = tk->mult;
- vdata->clock.shift = tk->shift;
+ vdata->clock.vclock_mode = tk->tkr.clock->archdata.vclock_mode;
+ vdata->clock.cycle_last = tk->tkr.cycle_last;
+ vdata->clock.mask = tk->tkr.mask;
+ vdata->clock.mult = tk->tkr.mult;
+ vdata->clock.shift = tk->tkr.shift;
vdata->boot_ns = boot_ns;
- vdata->nsec_base = tk->xtime_nsec;
+ vdata->nsec_base = tk->tkr.xtime_nsec;
write_seqcount_end(&vdata->seq);
}
Index: tip/include/linux/timekeeper_internal.h
===================================================================
--- tip.orig/include/linux/timekeeper_internal.h
+++ tip/include/linux/timekeeper_internal.h
@@ -10,80 +10,87 @@
#include <linux/jiffies.h>
#include <linux/time.h>
-/*
- * Structure holding internal timekeeping values.
+/**
+ * struct tk_read_base - base structure for timekeeping readout
+ * @clock: Current clocksource used for timekeeping.
+ * @read: Read function of @clock
+ * @mask: Bitmask for two's complement subtraction of non 64bit clocks
+ * @cycle_last: @clock cycle value at last update
+ * @mult: NTP adjusted multiplier for scaled math conversion
+ * @shift: Shift value for scaled math conversion
+ * @xtime_nsec: Shifted (fractional) nano seconds offset for readout
+ * @base_mono: ktime_t (nanoseconds) base time for readout
*
- * Note: wall_to_monotonic is what we need to add to xtime (or xtime
- * corrected for sub jiffie times) to get to monotonic time.
- * Monotonic is pegged at zero at system boot time, so
- * wall_to_monotonic will be negative, however, we will ALWAYS keep
- * the tv_nsec part positive so we can use the usual normalization.
+ * This struct has size 56 byte on 64 bit. Together with a seqcount it
+ * occupies a single 64byte cache line.
*
- * wall_to_monotonic is moved after resume from suspend for the
- * monotonic time not to jump. To calculate the real boot time offset
- * we need to do offs_real - offs_boot.
- *
- * - wall_to_monotonic is no longer the boot time, getboottime must be
- * used instead.
+ * The struct is separate from struct timekeeper as it is also used
+ * for a fast NMI safe accessor to clock monotonic.
*/
-struct timekeeper {
- /* Current clocksource used for timekeeping. */
+struct tk_read_base {
struct clocksource *clock;
- /* Read function of @clock */
cycle_t (*read)(struct clocksource *cs);
- /* Bitmask for two's complement subtraction of non 64bit counters */
cycle_t mask;
- /* Last cycle value */
cycle_t cycle_last;
- /* NTP adjusted clock multiplier */
u32 mult;
- /* The shift value of the current clocksource. */
u32 shift;
- /* Clock shifted nano seconds */
u64 xtime_nsec;
-
- /* Monotonic base time */
ktime_t base_mono;
+};
- /* Current CLOCK_REALTIME time in seconds */
+/**
+ * struct timekeeper - Structure holding internal timekeeping values.
+ * @tkr: The readout base structure
+ * @xtime_sec: Current CLOCK_REALTIME time in seconds
+ * @wall_to_monotonic: CLOCK_REALTIME to CLOCK_MONOTONIC offset
+ * @offs_real: Offset clock monotonic -> clock realtime
+ * @offs_boot: Offset clock monotonic -> clock boottime
+ * @offs_tai: Offset clock monotonic -> clock tai
+ * @tai_offset: The current UTC to TAI offset in seconds
+ * @base_raw: Monotonic raw base time in ktime_t format
+ * @raw_time: Monotonic raw base time in timespec64 format
+ * @cycle_interval: Number of clock cycles in one NTP interval
+ * @xtime_interval: Number of clock shifted nano seconds in one NTP
+ * interval.
+ * @xtime_remainder: Shifted nano seconds left over when rounding
+ * @cycle_interval
+ * @raw_interval: Raw nano seconds accumulated per NTP interval.
+ * @ntp_error: Difference between accumulated time and NTP time in ntp
+ * shifted nano seconds.
+ * @ntp_error_shift: Shift conversion between clock shifted nano seconds and
+ * ntp shifted nano seconds.
+ *
+ * Note: For timespec(64) based interfaces wall_to_monotonic is what
+ * we need to add to xtime (or xtime corrected for sub jiffie times)
+ * to get to monotonic time. Monotonic is pegged at zero at system
+ * boot time, so wall_to_monotonic will be negative, however, we will
+ * ALWAYS keep the tv_nsec part positive so we can use the usual
+ * normalization.
+ *
+ * wall_to_monotonic is moved after resume from suspend for the
+ * monotonic time not to jump. We need to add total_sleep_time to
+ * wall_to_monotonic to get the real boot based time offset.
+ *
+ * wall_to_monotonic is no longer the boot time, getboottime must be
+ * used instead.
+ */
+struct timekeeper {
+ struct tk_read_base tkr;
u64 xtime_sec;
- /* CLOCK_REALTIME to CLOCK_MONOTONIC offset */
struct timespec64 wall_to_monotonic;
-
- /* Offset clock monotonic -> clock realtime */
ktime_t offs_real;
- /* Offset clock monotonic -> clock boottime */
ktime_t offs_boot;
- /* Offset clock monotonic -> clock tai */
ktime_t offs_tai;
-
- /* The current UTC to TAI offset in seconds */
s32 tai_offset;
-
- /* Monotonic raw base time */
ktime_t base_raw;
-
- /* The raw monotonic time for the CLOCK_MONOTONIC_RAW posix clock. */
struct timespec64 raw_time;
- /* Number of clock cycles in one NTP interval. */
+ /* The following members are for timekeeping internal use */
cycle_t cycle_interval;
- /* Number of clock shifted nano seconds in one NTP interval. */
u64 xtime_interval;
- /* shifted nano seconds left over when rounding cycle_interval */
s64 xtime_remainder;
- /* Raw nano seconds accumulated per NTP interval. */
u32 raw_interval;
-
- /*
- * Difference between accumulated time and NTP time in ntp
- * shifted nano seconds.
- */
s64 ntp_error;
- /*
- * Shift conversion between clock shifted nano seconds and
- * ntp shifted nano seconds.
- */
u32 ntp_error_shift;
};
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -52,8 +52,8 @@ bool __read_mostly persistent_clock_exis
static inline void tk_normalize_xtime(struct timekeeper *tk)
{
- while (tk->xtime_nsec >= ((u64)NSEC_PER_SEC << tk->shift)) {
- tk->xtime_nsec -= (u64)NSEC_PER_SEC << tk->shift;
+ while (tk->tkr.xtime_nsec >= ((u64)NSEC_PER_SEC << tk->tkr.shift)) {
+ tk->tkr.xtime_nsec -= (u64)NSEC_PER_SEC << tk->tkr.shift;
tk->xtime_sec++;
}
}
@@ -63,20 +63,20 @@ static inline struct timespec64 tk_xtime
struct timespec64 ts;
ts.tv_sec = tk->xtime_sec;
- ts.tv_nsec = (long)(tk->xtime_nsec >> tk->shift);
+ ts.tv_nsec = (long)(tk->tkr.xtime_nsec >> tk->tkr.shift);
return ts;
}
static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
- tk->xtime_nsec = (u64)ts->tv_nsec << tk->shift;
+ tk->tkr.xtime_nsec = (u64)ts->tv_nsec << tk->tkr.shift;
}
static void tk_xtime_add(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec += ts->tv_sec;
- tk->xtime_nsec += (u64)ts->tv_nsec << tk->shift;
+ tk->tkr.xtime_nsec += (u64)ts->tv_nsec << tk->tkr.shift;
tk_normalize_xtime(tk);
}
@@ -119,11 +119,11 @@ static void tk_setup_internals(struct ti
u64 tmp, ntpinterval;
struct clocksource *old_clock;
- old_clock = tk->clock;
- tk->clock = clock;
- tk->read = clock->read;
- tk->mask = clock->mask;
- tk->cycle_last = tk->read(clock);
+ old_clock = tk->tkr.clock;
+ tk->tkr.clock = clock;
+ tk->tkr.read = clock->read;
+ tk->tkr.mask = clock->mask;
+ tk->tkr.cycle_last = tk->tkr.read(clock);
/* Do the ns -> cycle conversion first, using original mult */
tmp = NTP_INTERVAL_LENGTH;
@@ -147,11 +147,11 @@ static void tk_setup_internals(struct ti
if (old_clock) {
int shift_change = clock->shift - old_clock->shift;
if (shift_change < 0)
- tk->xtime_nsec >>= -shift_change;
+ tk->tkr.xtime_nsec >>= -shift_change;
else
- tk->xtime_nsec <<= shift_change;
+ tk->tkr.xtime_nsec <<= shift_change;
}
- tk->shift = clock->shift;
+ tk->tkr.shift = clock->shift;
tk->ntp_error = 0;
tk->ntp_error_shift = NTP_SCALE_SHIFT - clock->shift;
@@ -161,7 +161,7 @@ static void tk_setup_internals(struct ti
* active clocksource. These value will be adjusted via NTP
* to counteract clock drifting.
*/
- tk->mult = clock->mult;
+ tk->tkr.mult = clock->mult;
}
/* Timekeeper helper functions. */
@@ -179,13 +179,13 @@ static inline s64 timekeeping_get_ns(str
s64 nsec;
/* read clocksource: */
- cycle_now = tk->read(tk->clock);
+ cycle_now = tk->tkr.read(tk->tkr.clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
+ delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
- nsec = delta * tk->mult + tk->xtime_nsec;
- nsec >>= tk->shift;
+ nsec = delta * tk->tkr.mult + tk->tkr.xtime_nsec;
+ nsec >>= tk->tkr.shift;
/* If arch requires, add in get_arch_timeoffset() */
return nsec + arch_gettimeoffset();
@@ -193,15 +193,15 @@ static inline s64 timekeeping_get_ns(str
static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
{
- struct clocksource *clock = tk->clock;
+ struct clocksource *clock = tk->tkr.clock;
cycle_t cycle_now, delta;
s64 nsec;
/* read clocksource: */
- cycle_now = tk->read(clock);
+ cycle_now = tk->tkr.read(clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
+ delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
/* convert delta to nanoseconds. */
nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
@@ -217,8 +217,8 @@ static inline void update_vsyscall(struc
struct timespec xt;
xt = tk_xtime(tk);
- update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->clock, tk->mult,
- tk->cycle_last);
+ update_vsyscall_old(&xt, &tk->wall_to_monotonic, tk->tkr.clock, tk->tkr.mult,
+ tk->tkr.cycle_last);
}
static inline void old_vsyscall_fixup(struct timekeeper *tk)
@@ -235,11 +235,11 @@ static inline void old_vsyscall_fixup(st
* (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
* users are removed, this can be killed.
*/
- remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
- tk->xtime_nsec -= remainder;
- tk->xtime_nsec += 1ULL << tk->shift;
+ remainder = tk->tkr.xtime_nsec & ((1ULL << tk->tkr.shift) - 1);
+ tk->tkr.xtime_nsec -= remainder;
+ tk->tkr.xtime_nsec += 1ULL << tk->tkr.shift;
tk->ntp_error += remainder << tk->ntp_error_shift;
- tk->ntp_error -= (1ULL << tk->shift) << tk->ntp_error_shift;
+ tk->ntp_error -= (1ULL << tk->tkr.shift) << tk->ntp_error_shift;
}
#else
#define old_vsyscall_fixup(tk)
@@ -304,7 +304,7 @@ static inline void tk_update_ktime_data(
nsec = (s64)(tk->xtime_sec + tk->wall_to_monotonic.tv_sec);
nsec *= NSEC_PER_SEC;
nsec += tk->wall_to_monotonic.tv_nsec;
- tk->base_mono = ns_to_ktime(nsec);
+ tk->tkr.base_mono = ns_to_ktime(nsec);
/* Update the monotonic raw base */
tk->base_raw = timespec64_to_ktime(tk->raw_time);
@@ -336,18 +336,18 @@ static void timekeeping_update(struct ti
*/
static void timekeeping_forward_now(struct timekeeper *tk)
{
- struct clocksource *clock = tk->clock;
+ struct clocksource *clock = tk->tkr.clock;
cycle_t cycle_now, delta;
s64 nsec;
- cycle_now = tk->read(clock);
- delta = clocksource_delta(cycle_now, tk->cycle_last, tk->mask);
- tk->cycle_last = cycle_now;
+ cycle_now = tk->tkr.read(clock);
+ delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
+ tk->tkr.cycle_last = cycle_now;
- tk->xtime_nsec += delta * tk->mult;
+ tk->tkr.xtime_nsec += delta * tk->tkr.mult;
/* If arch requires, add in get_arch_timeoffset() */
- tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
+ tk->tkr.xtime_nsec += (u64)arch_gettimeoffset() << tk->tkr.shift;
tk_normalize_xtime(tk);
@@ -412,7 +412,7 @@ ktime_t ktime_get(void)
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->base_mono;
+ base = tk->tkr.base_mono;
nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -438,7 +438,7 @@ ktime_t ktime_get_with_offset(enum tk_of
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = ktime_add(tk->base_mono, *offset);
+ base = ktime_add(tk->tkr.base_mono, *offset);
nsecs = timekeeping_get_ns(tk);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -731,7 +731,7 @@ static int change_clocksource(void *data
*/
if (try_module_get(new->owner)) {
if (!new->enable || new->enable(new) == 0) {
- old = tk->clock;
+ old = tk->tkr.clock;
tk_setup_internals(tk, new);
if (old->disable)
old->disable(old);
@@ -759,11 +759,11 @@ int timekeeping_notify(struct clocksourc
{
struct timekeeper *tk = &tk_core.timekeeper;
- if (tk->clock == clock)
+ if (tk->tkr.clock == clock)
return 0;
stop_machine(change_clocksource, clock, NULL);
tick_clock_notify();
- return tk->clock == clock ? 0 : -1;
+ return tk->tkr.clock == clock ? 0 : -1;
}
/**
@@ -803,7 +803,7 @@ int timekeeping_valid_for_hres(void)
do {
seq = read_seqcount_begin(&tk_core.seq);
- ret = tk->clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
+ ret = tk->tkr.clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -822,7 +822,7 @@ u64 timekeeping_max_deferment(void)
do {
seq = read_seqcount_begin(&tk_core.seq);
- ret = tk->clock->max_idle_ns;
+ ret = tk->tkr.clock->max_idle_ns;
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -989,7 +989,7 @@ void timekeeping_inject_sleeptime(struct
static void timekeeping_resume(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
- struct clocksource *clock = tk->clock;
+ struct clocksource *clock = tk->tkr.clock;
unsigned long flags;
struct timespec64 ts_new, ts_delta;
struct timespec tmp;
@@ -1017,16 +1017,16 @@ static void timekeeping_resume(void)
* The less preferred source will only be tried if there is no better
* usable source. The rtc part is handled separately in rtc core code.
*/
- cycle_now = tk->read(clock);
+ cycle_now = tk->tkr.read(clock);
if ((clock->flags & CLOCK_SOURCE_SUSPEND_NONSTOP) &&
- cycle_now > tk->cycle_last) {
+ cycle_now > tk->tkr.cycle_last) {
u64 num, max = ULLONG_MAX;
u32 mult = clock->mult;
u32 shift = clock->shift;
s64 nsec = 0;
- cycle_delta = clocksource_delta(cycle_now, tk->cycle_last,
- tk->mask);
+ cycle_delta = clocksource_delta(cycle_now, tk->tkr.cycle_last,
+ tk->tkr.mask);
/*
* "cycle_delta * mutl" may cause 64 bits overflow, if the
@@ -1052,7 +1052,7 @@ static void timekeeping_resume(void)
__timekeeping_inject_sleeptime(tk, &ts_delta);
/* Re-base the last cycle value */
- tk->cycle_last = cycle_now;
+ tk->tkr.cycle_last = cycle_now;
tk->ntp_error = 0;
timekeeping_suspended = 0;
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
@@ -1239,12 +1239,12 @@ static void timekeeping_adjust(struct ti
}
}
- if (unlikely(tk->clock->maxadj &&
- (tk->mult + adj > tk->clock->mult + tk->clock->maxadj))) {
+ if (unlikely(tk->tkr.clock->maxadj &&
+ (tk->tkr.mult + adj > tk->tkr.clock->mult + tk->tkr.clock->maxadj))) {
printk_deferred_once(KERN_WARNING
"Adjusting %s more than 11%% (%ld vs %ld)\n",
- tk->clock->name, (long)tk->mult + adj,
- (long)tk->clock->mult + tk->clock->maxadj);
+ tk->tkr.clock->name, (long)tk->tkr.mult + adj,
+ (long)tk->tkr.clock->mult + tk->tkr.clock->maxadj);
}
/*
* So the following can be confusing.
@@ -1295,9 +1295,9 @@ static void timekeeping_adjust(struct ti
*
* XXX - TODO: Doc ntp_error calculation.
*/
- tk->mult += adj;
+ tk->tkr.mult += adj;
tk->xtime_interval += interval;
- tk->xtime_nsec -= offset;
+ tk->tkr.xtime_nsec -= offset;
tk->ntp_error -= (interval - offset) << tk->ntp_error_shift;
out_adjust:
@@ -1315,9 +1315,9 @@ out_adjust:
* We'll correct this error next time through this function, when
* xtime_nsec is not as small.
*/
- if (unlikely((s64)tk->xtime_nsec < 0)) {
- s64 neg = -(s64)tk->xtime_nsec;
- tk->xtime_nsec = 0;
+ if (unlikely((s64)tk->tkr.xtime_nsec < 0)) {
+ s64 neg = -(s64)tk->tkr.xtime_nsec;
+ tk->tkr.xtime_nsec = 0;
tk->ntp_error += neg << tk->ntp_error_shift;
}
@@ -1333,13 +1333,13 @@ out_adjust:
*/
static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk)
{
- u64 nsecps = (u64)NSEC_PER_SEC << tk->shift;
+ u64 nsecps = (u64)NSEC_PER_SEC << tk->tkr.shift;
unsigned int clock_set = 0;
- while (tk->xtime_nsec >= nsecps) {
+ while (tk->tkr.xtime_nsec >= nsecps) {
int leap;
- tk->xtime_nsec -= nsecps;
+ tk->tkr.xtime_nsec -= nsecps;
tk->xtime_sec++;
/* Figure out if its a leap sec and apply if needed */
@@ -1384,9 +1384,9 @@ static cycle_t logarithmic_accumulation(
/* Accumulate one shifted interval */
offset -= interval;
- tk->cycle_last += interval;
+ tk->tkr.cycle_last += interval;
- tk->xtime_nsec += tk->xtime_interval << shift;
+ tk->tkr.xtime_nsec += tk->xtime_interval << shift;
*clock_set |= accumulate_nsecs_to_secs(tk);
/* Accumulate raw time */
@@ -1429,8 +1429,8 @@ void update_wall_time(void)
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = clocksource_delta(tk->read(tk->clock), tk->cycle_last,
- tk->mask);
+ offset = clocksource_delta(tk->tkr.read(tk->tkr.clock),
+ tk->tkr.cycle_last, tk->tkr.mask);
#endif
/* Check if there's really nothing to do */
@@ -1591,8 +1591,8 @@ ktime_t ktime_get_update_offsets_tick(kt
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->base_mono;
- nsecs = tk->xtime_nsec >> tk->shift;
+ base = tk->tkr.base_mono;
+ nsecs = tk->tkr.xtime_nsec >> tk->tkr.shift;
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
@@ -1623,7 +1623,7 @@ ktime_t ktime_get_update_offsets_now(kti
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->base_mono;
+ base = tk->tkr.base_mono;
nsecs = timekeeping_get_ns(tk);
*offs_real = tk->offs_real;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 60/64] timekeeping: Use tk_read_base as argument for timekeeping_get_ns()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (58 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 59/64] timekeeping: Create struct tk_read_base and use it in struct timekeeper Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 61/64] seqcount: Provide raw_read_seqcount() Thomas Gleixner
` (4 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: timekeeping-change-get-ns-arg-to-tkr.patch --]
[-- Type: text/plain, Size: 3227 bytes --]
All the function needs is in the tk_read_base struct. No functional
change for the current code, just a preparatory patch for the NMI safe
accessor to clock monotonic which will use struct tk_read_base as well.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/time/timekeeping.c | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -173,19 +173,19 @@ u32 (*arch_gettimeoffset)(void) = defaul
static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
-static inline s64 timekeeping_get_ns(struct timekeeper *tk)
+static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
cycle_t cycle_now, delta;
s64 nsec;
/* read clocksource: */
- cycle_now = tk->tkr.read(tk->tkr.clock);
+ cycle_now = tkr->read(tkr->clock);
/* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
+ delta = clocksource_delta(cycle_now, tkr->cycle_last, tkr->mask);
- nsec = delta * tk->tkr.mult + tk->tkr.xtime_nsec;
- nsec >>= tk->tkr.shift;
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ nsec >>= tkr->shift;
/* If arch requires, add in get_arch_timeoffset() */
return nsec + arch_gettimeoffset();
@@ -372,7 +372,7 @@ int __getnstimeofday64(struct timespec64
seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
- nsecs = timekeeping_get_ns(tk);
+ nsecs = timekeeping_get_ns(&tk->tkr);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -413,7 +413,7 @@ ktime_t ktime_get(void)
do {
seq = read_seqcount_begin(&tk_core.seq);
base = tk->tkr.base_mono;
- nsecs = timekeeping_get_ns(tk);
+ nsecs = timekeeping_get_ns(&tk->tkr);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -439,7 +439,7 @@ ktime_t ktime_get_with_offset(enum tk_of
do {
seq = read_seqcount_begin(&tk_core.seq);
base = ktime_add(tk->tkr.base_mono, *offset);
- nsecs = timekeeping_get_ns(tk);
+ nsecs = timekeeping_get_ns(&tk->tkr);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -509,7 +509,7 @@ void ktime_get_ts64(struct timespec64 *t
do {
seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
- nsec = timekeeping_get_ns(tk);
+ nsec = timekeeping_get_ns(&tk->tkr);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -547,7 +547,7 @@ void getnstime_raw_and_real(struct times
ts_real->tv_nsec = 0;
nsecs_raw = timekeeping_get_ns_raw(tk);
- nsecs_real = timekeeping_get_ns(tk);
+ nsecs_real = timekeeping_get_ns(&tk->tkr);
} while (read_seqcount_retry(&tk_core.seq, seq));
@@ -1624,7 +1624,7 @@ ktime_t ktime_get_update_offsets_now(kti
seq = read_seqcount_begin(&tk_core.seq);
base = tk->tkr.base_mono;
- nsecs = timekeeping_get_ns(tk);
+ nsecs = timekeeping_get_ns(&tk->tkr);
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 61/64] seqcount: Provide raw_read_seqcount()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (59 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 60/64] timekeeping: Use tk_read_base as argument for timekeeping_get_ns() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 62/64] seqcount: Add raw_write_seqcount_latch() Thomas Gleixner
` (3 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Ingo Molnar, Mathieu Desnoyers
[-- Attachment #1: seqcount-add-raw-read-seqcount.patch --]
[-- Type: text/plain, Size: 1434 bytes --]
raw_read_seqcount opens a read critical section of the given seqcount
without any lockdep checking and without checking or masking the
LSB. Calling code is responsible for handling that.
Preparatory patch to provide a NMI safe clock monotonic accessor
function.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
include/linux/seqlock.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
Index: tip/include/linux/seqlock.h
===================================================================
--- tip.orig/include/linux/seqlock.h
+++ tip/include/linux/seqlock.h
@@ -117,6 +117,22 @@ repeat:
}
/**
+ * raw_read_seqcount - Read the raw seqcount
+ * @s: pointer to seqcount_t
+ * Returns: count to be passed to read_seqcount_retry
+ *
+ * raw_read_seqcount opens a read critical section of the given
+ * seqcount without any lockdep checking and without checking or
+ * masking the LSB. Calling code is responsible for handling that.
+ */
+static inline unsigned raw_read_seqcount(const seqcount_t *s)
+{
+ unsigned ret = ACCESS_ONCE(s->sequence);
+ smp_rmb();
+ return ret;
+}
+
+/**
* raw_read_seqcount_begin - start seq-read critical section w/o lockdep
* @s: pointer to seqcount_t
* Returns: count to be passed to read_seqcount_retry
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 62/64] seqcount: Add raw_write_seqcount_latch()
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (60 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 61/64] seqcount: Provide raw_read_seqcount() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 63/64] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC Thomas Gleixner
` (2 subsequent siblings)
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML
Cc: John Stultz, Peter Zijlstra, Mathieu Desnoyers, Ingo Molnar,
Steven Rostedt
[-- Attachment #1: seqcount-add-raw-write-seqcount-latch.patch --]
[-- Type: text/plain, Size: 1668 bytes --]
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
For NMI safe access to clock monotonic we use the seqcount LSB as
index of a timekeeper array. The update sequence looks like this:
smp_wmb(); <- prior stores to a[1]
seq++;
smp_wmb(); <- seq increment before update of a[0]
update(a[0]);
smp_wmb(); <- update of a[0]
seq++;
smp_wmb(); <- seq increment before update of a[1]
update(a[1]);
To avoid open coded barriers, provide a helper function.
[ tglx: Split out of a combo patch against the first implementation of
the NMI safe accessor ]
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/seqlock.h | 11 +++++++++++
1 file changed, 11 insertions(+)
Index: tip/include/linux/seqlock.h
===================================================================
--- tip.orig/include/linux/seqlock.h
+++ tip/include/linux/seqlock.h
@@ -236,6 +236,17 @@ static inline void raw_write_seqcount_en
}
/*
+ * raw_write_seqcount_latch - redirect readers to even/odd copy
+ * @s: pointer to seqcount_t
+ */
+static inline void raw_write_seqcount_latch(seqcount_t *s)
+{
+ smp_wmb(); /* prior stores before incrementing "sequence" */
+ s->sequence++;
+ smp_wmb(); /* increment "sequence" before following stores */
+}
+
+/*
* Sequence counter only version assumes that callers are using their
* own mutexing.
*/
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 63/64] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (61 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 62/64] seqcount: Add raw_write_seqcount_latch() Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-16 21:05 ` [patch V2 64/64] ftrace: Provide trace clocks monotonic Thomas Gleixner
2014-07-17 22:32 ` [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: timekeeping-nmi-safe-access-to-mono-raw.patch --]
[-- Type: text/plain, Size: 6837 bytes --]
Tracers want a correlated time between the kernel instrumentation and
user space. We really do not want to export sched_clock() to user
space, so we need to provide something sensible for this.
Using separate data structures with an non blocking sequence count
based update mechanism allows us to do that. The data structure
required for the readout has a sequence counter and two copies of the
timekeeping data.
On the update side:
smp_wmb();
tkf->seq++;
smp_wmb();
update(tkf->base[0], tk);
smp_wmb();
tkf->seq++;
smp_wmb();
update(tkf->base[1], tk);
On the reader side:
do {
seq = tkf->seq;
smp_rmb();
idx = seq & 0x01;
now = now(tkf->base[idx]);
smp_rmb();
} while (seq != tkf->seq)
So if a NMI hits the update of base[0] it will use base[1] which is
still consistent, but this timestamp is not guaranteed to be monotonic
across an update.
The timestamp is calculated by:
now = base_mono + clock_delta * slope
So if the update lowers the slope, readers who are forced to the
not yet updated second array are still using the old steeper slope.
tmono
^
| o n
| o n
| u
| o
|o
|12345678---> reader order
o = old slope
u = update
n = new slope
So reader 6 will observe time going backwards versus reader 5.
While other CPUs are likely to be able observe that, the only way
for a CPU local observation is when an NMI hits in the middle of
the update. Timestamps taken from that NMI context might be ahead
of the following timestamps. Callers need to be aware of that and
deal with it.
V2: Got rid of clock monotonic raw and reorganized the data
structures. Folded in the barrier fix from Mathieu.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
include/linux/timekeeping.h | 2
kernel/time/timekeeping.c | 124 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 126 insertions(+)
Index: tip/include/linux/timekeeping.h
===================================================================
--- tip.orig/include/linux/timekeeping.h
+++ tip/include/linux/timekeeping.h
@@ -164,6 +164,8 @@ static inline u64 ktime_get_raw_ns(void)
return ktime_to_ns(ktime_get_raw());
}
+extern u64 ktime_get_mono_fast_ns(void);
+
/*
* Timespec interfaces utilizing the ktime based ones
*/
Index: tip/kernel/time/timekeeping.c
===================================================================
--- tip.orig/kernel/time/timekeeping.c
+++ tip/kernel/time/timekeeping.c
@@ -44,6 +44,22 @@ static struct {
static DEFINE_RAW_SPINLOCK(timekeeper_lock);
static struct timekeeper shadow_timekeeper;
+/**
+ * struct tk_fast - NMI safe timekeeper
+ * @seq: Sequence counter for protecting updates. The lowest bit
+ * is the index for the tk_read_base array
+ * @base: tk_read_base array. Access is indexed by the lowest bit of
+ * @seq.
+ *
+ * See @update_fast_timekeeper() below.
+ */
+struct tk_fast {
+ seqcount_t seq;
+ struct tk_read_base base[2];
+};
+
+static struct tk_fast tk_fast_mono ____cacheline_aligned;
+
/* flag for if timekeeping is suspended */
int __read_mostly timekeeping_suspended;
@@ -210,6 +226,112 @@ static inline s64 timekeeping_get_ns_raw
return nsec + arch_gettimeoffset();
}
+/**
+ * update_fast_timekeeper - Update the fast and NMI safe monotonic timekeeper.
+ * @tk: The timekeeper from which we take the update
+ * @tkf: The fast timekeeper to update
+ * @tbase: The time base for the fast timekeeper (mono/raw)
+ *
+ * We want to use this from any context including NMI and tracing /
+ * instrumenting the timekeeping code itself.
+ *
+ * So we handle this differently than the other timekeeping accessor
+ * functions which retry when the sequence count has changed. The
+ * update side does:
+ *
+ * smp_wmb(); <- Ensure that the last base[1] update is visible
+ * tkf->seq++;
+ * smp_wmb(); <- Ensure that the seqcount update is visible
+ * update(tkf->base[0], tk);
+ * smp_wmb(); <- Ensure that the base[0] update is visible
+ * tkf->seq++;
+ * smp_wmb(); <- Ensure that the seqcount update is visible
+ * update(tkf->base[1], tk);
+ *
+ * The reader side does:
+ *
+ * do {
+ * seq = tkf->seq;
+ * smp_rmb();
+ * idx = seq & 0x01;
+ * now = now(tkf->base[idx]);
+ * smp_rmb();
+ * } while (seq != tkf->seq)
+ *
+ * As long as we update base[0] readers are forced off to
+ * base[1]. Once base[0] is updated readers are redirected to base[0]
+ * and the base[1] update takes place.
+ *
+ * So if a NMI hits the update of base[0] then it will use base[1]
+ * which is still consistent. In the worst case this can result is a
+ * slightly wrong timestamp (a few nanoseconds). See
+ * @ktime_get_mono_fast_ns.
+ */
+static void update_fast_timekeeper(struct timekeeper *tk)
+{
+ struct tk_read_base *base = tk_fast_mono.base;
+
+ /* Force readers off to base[1] */
+ raw_write_seqcount_latch(&tk_fast_mono.seq);
+
+ /* Update base[0] */
+ memcpy(base, &tk->tkr, sizeof(*base));
+
+ /* Force readers back to base[0] */
+ raw_write_seqcount_latch(&tk_fast_mono.seq);
+
+ /* Update base[1] */
+ memcpy(base + 1, base, sizeof(*base));
+}
+
+/**
+ * ktime_get_mono_fast_ns - Fast NMI safe access to clock monotonic
+ *
+ * This timestamp is not guaranteed to be monotonic across an update.
+ * The timestamp is calculated by:
+ *
+ * now = base_mono + clock_delta * slope
+ *
+ * So if the update lowers the slope, readers who are forced to the
+ * not yet updated second array are still using the old steeper slope.
+ *
+ * tmono
+ * ^
+ * | o n
+ * | o n
+ * | u
+ * | o
+ * |o
+ * |12345678---> reader order
+ *
+ * o = old slope
+ * u = update
+ * n = new slope
+ *
+ * So reader 6 will observe time going backwards versus reader 5.
+ *
+ * While other CPUs are likely to be able observe that, the only way
+ * for a CPU local observation is when an NMI hits in the middle of
+ * the update. Timestamps taken from that NMI context might be ahead
+ * of the following timestamps. Callers need to be aware of that and
+ * deal with it.
+ */
+u64 notrace ktime_get_mono_fast_ns(void)
+{
+ struct tk_read_base *tkr;
+ unsigned int seq;
+ u64 now;
+
+ do {
+ seq = raw_read_seqcount(&tk_fast_mono.seq);
+ tkr = tk_fast_mono.base + (seq & 0x01);
+ now = ktime_to_ns(tkr->base_mono) + timekeeping_get_ns(tkr);
+
+ } while (read_seqcount_retry(&tk_fast_mono.seq, seq));
+ return now;
+}
+EXPORT_SYMBOL_GPL(ktime_get_mono_fast_ns);
+
#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
static inline void update_vsyscall(struct timekeeper *tk)
@@ -325,6 +447,8 @@ static void timekeeping_update(struct ti
if (action & TK_MIRROR)
memcpy(&shadow_timekeeper, &tk_core.timekeeper,
sizeof(tk_core.timekeeper));
+
+ update_fast_timekeeper(tk);
}
/**
^ permalink raw reply [flat|nested] 76+ messages in thread* [patch V2 64/64] ftrace: Provide trace clocks monotonic
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (62 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 63/64] timekeeping: Provide fast and NMI safe access to CLOCK_MONOTONIC Thomas Gleixner
@ 2014-07-16 21:05 ` Thomas Gleixner
2014-07-17 12:55 ` Steven Rostedt
2014-07-17 22:32 ` [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
64 siblings, 1 reply; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-16 21:05 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra, Steven Rostedt, Mathieu Desnoyers
[-- Attachment #1: ftrace-use-clock-mono-raw.patch --]
[-- Type: text/plain, Size: 1099 bytes --]
Expose the new NMI safe accessor to clock monotonic to the tracer.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
kernel/trace/trace.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
Index: tip/kernel/trace/trace.c
===================================================================
--- tip.orig/kernel/trace/trace.c
+++ tip/kernel/trace/trace.c
@@ -806,11 +806,12 @@ static struct {
const char *name;
int in_ns; /* is this clock in nanoseconds? */
} trace_clocks[] = {
- { trace_clock_local, "local", 1 },
- { trace_clock_global, "global", 1 },
- { trace_clock_counter, "counter", 0 },
- { trace_clock_jiffies, "uptime", 1 },
- { trace_clock, "perf", 1 },
+ { trace_clock_local, "local", 1 },
+ { trace_clock_global, "global", 1 },
+ { trace_clock_counter, "counter", 0 },
+ { trace_clock_jiffies, "uptime", 1 },
+ { trace_clock, "perf", 1 },
+ { ktime_get_mono_fast_ns, "mono", 1 },
ARCH_TRACE_CLOCKS
};
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 64/64] ftrace: Provide trace clocks monotonic
2014-07-16 21:05 ` [patch V2 64/64] ftrace: Provide trace clocks monotonic Thomas Gleixner
@ 2014-07-17 12:55 ` Steven Rostedt
2014-07-17 13:12 ` Steven Rostedt
0 siblings, 1 reply; 76+ messages in thread
From: Steven Rostedt @ 2014-07-17 12:55 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: LKML, John Stultz, Peter Zijlstra, Mathieu Desnoyers
On Wed, 16 Jul 2014 21:05:25 -0000
Thomas Gleixner <tglx@linutronix.de> wrote:
> Expose the new NMI safe accessor to clock monotonic to the tracer.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> ---
> kernel/trace/trace.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> Index: tip/kernel/trace/trace.c
> ===================================================================
> --- tip.orig/kernel/trace/trace.c
> +++ tip/kernel/trace/trace.c
> @@ -806,11 +806,12 @@ static struct {
> const char *name;
> int in_ns; /* is this clock in nanoseconds? */
> } trace_clocks[] = {
> - { trace_clock_local, "local", 1 },
> - { trace_clock_global, "global", 1 },
> - { trace_clock_counter, "counter", 0 },
> - { trace_clock_jiffies, "uptime", 1 },
> - { trace_clock, "perf", 1 },
> + { trace_clock_local, "local", 1 },
> + { trace_clock_global, "global", 1 },
> + { trace_clock_counter, "counter", 0 },
> + { trace_clock_jiffies, "uptime", 1 },
> + { trace_clock, "perf", 1 },
> + { ktime_get_mono_fast_ns, "mono", 1 },
I'm not sure I like the name "mono" because all the clocks listed are
considered monolithic (with the view of tracing). What about calling it
"user" as I believe this clock is the one that will be used to
synchronize userspace times with the kernel. Right?
-- Steve
> ARCH_TRACE_CLOCKS
> };
>
>
^ permalink raw reply [flat|nested] 76+ messages in thread* Re: [patch V2 64/64] ftrace: Provide trace clocks monotonic
2014-07-17 12:55 ` Steven Rostedt
@ 2014-07-17 13:12 ` Steven Rostedt
0 siblings, 0 replies; 76+ messages in thread
From: Steven Rostedt @ 2014-07-17 13:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Thomas Gleixner, LKML, John Stultz, Peter Zijlstra, Mathieu Desnoyers
On Thu, 17 Jul 2014 08:55:35 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
> On Wed, 16 Jul 2014 21:05:25 -0000
> Thomas Gleixner <tglx@linutronix.de> wrote:
>
> I'm not sure I like the name "mono" because all the clocks listed are
> considered monolithic (with the view of tracing). What about calling it
Peter nicely informed me on IRC that I just confused monotonic with
monolithic. I think this is an improvement over my usual embarrassing
moments when I confuse the words generic with genetic.
-- Steve
> "user" as I believe this clock is the one that will be used to
> synchronize userspace times with the kernel. Right?
>
> -- Steve
>
> > ARCH_TRACE_CLOCKS
> > };
> >
> >
>
^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors
2014-07-16 21:03 [patch V2 00/64] timekeeping: 2038, optimizations, NMI safe accessors Thomas Gleixner
` (63 preceding siblings ...)
2014-07-16 21:05 ` [patch V2 64/64] ftrace: Provide trace clocks monotonic Thomas Gleixner
@ 2014-07-17 22:32 ` Thomas Gleixner
64 siblings, 0 replies; 76+ messages in thread
From: Thomas Gleixner @ 2014-07-17 22:32 UTC (permalink / raw)
To: LKML; +Cc: John Stultz, Peter Zijlstra
On Wed, 16 Jul 2014, Thomas Gleixner wrote:
> The series is against tip/timers/core. It's also available in git at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git tglx/timers/core
Anyone, who pulled from there, please do again. I just got aware that
the branch was stale.
I could swear that I updated it and according to my bash history I
really did a force push on that branch and a verify that
origin/tglx/timers/core is the same as tglx/timers/core, that's the
same procedure I use before sending out a pull request to Linus - got
burned once not doing that :)
The same thing happened about half a year ago, and looking at the mail
thread it was also a forced update, i.e. git push -f
I'm not blaming anyone except me as long as I can't be sure that my
scripting around git is not to blame.
Sorry for the inconveniance!
Thanks,
tglx
^ permalink raw reply [flat|nested] 76+ messages in thread