TSC marked unstable on suspend and resume

TSC marked unstable on suspend and resume

[David Fries]

I'm getting TSC marked as unsable on hibernate to disk with
2.6.29-rc3.  The last kernel I ran 2.6.24.4 ran 300+ days without a
problem and does not loose TSC on hibernate.

I'm hibernating with `echo disk > /sys/power/state`.
These look like the relavant messages,

CPU 0 irqstacks, hard=c0379000 soft=c0378000
PID hash table entries: 1024 (order: 10, 4096 bytes)
Fast TSC calibration using PIT
Detected 300.705 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
hibernate to disk,
Restarting tasks ... done.
Clocksource tsc unstable (delta = 499883531 ns)

another reboot,
Restarting tasks ... done.
Clocksource tsc unstable (delta = 499883531 ns)

another reboot,
Freezing user space processes ... (elapsed 0.07 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
PM: Shrinking memory...  ^H-<4>Clocksource tsc unstable (delta = 78961880 ns)
^H\^H|^H/^Hdone (34745 pages freed)



I didn't spot the problem until I raw the HP48 calculator emulator x48
and it was running like a dog, but the same binary ran fine on my
other systems.  It was looping on gettimeofday intending to delay 2us,
but was getting 1ms when TSC wasn't available.

On Thu, Feb 05, 2009 at 10:02:39PM +0100, Ingo Molnar wrote:
> What does this tool output:
> 
>  http://people.redhat.com/mingo/time-warp-test/time-warp-test.c

time-warp-test is getting the number of CPUs wrong here, trivial
patch included.

./time-warp-test  
2 CPUs, running 2 parallel test-tasks.

grep processor /proc/cpuinfo
processor       : 0
model name      : AMD-K6(tm) 3D processor


--- time-warp-test.c.orig	2009-02-19 20:52:40.000000000 -0600
+++ time-warp-test.c	2009-02-19 21:32:58.000000000 -0600
@@ -295,7 +295,7 @@ int main(int argc, char **argv)
 	unsigned long *shared;
 	unsigned long cpus, tasks;
 
-	cpus = system("exit `grep processor /proc/cpuinfo  | wc -l`");
+	cpus = system("exit `grep ^processor /proc/cpuinfo  | wc -l`");
 	cpus = WEXITSTATUS(cpus);
 
 	if (argc > 2) {




With any optimizations, -O or -O1 or -O2 I don't get any status
output, with -O0 I get,

./time-warp-test 
1 CPUs, running 1 parallel test-tasks.
checking for time-warps via:
- read time stamp counter (RDTSC) instruction (cycle resolution)

 | TSC: 0.82us, fail:0 /


-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

Re: TSC marked unstable on suspend and resume

[Rafael J. Wysocki]

On Saturday 21 February 2009, David Fries wrote:
> I'm getting TSC marked as unsable on hibernate to disk with
> 2.6.29-rc3.  The last kernel I ran 2.6.24.4 ran 300+ days without a
> problem and does not loose TSC on hibernate.

Ingo, Andrew, can you please tell us who's the right person to look into this
problem?

> I'm hibernating with `echo disk > /sys/power/state`.
> These look like the relavant messages,
> 
> CPU 0 irqstacks, hard=c0379000 soft=c0378000
> PID hash table entries: 1024 (order: 10, 4096 bytes)
> Fast TSC calibration using PIT
> Detected 300.705 MHz processor.
> Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> hibernate to disk,
> Restarting tasks ... done.
> Clocksource tsc unstable (delta = 499883531 ns)
> 
> another reboot,
> Restarting tasks ... done.
> Clocksource tsc unstable (delta = 499883531 ns)
> 
> another reboot,
> Freezing user space processes ... (elapsed 0.07 seconds) done.
> Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
> PM: Shrinking memory...  ^H-<4>Clocksource tsc unstable (delta = 78961880 ns)
> ^H\^H|^H/^Hdone (34745 pages freed)
> 
> 
> 
> I didn't spot the problem until I raw the HP48 calculator emulator x48
> and it was running like a dog, but the same binary ran fine on my
> other systems.  It was looping on gettimeofday intending to delay 2us,
> but was getting 1ms when TSC wasn't available.
> 
> On Thu, Feb 05, 2009 at 10:02:39PM +0100, Ingo Molnar wrote:
> > What does this tool output:
> > 
> >  http://people.redhat.com/mingo/time-warp-test/time-warp-test.c
> 
> time-warp-test is getting the number of CPUs wrong here, trivial
> patch included.
> 
> ./time-warp-test  
> 2 CPUs, running 2 parallel test-tasks.
> 
> grep processor /proc/cpuinfo
> processor       : 0
> model name      : AMD-K6(tm) 3D processor
> 
> 
> --- time-warp-test.c.orig	2009-02-19 20:52:40.000000000 -0600
> +++ time-warp-test.c	2009-02-19 21:32:58.000000000 -0600
> @@ -295,7 +295,7 @@ int main(int argc, char **argv)
>  	unsigned long *shared;
>  	unsigned long cpus, tasks;
>  
> -	cpus = system("exit `grep processor /proc/cpuinfo  | wc -l`");
> +	cpus = system("exit `grep ^processor /proc/cpuinfo  | wc -l`");
>  	cpus = WEXITSTATUS(cpus);
>  
>  	if (argc > 2) {
> 
> 
> 
> 
> With any optimizations, -O or -O1 or -O2 I don't get any status
> output, with -O0 I get,
> 
> ./time-warp-test 
> 1 CPUs, running 1 parallel test-tasks.
> checking for time-warps via:
> - read time stamp counter (RDTSC) instruction (cycle resolution)
> 
>  | TSC: 0.82us, fail:0 /

Re: TSC marked unstable on suspend and resume

[David Fries]

On Tue, Feb 24, 2009 at 12:23:51AM +0100, Rafael J. Wysocki wrote:
> On Saturday 21 February 2009, David Fries wrote:
> > I'm getting TSC marked as unsable on hibernate to disk with
> > 2.6.29-rc3.  The last kernel I ran 2.6.24.4 ran 300+ days without a
> > problem and does not loose TSC on hibernate.
> 
> Ingo, Andrew, can you please tell us who's the right person to look into this
> problem?

I'll be looking into it, I haven't yet started debugging it.  If
anyone has a theory of operation or what I should be lookking for, I'm
all ears.  This is an old AMD K6-2 300MHz system.  I don't know if it
matters or not, but I have NO_HZ enabled, but the timer is still going
off at 1000 Hz.

I have to decide if it is worth it to split my swap partition in two
so I can hibernate both my working kernel, and the test kernel.  It
would be nice to test a kernel and pick up where I left off.  I expect
rootfstype=ext2 on the test kernel and it really will keep my
filesystem read only.

> > I'm hibernating with `echo disk > /sys/power/state`.
> > These look like the relavant messages,
> > 
> > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > Fast TSC calibration using PIT
> > Detected 300.705 MHz processor.
> > Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> > hibernate to disk,
> > Restarting tasks ... done.
> > Clocksource tsc unstable (delta = 499883531 ns)
> > 
> > another reboot,
> > Restarting tasks ... done.
> > Clocksource tsc unstable (delta = 499883531 ns)
> > 
> > another reboot,
> > Freezing user space processes ... (elapsed 0.07 seconds) done.
> > Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
> > PM: Shrinking memory...  ^H-<4>Clocksource tsc unstable (delta = 78961880 ns)
> > ^H\^H|^H/^Hdone (34745 pages freed)
> > 
> > 
> > 
> > I didn't spot the problem until I raw the HP48 calculator emulator x48
> > and it was running like a dog, but the same binary ran fine on my
> > other systems.  It was looping on gettimeofday intending to delay 2us,
> > but was getting 1ms when TSC wasn't available.
> > 
> > On Thu, Feb 05, 2009 at 10:02:39PM +0100, Ingo Molnar wrote:
> > > What does this tool output:
> > > 
> > >  http://people.redhat.com/mingo/time-warp-test/time-warp-test.c
> > 
> > time-warp-test is getting the number of CPUs wrong here, trivial
> > patch included.
> > 
> > ./time-warp-test  
> > 2 CPUs, running 2 parallel test-tasks.
> > 
> > grep processor /proc/cpuinfo
> > processor       : 0
> > model name      : AMD-K6(tm) 3D processor
> > 
> > 
> > --- time-warp-test.c.orig	2009-02-19 20:52:40.000000000 -0600
> > +++ time-warp-test.c	2009-02-19 21:32:58.000000000 -0600
> > @@ -295,7 +295,7 @@ int main(int argc, char **argv)
> >  	unsigned long *shared;
> >  	unsigned long cpus, tasks;
> >  
> > -	cpus = system("exit `grep processor /proc/cpuinfo  | wc -l`");
> > +	cpus = system("exit `grep ^processor /proc/cpuinfo  | wc -l`");
> >  	cpus = WEXITSTATUS(cpus);
> >  
> >  	if (argc > 2) {
> > 
> > 
> > 
> > 
> > With any optimizations, -O or -O1 or -O2 I don't get any status
> > output, with -O0 I get,
> > 
> > ./time-warp-test 
> > 1 CPUs, running 1 parallel test-tasks.
> > checking for time-warps via:
> > - read time stamp counter (RDTSC) instruction (cycle resolution)
> > 
> >  | TSC: 0.82us, fail:0 /
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

Re: TSC marked unstable on suspend and resume

[Pavel Machek]

On Mon 2009-02-23 22:58:03, David Fries wrote:
> On Tue, Feb 24, 2009 at 12:23:51AM +0100, Rafael J. Wysocki wrote:
> > On Saturday 21 February 2009, David Fries wrote:
> > > I'm getting TSC marked as unsable on hibernate to disk with
> > > 2.6.29-rc3.  The last kernel I ran 2.6.24.4 ran 300+ days without a
> > > problem and does not loose TSC on hibernate.
> > 
> > Ingo, Andrew, can you please tell us who's the right person to look into this
> > problem?
> 
> I'll be looking into it, I haven't yet started debugging it.  If
> anyone has a theory of operation or what I should be lookking for, I'm
> all ears.  This is an old AMD K6-2 300MHz system.  I don't know if it
> matters or not, but I have NO_HZ enabled, but the timer is still going
> off at 1000 Hz.
> 
> I have to decide if it is worth it to split my swap partition in two
> so I can hibernate both my working kernel, and the test kernel.  It
> would be nice to test a kernel and pick up where I left off.  I expect
> rootfstype=ext2 on the test kernel and it really will keep my
> filesystem read only.

Be very careful there... if you accidentally mount rw or have shared
writable partition between production and test environment, you'll
have problems... 

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: TSC marked unstable on suspend and resume

[john stultz]

On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> I'm getting TSC marked as unsable on hibernate to disk with
> 2.6.29-rc3.  The last kernel I ran 2.6.24.4 ran 300+ days without a
> problem and does not loose TSC on hibernate.
>
> I'm hibernating with `echo disk > /sys/power/state`.
> These look like the relavant messages,
>
> CPU 0 irqstacks, hard=c0379000 soft=c0378000
> PID hash table entries: 1024 (order: 10, 4096 bytes)
> Fast TSC calibration using PIT
> Detected 300.705 MHz processor.
> Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> hibernate to disk,
> Restarting tasks ... done.
> Clocksource tsc unstable (delta = 499883531 ns)

Hmm. Seems like the clocksource watchdog is having some problem
handling state around hibernate.

Out of curiousity: Does suspend-to-ram also trigger this?

Can you send me a full dmesg log?

thanks
-john

Re: TSC marked unstable on suspend and resume

[David Fries]

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > I'm getting TSC marked as unsable on hibernate to disk with
> > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without a
> > problem and does not loose TSC on hibernate.
> >
> > I'm hibernating with `echo disk > /sys/power/state`.
> > These look like the relavant messages,
> >
> > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > Fast TSC calibration using PIT
> > Detected 300.705 MHz processor.
> > Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> > hibernate to disk,
> > Restarting tasks ... done.
> > Clocksource tsc unstable (delta = 499883531 ns)
> 
> Hmm. Seems like the clocksource watchdog is having some problem
> handling state around hibernate.
> 
> Out of curiousity: Does suspend-to-ram also trigger this?

Suspend to ram isn't supported by the system, disk is the only
available /sys/power/state.

> Can you send me a full dmesg log?

Attached, it was the only log I saw that had bootup through the tsc
marked unstable.  Looking at the log I assume I booted to a shell,
hibernated, resumed, then ran init, in that case the system is as
fresh and idle as it gets.  This system's been up for two months and I
don't reboot much so I don't remember for sure.

> thanks
> -john

-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

[-- Attachment #2: dmesg.3.gz --]
[-- Type: application/octet-stream, Size: 5955 bytes --]

Re: TSC marked unstable on suspend and resume

[john stultz]

On Wed, 2009-04-29 at 18:25 -0500, David Fries wrote:
> On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> > On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > > I'm getting TSC marked as unsable on hibernate to disk with
> > > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without a
> > > problem and does not loose TSC on hibernate.
> > >
> > > I'm hibernating with `echo disk > /sys/power/state`.
> > > These look like the relavant messages,
> > >
> > > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > > Fast TSC calibration using PIT
> > > Detected 300.705 MHz processor.
> > > Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> > > hibernate to disk,
> > > Restarting tasks ... done.
> > > Clocksource tsc unstable (delta = 499883531 ns)
> > 
> > Hmm. Seems like the clocksource watchdog is having some problem
> > handling state around hibernate.

Yea. This seems to be the case here. There's watchdog code and variables
to handle resume properly, but there isn't a corresponding suspend hook,
so the resume path doesn't actually change anything.

Here's a patch you can try, I only had time to compile test it today, so
its untested. Hopefully I didn't miss anything obvious.

Thomas, I suspect you intended to have something like the following?

Signed-off-by: John Stultz <johnstul@us.ibm.com>

diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 5a40d14..450431f 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -360,6 +360,7 @@ extern void clocksource_touch_watchdog(void);
 extern struct clocksource* clocksource_get_next(void);
 extern void clocksource_change_rating(struct clocksource *cs, int rating);
 extern void clocksource_resume(void);
+extern void clocksource_suspend(void);
 
 #ifdef CONFIG_GENERIC_TIME_VSYSCALL
 extern void update_vsyscall(struct timespec *ts, struct clocksource *c);
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index ecfd7b5..3dbe55f 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -236,6 +236,11 @@ static void clocksource_resume_watchdog(void)
 	set_bit(0, &watchdog_resumed);
 }
 
+static void clocksource_suspend_watchdog(void)
+{
+	set_bit(1, &watchdog_resumed);
+}
+
 static void clocksource_check_watchdog(struct clocksource *cs)
 {
 	struct clocksource *cse;
@@ -278,6 +283,7 @@ static void clocksource_check_watchdog(struct clocksource *cs)
 	}
 	spin_unlock_irqrestore(&watchdog_lock, flags);
 }
+
 #else
 static void clocksource_check_watchdog(struct clocksource *cs)
 {
@@ -286,6 +292,7 @@ static void clocksource_check_watchdog(struct clocksource *cs)
 }
 
 static inline void clocksource_resume_watchdog(void) { }
+static inline void clocksource_suspend_watchdog(void) { }
 #endif
 
 /**
@@ -321,6 +328,18 @@ void clocksource_touch_watchdog(void)
 }
 
 /**
+ * clocksource_suspend - suspend the clocksource(s)
+ */
+void clocksource_suspend(void)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&clocksource_lock, flags);
+	clocksource_suspend_watchdog();
+	spin_unlock_irqrestore(&clocksource_lock, flags);
+}
+
+/**
  * clocksource_get_next - Returns the selected clocksource
  *
  */
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 687dff4..e39e3d3 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -368,6 +368,7 @@ static int timekeeping_suspend(struct sys_device *dev, pm_message_t state)
 
 	clockevents_notify(CLOCK_EVT_NOTIFY_SUSPEND, NULL);
 
+	clocksource_suspend();
 	return 0;
 }
 

Re: TSC marked unstable on suspend and resume

[David Fries]

[-- Attachment #1: Type: text/plain, Size: 2440 bytes --]

On Wed, Apr 29, 2009 at 08:37:33PM -0700, john stultz wrote:
> On Wed, 2009-04-29 at 18:25 -0500, David Fries wrote:
> > On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> > > On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > > > I'm getting TSC marked as unsable on hibernate to disk with
> > > > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without a
> > > > problem and does not loose TSC on hibernate.
> > > >
> > > > I'm hibernating with `echo disk > /sys/power/state`.
> > > > These look like the relavant messages,
> > > >
> > > > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > > > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > > > Fast TSC calibration using PIT
> > > > Detected 300.705 MHz processor.
> > > > Calibrating delay loop (skipped), value calculated using timer frequency.. 601.41 BogoMIPS (lpj=300705)
> > > > hibernate to disk,
> > > > Restarting tasks ... done.
> > > > Clocksource tsc unstable (delta = 499883531 ns)
> > > 
> > > Hmm. Seems like the clocksource watchdog is having some problem
> > > handling state around hibernate.
> 
> Yea. This seems to be the case here. There's watchdog code and variables
> to handle resume properly, but there isn't a corresponding suspend hook,
> so the resume path doesn't actually change anything.
> 
> Here's a patch you can try, I only had time to compile test it today, so
> its untested. Hopefully I didn't miss anything obvious.
> 
> Thomas, I suspect you intended to have something like the following?
> 
> Signed-off-by: John Stultz <johnstul@us.ibm.com>

I added my own patch to verify your changes were being called, no
dice.  I do wonder, is it too late when the clocksource_suspend is
called after Suspending consoles?  Any theories why this is the only
system I have that is showing the problem?  It is an older system.

In the attached dmesg dump I suspended without a swap file, so it
never hibernated or rebooted.

diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index 7575f23..387fed4 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -261,6 +261,7 @@ void clocksource_suspend(void)
 	spin_lock_irqsave(&clocksource_lock, flags);
 	clocksource_suspend_watchdog();
 	spin_unlock_irqrestore(&clocksource_lock, flags);
+	printk("Calling %s\n", __func__);
 }
 
 /**

-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

[-- Attachment #2: clocksource_suspend.txt.bz2 --]
[-- Type: application/octet-stream, Size: 4502 bytes --]

Re: TSC marked unstable on suspend and resume

[Ondrej Zary]

On Thursday 30 April 2009 05:37:33 john stultz wrote:
> On Wed, 2009-04-29 at 18:25 -0500, David Fries wrote:
> > On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> > > On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > > > I'm getting TSC marked as unsable on hibernate to disk with
> > > > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without a
> > > > problem and does not loose TSC on hibernate.
> > > >
> > > > I'm hibernating with `echo disk > /sys/power/state`.
> > > > These look like the relavant messages,
> > > >
> > > > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > > > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > > > Fast TSC calibration using PIT
> > > > Detected 300.705 MHz processor.
> > > > Calibrating delay loop (skipped), value calculated using timer
> > > > frequency.. 601.41 BogoMIPS (lpj=300705) hibernate to disk,
> > > > Restarting tasks ... done.
> > > > Clocksource tsc unstable (delta = 499883531 ns)
> > >
> > > Hmm. Seems like the clocksource watchdog is having some problem
> > > handling state around hibernate.
>
> Yea. This seems to be the case here. There's watchdog code and variables
> to handle resume properly, but there isn't a corresponding suspend hook,
> so the resume path doesn't actually change anything.
>
> Here's a patch you can try, I only had time to compile test it today, so
> its untested. Hopefully I didn't miss anything obvious.

I have the same problem and the patch does not solve it for me too.

I noticed that after resume, ping is unable to measue short time correctly:
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.000 ms

"echo tsc >/sys/devices/system/clocksource/clocksource0/current_clocksource" 
fixes the problem until next hibernation:
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.223 ms


>
> Thomas, I suspect you intended to have something like the following?
>
> Signed-off-by: John Stultz <johnstul@us.ibm.com>
>
> diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
> index 5a40d14..450431f 100644
> --- a/include/linux/clocksource.h
> +++ b/include/linux/clocksource.h
> @@ -360,6 +360,7 @@ extern void clocksource_touch_watchdog(void);
>  extern struct clocksource* clocksource_get_next(void);
>  extern void clocksource_change_rating(struct clocksource *cs, int rating);
>  extern void clocksource_resume(void);
> +extern void clocksource_suspend(void);
>
>  #ifdef CONFIG_GENERIC_TIME_VSYSCALL
>  extern void update_vsyscall(struct timespec *ts, struct clocksource *c);
> diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
> index ecfd7b5..3dbe55f 100644
> --- a/kernel/time/clocksource.c
> +++ b/kernel/time/clocksource.c
> @@ -236,6 +236,11 @@ static void clocksource_resume_watchdog(void)
>  	set_bit(0, &watchdog_resumed);
>  }
>
> +static void clocksource_suspend_watchdog(void)
> +{
> +	set_bit(1, &watchdog_resumed);
> +}
> +
>  static void clocksource_check_watchdog(struct clocksource *cs)
>  {
>  	struct clocksource *cse;
> @@ -278,6 +283,7 @@ static void clocksource_check_watchdog(struct
> clocksource *cs) }
>  	spin_unlock_irqrestore(&watchdog_lock, flags);
>  }
> +
>  #else
>  static void clocksource_check_watchdog(struct clocksource *cs)
>  {
> @@ -286,6 +292,7 @@ static void clocksource_check_watchdog(struct
> clocksource *cs) }
>
>  static inline void clocksource_resume_watchdog(void) { }
> +static inline void clocksource_suspend_watchdog(void) { }
>  #endif
>
>  /**
> @@ -321,6 +328,18 @@ void clocksource_touch_watchdog(void)
>  }
>
>  /**
> + * clocksource_suspend - suspend the clocksource(s)
> + */
> +void clocksource_suspend(void)
> +{
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&clocksource_lock, flags);
> +	clocksource_suspend_watchdog();
> +	spin_unlock_irqrestore(&clocksource_lock, flags);
> +}
> +
> +/**
>   * clocksource_get_next - Returns the selected clocksource
>   *
>   */
> diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
> index 687dff4..e39e3d3 100644
> --- a/kernel/time/timekeeping.c
> +++ b/kernel/time/timekeeping.c
> @@ -368,6 +368,7 @@ static int timekeeping_suspend(struct sys_device *dev,
> pm_message_t state)
>
>  	clockevents_notify(CLOCK_EVT_NOTIFY_SUSPEND, NULL);
>
> +	clocksource_suspend();
>  	return 0;
>  }
>

-- 
Ondrej Zary

Re: TSC marked unstable on suspend and resume

[David Fries]

On Fri, May 08, 2009 at 11:53:12PM +0200, Ondrej Zary wrote:
> On Thursday 30 April 2009 05:37:33 john stultz wrote:
> > On Wed, 2009-04-29 at 18:25 -0500, David Fries wrote:
> > > On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> > > > On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > > > > I'm getting TSC marked as unsable on hibernate to disk with
> > > > > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without a
> > > > > problem and does not loose TSC on hibernate.
> > > > >
> > > > > I'm hibernating with `echo disk > /sys/power/state`.
> > > > > These look like the relavant messages,
> > > > >
> > > > > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > > > > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > > > > Fast TSC calibration using PIT
> > > > > Detected 300.705 MHz processor.
> > > > > Calibrating delay loop (skipped), value calculated using timer
> > > > > frequency.. 601.41 BogoMIPS (lpj=300705) hibernate to disk,
> > > > > Restarting tasks ... done.
> > > > > Clocksource tsc unstable (delta = 499883531 ns)
> > > > >
> > > > > model name      : AMD-K6(tm) 3D processor
> > > > > Host bridge: VIA Technologies, Inc. VT82C598 [Apollo MVP3] (rev 04)
> > > >
> > > > Hmm. Seems like the clocksource watchdog is having some problem
> > > > handling state around hibernate.
> >
> > Yea. This seems to be the case here. There's watchdog code and variables
> > to handle resume properly, but there isn't a corresponding suspend hook,
> > so the resume path doesn't actually change anything.
> >
> > Here's a patch you can try, I only had time to compile test it today, so
> > its untested. Hopefully I didn't miss anything obvious.
> 
> I have the same problem and the patch does not solve it for me too.

What CPU and motherboard chipset?

> I noticed that after resume, ping is unable to measue short time correctly:
> 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.000 ms
> 
> "echo tsc >/sys/devices/system/clocksource/clocksource0/current_clocksource" 
> fixes the problem until next hibernation:
> 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.223 ms

I didn't know about that, thanks for posting the tip.

-- 
David Fries <david@fries.net>
http://fries.net/~david/ (PGP encryption key available)

Re: TSC marked unstable on suspend and resume

[Ondrej Zary]

On Saturday 09 May 2009 02:34:15 David Fries wrote:
> On Fri, May 08, 2009 at 11:53:12PM +0200, Ondrej Zary wrote:
> > On Thursday 30 April 2009 05:37:33 john stultz wrote:
> > > On Wed, 2009-04-29 at 18:25 -0500, David Fries wrote:
> > > > On Wed, Apr 29, 2009 at 09:45:40AM -0700, john stultz wrote:
> > > > > On Fri, Feb 20, 2009 at 7:39 PM, David Fries <david@fries.net> wrote:
> > > > > > I'm getting TSC marked as unsable on hibernate to disk with
> > > > > > 2.6.29-rc3. ?The last kernel I ran 2.6.24.4 ran 300+ days without
> > > > > > a problem and does not loose TSC on hibernate.
> > > > > >
> > > > > > I'm hibernating with `echo disk > /sys/power/state`.
> > > > > > These look like the relavant messages,
> > > > > >
> > > > > > CPU 0 irqstacks, hard=c0379000 soft=c0378000
> > > > > > PID hash table entries: 1024 (order: 10, 4096 bytes)
> > > > > > Fast TSC calibration using PIT
> > > > > > Detected 300.705 MHz processor.
> > > > > > Calibrating delay loop (skipped), value calculated using timer
> > > > > > frequency.. 601.41 BogoMIPS (lpj=300705) hibernate to disk,
> > > > > > Restarting tasks ... done.
> > > > > > Clocksource tsc unstable (delta = 499883531 ns)
> > > > > >
> > > > > > model name      : AMD-K6(tm) 3D processor
> > > > > > Host bridge: VIA Technologies, Inc. VT82C598 [Apollo MVP3] (rev
> > > > > > 04)
> > > > >
> > > > > Hmm. Seems like the clocksource watchdog is having some problem
> > > > > handling state around hibernate.
> > >
> > > Yea. This seems to be the case here. There's watchdog code and
> > > variables to handle resume properly, but there isn't a corresponding
> > > suspend hook, so the resume path doesn't actually change anything.
> > >
> > > Here's a patch you can try, I only had time to compile test it today,
> > > so its untested. Hopefully I didn't miss anything obvious.
> >
> > I have the same problem and the patch does not solve it for me too.
>
> What CPU and motherboard chipset?

Cyrix MII PR300 225MHz
PCPartner TXB820DS (i430TX)

> > I noticed that after resume, ping is unable to measue short time
> > correctly: 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.000 ms
> >
> > "echo tsc
> > >/sys/devices/system/clocksource/clocksource0/current_clocksource" fixes
> > the problem until next hibernation:
> > 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.223 ms
>
> I didn't know about that, thanks for posting the tip.



-- 
Ondrej Zary