[PATCH] i386: Selectable Frequency of the Timer Interrupt.

[PATCH] i386: Selectable Frequency of the Timer Interrupt.

[christoph]

Make the timer frequency selectable. The timer interrupt may cause bus
and memory contention in large NUMA systems since the interrupt occurs
on each processor HZ times per second.

Signed-off-by: Christoph Lameter <christoph@scale86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>

Index: linux-2.6.11/arch/i386/Kconfig
===================================================================
--- linux-2.6.11.orig/arch/i386/Kconfig	2005-05-16 12:07:31.000000000 -0700
+++ linux-2.6.11/arch/i386/Kconfig	2005-05-16 12:39:48.000000000 -0700
@@ -939,6 +939,20 @@ config SECCOMP
 
 	  If unsure, say Y. Only embedded should say N here.
 
+config HZ
+	int "Frequency of the Timer Interrupt (1000 or 100)"
+	range 100 1000
+	default 1000
+	help
+	  Allows the configuration of the timer frequency. It is customary
+	  to have the timer interrupt run at 1000 HZ but 100 HZ may be more
+	  beneficial for servers and NUMA systems that do not need to have
+	  a fast response for user interaction and that may experience bus
+	  contention and cacheline bounces as a result of timer interrupts.
+	  Note that the timer interrupt occurs on each processor in an SMP
+	  environment leading to NR_CPUS * HZ number of timer interrupts
+	  per second.
+
 endmenu
 
 
Index: linux-2.6.11/include/asm-i386/param.h
===================================================================
--- linux-2.6.11.orig/include/asm-i386/param.h	2005-05-16 12:07:25.000000000 -0700
+++ linux-2.6.11/include/asm-i386/param.h	2005-05-16 12:09:04.000000000 -0700
@@ -2,7 +2,7 @@
 #define _ASMi386_PARAM_H
 
 #ifdef __KERNEL__
-# define HZ		1000		/* Internal kernel timer frequency */
+# define HZ		CONFIG_HZ	/* Internal kernel timer frequency */
 # define USER_HZ	100		/* .. some user interfaces are in "ticks" */
 # define CLOCKS_PER_SEC		(USER_HZ)	/* like times() */
 #endif

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Lee Revell]

On Mon, 2005-05-16 at 12:45 -0700, christoph wrote:
> Make the timer frequency selectable. The timer interrupt may cause bus
> and memory contention in large NUMA systems since the interrupt occurs
> on each processor HZ times per second.

Isn't there already a patch in the -ac kernel that allows HZ to be
selected at runtime?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[christoph]

On Mon, 16 May 2005, Lee Revell wrote:

> On Mon, 2005-05-16 at 12:45 -0700, christoph wrote:
> > Make the timer frequency selectable. The timer interrupt may cause bus
> > and memory contention in large NUMA systems since the interrupt occurs
> > on each processor HZ times per second.
> 
> Isn't there already a patch in the -ac kernel that allows HZ to be
> selected at runtime?

Runtime? That seems to be a bad idea. It would be better to rewrite the 
timer subsystem to be able to work tickless.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Lee Revell]

On Mon, 2005-05-16 at 17:55 -0700, christoph wrote:
> 
> Runtime? That seems to be a bad idea. It would be better to rewrite
> the timer subsystem to be able to work tickless.
> 

I agree 100%, I think it's especially crazy to allow selecting 100, 250,
500, etc, whether at runtime or compile time.  Might as well just go
tickless.

How do you expect application developers to handle not being able to
count on the resolution of nanosleep()?  Currently they can at least
assume 10ms on 2.4, 1ms on 2.6.  Seems to me that if you are no longer
guaranteed to be able to sleep 5ms on 2.6, you would just have to
busywait.  Is it me, or does that way lie madness?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Nish Aravamudan]

On 5/17/05, Lee Revell <rlrevell@joe-job.com> wrote:
> On Mon, 2005-05-16 at 17:55 -0700, christoph wrote:
> >
> > Runtime? That seems to be a bad idea. It would be better to rewrite
> > the timer subsystem to be able to work tickless.
> >
> 
> I agree 100%, I think it's especially crazy to allow selecting 100, 250,
> 500, etc, whether at runtime or compile time.  Might as well just go
> tickless.
> 
> How do you expect application developers to handle not being able to
> count on the resolution of nanosleep()?  Currently they can at least
> assume 10ms on 2.4, 1ms on 2.6.  Seems to me that if you are no longer
> guaranteed to be able to sleep 5ms on 2.6, you would just have to
> busywait.  Is it me, or does that way lie madness?

>From my meager understanding of sys_nanosleep() in 2.6 -- we'd round
up currently, If you request a microsecond of sleep, we'll sleep for a
jiffy + 1 (or 2, maybe). I am not sure we want a syscall that allows
busy-waiting, but I'm not certain. If you're interesting, my patch
(just posted again to LKML) tries to divorce HZ and soft-timers
somewhat.

-Nish

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

On Tue, 17 May 2005 19:25:41 EDT, Lee Revell said:

> How do you expect application developers to handle not being able to
> count on the resolution of nanosleep()?  Currently they can at least
> assume 10ms on 2.4, 1ms on 2.6.  Seems to me that if you are no longer
> guaranteed to be able to sleep 5ms on 2.6, you would just have to
> busywait.  Is it me, or does that way lie madness?

If you're running tickless, wouldn't a 'sleep 5ms' cause a timer event to be
queued, and we wake up (approx) 5ms later?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Lee Revell]

On Tue, 2005-05-17 at 20:03 -0400, Valdis.Kletnieks@vt.edu wrote:
> On Tue, 17 May 2005 19:25:41 EDT, Lee Revell said:
> 
> > How do you expect application developers to handle not being able to
> > count on the resolution of nanosleep()?  Currently they can at least
> > assume 10ms on 2.4, 1ms on 2.6.  Seems to me that if you are no longer
> > guaranteed to be able to sleep 5ms on 2.6, you would just have to
> > busywait.  Is it me, or does that way lie madness?
> 
> If you're running tickless, wouldn't a 'sleep 5ms' cause a timer event to be
> queued, and we wake up (approx) 5ms later?

Yes, exactly.  This is why I think going tickless is a good solution,
and CONFIG_HZ is bad, because with HZ=100 "sleep 5ms" would cause us to
sleep for 10ms.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Andrew Morton]

christoph <christoph@scalex86.org> wrote:
>
> Make the timer frequency selectable. The timer interrupt may cause bus
> and memory contention in large NUMA systems since the interrupt occurs
> on each processor HZ times per second.
> 
> Signed-off-by: Christoph Lameter <christoph@scale86.org>
> Signed-off-by: Shai Fultheim <shai@scalex86.org>
> 
> Index: linux-2.6.11/arch/i386/Kconfig
> ===================================================================
> --- linux-2.6.11.orig/arch/i386/Kconfig	2005-05-16 12:07:31.000000000 -0700
> +++ linux-2.6.11/arch/i386/Kconfig	2005-05-16 12:39:48.000000000 -0700
> @@ -939,6 +939,20 @@ config SECCOMP
>  
>  	  If unsure, say Y. Only embedded should say N here.
>  
> +config HZ
> +	int "Frequency of the Timer Interrupt (1000 or 100)"
> +	range 100 1000

Linus spat this patch back a couple of years ago.  Last time we discussed
it, a year ago, he said

  On Fri, 21 May 2004, Andrew Morton wrote:
  > 
  > Len, do you have any numbers on this?  Do you think we need to address
  > this?  If so, is there any sane alternative to CONFIG_HZ?

  100Hz is too little for a number of users, and yes, 1kHz is too high - I 
  selected it partly because it made it oh-so-much-more-obvious when 
  some pieces weren't converted. 

  1kHz is also good in that it makes it easy to convert both to USER_HZ and 
  to ms/ns. But maybe something like 250Hz would be better - still high 
  enough that things like multimedia (which really wants higher frequencies 
  in order to be able to sleep for fractional video-frames) should be happy, 
  low enough that we use less CPU.

(The issue being that the latency of entering ACPI low-power mode is of the
order of one millisecond on some machines, so HZ=1000 whacks the battery).

So yes, the time has come around again to work out what we're going to do
about this.  I'd be a bit worried about allowing users to set HZ=724,
simply because nobody tests with that, and it might expose odd timing
relationships and unfortunate arithmetic rounding artifacts.  So if we're
going to do this thing it might be better to just offer 100, 250 and 1000.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Christoph Lameter]

On Mon, 16 May 2005, Andrew Morton wrote:

> So yes, the time has come around again to work out what we're going to do
> about this.  I'd be a bit worried about allowing users to set HZ=724,
> simply because nobody tests with that, and it might expose odd timing
> relationships and unfortunate arithmetic rounding artifacts.  So if we're
> going to do this thing it might be better to just offer 100, 250 and 1000.

Ok. Here is the patch allowing 100, 250 and 1000 HZ for i386 and x86_64:

-----

Make the timer frequency selectable. The timer interrupt may cause bus
and memory contention in large NUMA systems since the interrupt occurs
on each processor HZ times per second.

Signed-off-by: Christoph Lameter <christoph@lameter.com>
Signed-off-by: Shai Fultheim <shai@scalex86.org>

Index: linux-2.6.12-rc4/arch/i386/Kconfig
===================================================================
--- linux-2.6.12-rc4.orig/arch/i386/Kconfig	2005-05-17 02:19:55.000000000 +0000
+++ linux-2.6.12-rc4/arch/i386/Kconfig	2005-05-17 02:27:12.000000000 +0000
@@ -1133,6 +1133,20 @@
 	  a work-around for a number of buggy BIOSes. Switch this option on if
 	  your computer crashes instead of powering off properly.
 
+config HZ
+	int "Frequency of the Timer Interrupt (100, 250 or 1000 per second)"
+	range 100 1000
+	default 1000
+	help
+	  Allows the configuration of the timer frequency. It is customary
+	  to have the timer interrupt run at 1000 HZ but 100 HZ may be more
+	  beneficial for servers and NUMA systems that do not need to have
+	  a fast response for user interaction and that may experience bus
+	  contention and cacheline bounces as a result of timer interrupts.
+	  Note that the timer interrupt occurs on each processor in an SMP
+	  environment leading to NR_CPUS * HZ number of timer interrupts
+	  per second.
+
 endmenu
 
 source "arch/i386/kernel/cpu/cpufreq/Kconfig"
Index: linux-2.6.12-rc4/include/asm-i386/param.h
===================================================================
--- linux-2.6.12-rc4.orig/include/asm-i386/param.h	2005-05-17 02:15:57.000000000 +0000
+++ linux-2.6.12-rc4/include/asm-i386/param.h	2005-05-17 02:30:22.000000000 +0000
@@ -1,8 +1,19 @@
+#include <linux/config.h>
+
 #ifndef _ASMi386_PARAM_H
 #define _ASMi386_PARAM_H
 
 #ifdef __KERNEL__
-# define HZ		1000		/* Internal kernel timer frequency */
+#if CONFIG_HZ == 1000
+#define HZ 1000				/* Internal kernel timer frequency */
+#elif CONFIG_HZ == 250
+#define CONFIG_HZ 250
+#elif CONFIG_HZ == 100
+#define HZ = 100
+#else
+#error "Invalid Timer Interrupt Frequency"
+#endif
+
 # define USER_HZ	100		/* .. some user interfaces are in "ticks" */
 # define CLOCKS_PER_SEC		(USER_HZ)	/* like times() */
 #endif
Index: linux-2.6.12-rc4/arch/x86_64/Kconfig
===================================================================
--- linux-2.6.12-rc4.orig/arch/x86_64/Kconfig	2005-05-17 02:19:54.000000000 +0000
+++ linux-2.6.12-rc4/arch/x86_64/Kconfig	2005-05-17 02:26:38.000000000 +0000
@@ -410,6 +410,20 @@
 
 	  If unsure, say Y. Only embedded should say N here.
 
+config HZ
+	int "Frequency of the Timer Interrupt (100, 250 or 1000 per second)"
+	range 100 1000
+	default 1000
+	help
+	  Allows the configuration of the timer frequency. It is customary
+	  to have the timer interrupt run at 1000 HZ but 100 HZ may be more
+	  beneficial for servers and NUMA systems that do not need to have
+	  a fast response for user interaction and that may experience bus
+	  contention and cacheline bounces as a result of timer interrupts.
+	  Note that the timer interrupt occurs on each processor in an SMP
+	  environment leading to NR_CPUS * HZ number of timer interrupts
+	  per second.
+
 endmenu
 
 #
Index: linux-2.6.12-rc4/include/asm-x86_64/param.h
===================================================================
--- linux-2.6.12-rc4.orig/include/asm-x86_64/param.h	2005-03-02 07:38:10.000000000 +0000
+++ linux-2.6.12-rc4/include/asm-x86_64/param.h	2005-05-17 02:28:04.000000000 +0000
@@ -1,8 +1,20 @@
+#include <linux/config.h>
+
 #ifndef _ASMx86_64_PARAM_H
 #define _ASMx86_64_PARAM_H
 
 #ifdef __KERNEL__
-# define HZ            1000            /* Internal kernel timer frequency */
+
+#if CONFIG_HZ == 1000
+# define HZ 1000            /* Internal kernel timer frequency */
+#elif CONFIG_HZ == 250
+# define HZ 250
+#elif CONFIG_HZ == 100
+# define HZ 100
+#else
+#error "Invalid Timer Interrupt Frequency"
+#endif
+
 # define USER_HZ       100          /* .. some user interfaces are in "ticks */
 #define CLOCKS_PER_SEC        (USER_HZ)       /* like times() */
 #endif

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[randy_dunlap]

On Mon, 16 May 2005 19:38:36 -0700 (PDT) Christoph Lameter wrote:

| On Mon, 16 May 2005, Andrew Morton wrote:
| 
| > So yes, the time has come around again to work out what we're going to do
| > about this.  I'd be a bit worried about allowing users to set HZ=724,
| > simply because nobody tests with that, and it might expose odd timing
| > relationships and unfortunate arithmetic rounding artifacts.  So if we're
| > going to do this thing it might be better to just offer 100, 250 and 1000.
| 
| Ok. Here is the patch allowing 100, 250 and 1000 HZ for i386 and x86_64:
| 
| -----
| 
| Make the timer frequency selectable. The timer interrupt may cause bus
| and memory contention in large NUMA systems since the interrupt occurs
| on each processor HZ times per second.
| 
| Signed-off-by: Christoph Lameter <christoph@lameter.com>
| Signed-off-by: Shai Fultheim <shai@scalex86.org>
| 
| Index: linux-2.6.12-rc4/arch/i386/Kconfig
| ===================================================================
| --- linux-2.6.12-rc4.orig/arch/i386/Kconfig	2005-05-17 02:19:55.000000000 +0000
| +++ linux-2.6.12-rc4/arch/i386/Kconfig	2005-05-17 02:27:12.000000000 +0000
| @@ -1133,6 +1133,20 @@
|  	  a work-around for a number of buggy BIOSes. Switch this option on if
|  	  your computer crashes instead of powering off properly.
|  
| +config HZ
| +	int "Frequency of the Timer Interrupt (100, 250 or 1000 per second)"
| +	range 100 1000
| +	default 1000
| +	help
| +	  Allows the configuration of the timer frequency. It is customary
| +	  to have the timer interrupt run at 1000 HZ but 100 HZ may be more
| +	  beneficial for servers and NUMA systems that do not need to have
| +	  a fast response for user interaction and that may experience bus
| +	  contention and cacheline bounces as a result of timer interrupts.
| +	  Note that the timer interrupt occurs on each processor in an SMP
| +	  environment leading to NR_CPUS * HZ number of timer interrupts
| +	  per second.
| +
|  endmenu

How about using choice / endchoice to that an improper HZ value
cannot be entered at all?  (instead of using range M N)


|  source "arch/i386/kernel/cpu/cpufreq/Kconfig"
| Index: linux-2.6.12-rc4/include/asm-i386/param.h
| ===================================================================
| --- linux-2.6.12-rc4.orig/include/asm-i386/param.h	2005-05-17 02:15:57.000000000 +0000
| +++ linux-2.6.12-rc4/include/asm-i386/param.h	2005-05-17 02:30:22.000000000 +0000
| @@ -1,8 +1,19 @@
| +#include <linux/config.h>
| +
|  #ifndef _ASMi386_PARAM_H
|  #define _ASMi386_PARAM_H
|  
|  #ifdef __KERNEL__
| -# define HZ		1000		/* Internal kernel timer frequency */
| +#if CONFIG_HZ == 1000
| +#define HZ 1000				/* Internal kernel timer frequency */
| +#elif CONFIG_HZ == 250
| +#define CONFIG_HZ 250
| +#elif CONFIG_HZ == 100
| +#define HZ = 100
| +#else
| +#error "Invalid Timer Interrupt Frequency"
| +#endif
| +
|  # define USER_HZ	100		/* .. some user interfaces are in "ticks" */
|  # define CLOCKS_PER_SEC		(USER_HZ)	/* like times() */
|  #endif
| Index: linux-2.6.12-rc4/arch/x86_64/Kconfig
| ===================================================================
| --- linux-2.6.12-rc4.orig/arch/x86_64/Kconfig	2005-05-17 02:19:54.000000000 +0000
| +++ linux-2.6.12-rc4/arch/x86_64/Kconfig	2005-05-17 02:26:38.000000000 +0000
| @@ -410,6 +410,20 @@
|  
|  	  If unsure, say Y. Only embedded should say N here.
|  
| +config HZ
| +	int "Frequency of the Timer Interrupt (100, 250 or 1000 per second)"
| +	range 100 1000
| +	default 1000
| +	help
| +	  Allows the configuration of the timer frequency. It is customary
| +	  to have the timer interrupt run at 1000 HZ but 100 HZ may be more
| +	  beneficial for servers and NUMA systems that do not need to have
| +	  a fast response for user interaction and that may experience bus
| +	  contention and cacheline bounces as a result of timer interrupts.
| +	  Note that the timer interrupt occurs on each processor in an SMP
| +	  environment leading to NR_CPUS * HZ number of timer interrupts
| +	  per second.
| +
|  endmenu
|  
|  #
| Index: linux-2.6.12-rc4/include/asm-x86_64/param.h
| ===================================================================
| --- linux-2.6.12-rc4.orig/include/asm-x86_64/param.h	2005-03-02 07:38:10.000000000 +0000
| +++ linux-2.6.12-rc4/include/asm-x86_64/param.h	2005-05-17 02:28:04.000000000 +0000
| @@ -1,8 +1,20 @@
| +#include <linux/config.h>
| +
|  #ifndef _ASMx86_64_PARAM_H
|  #define _ASMx86_64_PARAM_H
|  
|  #ifdef __KERNEL__
| -# define HZ            1000            /* Internal kernel timer frequency */
| +
| +#if CONFIG_HZ == 1000
| +# define HZ 1000            /* Internal kernel timer frequency */
| +#elif CONFIG_HZ == 250
| +# define HZ 250
| +#elif CONFIG_HZ == 100
| +# define HZ 100
| +#else
| +#error "Invalid Timer Interrupt Frequency"
| +#endif
| +|  # define USER_HZ       100          /* .. some user interfaces are in "ticks */
|  #define CLOCKS_PER_SEC        (USER_HZ)       /* like times() */
|  #endif


---
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Christoph Lameter]

On Mon, 16 May 2005, randy_dunlap wrote:

> |  endmenu
> 
> How about using choice / endchoice to that an improper HZ value
> cannot be entered at all?  (instead of using range M N)

That would not allow to set the value of CONFIG_HZ to a numeric value.
I would have CONFIG_HZ_100 CONFIG_HZ_250 etc. Gets a bit complicated
to handle.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[randy_dunlap]

On Mon, 16 May 2005 19:55:42 -0700 (PDT) Christoph Lameter wrote:

| On Mon, 16 May 2005, randy_dunlap wrote:
| 
| > |  endmenu
| > 
| > How about using choice / endchoice to that an improper HZ value
| > cannot be entered at all?  (instead of using range M N)
| 
| That would not allow to set the value of CONFIG_HZ to a numeric value.
| I would have CONFIG_HZ_100 CONFIG_HZ_250 etc. Gets a bit complicated
| to handle.

Ack, I see.

Thanks for explaining.

---
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Linus Torvalds]

On Mon, 16 May 2005, Christoph Lameter wrote:
> 
> That would not allow to set the value of CONFIG_HZ to a numeric value.
> I would have CONFIG_HZ_100 CONFIG_HZ_250 etc. Gets a bit complicated
> to handle.

I don't think it gets any more complex to handle than the stuff you need 
to do now (#ifdef's, and the #define HZ CONFIG_HZ games).

Also, I think you can do it in the Kconfig file, which at least makes it a 
fairly localized thing:

	choice
		prompt "Timer frequency"
		default HZ_250

	config HZ_100
		bool "100 Hz"

	confic HZ_250
		bool "250 Hz"

	config HZ_1000
		bool "1000 Hz"

	endchoice

	config HZ
		int
		default 100 if HZ_100
		default 250 if HZ_250
		default 1000 if HZ_1000

and now you can just do

	#define HZ CONFIG_HZ

or something. You can even maje the Kconfig parts be a separate Kconfig.HZ
file, and have both the x86 and x86-64 Kconfig files just include the
common part (since it's a generic issue, not even PC-related: we might
want to allow things like 60Hz frequencies for CONFIG_EMBEDDED etc, and
these choices are really valid on any system that allows for the timer to
be reprogrammed)

The above is obviously totally untested, but it doesn't look any more
complex than having a fairly ugly (and much less user-friendly) check at
compile-time.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Andi Kleen]

> 	choice
> 		prompt "Timer frequency"
> 		default HZ_250
> 
> 	config HZ_100
> 		bool "100 Hz"
> 
> 	confic HZ_250
> 		bool "250 Hz"
> 
> 	config HZ_1000
> 		bool "1000 Hz"

I would add a 

	config HZ_10 if EMBEDDED 
		bool "10 Hz" 

that is useful for compute servers (although it will violate the TCP
specification). EMBEDDED would ensure only people who know what they're
doing set it.

-Andi

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Paulo Marques]

Andi Kleen wrote:
>>[...]
> I would add a 
> 
> 	config HZ_10 if EMBEDDED 
> 		bool "10 Hz" 
> 
> that is useful for compute servers (although it will violate the TCP
> specification). EMBEDDED would ensure only people who know what they're
> doing set it.

I thought the lowest frequency the PIT timer would give was around 18 Hz.

Am I wrong, or are you thinking of other timing devices / different 
platforms?

-- 
Paulo Marques - www.grupopie.com

An expert is a person who has made all the mistakes that can be
made in a very narrow field.
Niels Bohr (1885 - 1962)

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Andi Kleen]

On Tue, May 17, 2005 at 11:51:03AM +0100, Paulo Marques wrote:
> Andi Kleen wrote:
> >>[...]
> >I would add a 
> >
> >	config HZ_10 if EMBEDDED 
> >		bool "10 Hz" 
> >
> >that is useful for compute servers (although it will violate the TCP
> >specification). EMBEDDED would ensure only people who know what they're
> >doing set it.
> 
> I thought the lowest frequency the PIT timer would give was around 18 Hz.
> 
> Am I wrong, or are you thinking of other timing devices / different 
> platforms?

I was thinking of HPET. You're right it would probably not work with 
PIT. 

Oh well, I guess it wasn't that great an idea anyways. I merely
suggested it because I know some people do it already.

-Andi

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Christoph Lameter]

On Mon, 16 May 2005, Linus Torvalds wrote:

> or something. You can even maje the Kconfig parts be a separate Kconfig.HZ
> file, and have both the x86 and x86-64 Kconfig files just include the
> common part (since it's a generic issue, not even PC-related: we might
> want to allow things like 60Hz frequencies for CONFIG_EMBEDDED etc, and
> these choices are really valid on any system that allows for the timer to
> be reprogrammed)

Ok. Here is the patch redone. The location for Kconfig.hz is in the
kernel directory since the other timer related stuff is there too:

---

Make the timer frequency selectable. The timer interrupt may cause bus
and memory contention in large NUMA systems since the interrupt occurs
on each processor HZ times per second.

Signed-off-by: Christoph Lameter <christoph@lameter.com>
Signed-off-by: Shai Fultheim <shai@scalex86.org>

Index: linux-2.6.12-rc4/arch/i386/Kconfig
===================================================================
--- linux-2.6.12-rc4.orig/arch/i386/Kconfig	2005-05-17 02:19:55.000000000 +0000
+++ linux-2.6.12-rc4/arch/i386/Kconfig	2005-05-17 05:27:31.000000000 +0000
@@ -1133,6 +1133,8 @@
 	  a work-around for a number of buggy BIOSes. Switch this option on if
 	  your computer crashes instead of powering off properly.
 
+source kernel/Kconfig.hz
+
 endmenu
 
 source "arch/i386/kernel/cpu/cpufreq/Kconfig"
Index: linux-2.6.12-rc4/include/asm-i386/param.h
===================================================================
--- linux-2.6.12-rc4.orig/include/asm-i386/param.h	2005-05-17 05:08:56.000000000 +0000
+++ linux-2.6.12-rc4/include/asm-i386/param.h	2005-05-17 05:10:08.000000000 +0000
@@ -1,8 +1,10 @@
+#include <linux/config.h>
+
 #ifndef _ASMi386_PARAM_H
 #define _ASMi386_PARAM_H
 
 #ifdef __KERNEL__
-# define HZ		1000		/* Internal kernel timer frequency */
+# define HZ		CONFIG_HZ	/* Internal kernel timer frequency */
 # define USER_HZ	100		/* .. some user interfaces are in "ticks" */
 # define CLOCKS_PER_SEC		(USER_HZ)	/* like times() */
 #endif
Index: linux-2.6.12-rc4/arch/x86_64/Kconfig
===================================================================
--- linux-2.6.12-rc4.orig/arch/x86_64/Kconfig	2005-05-17 02:19:54.000000000 +0000
+++ linux-2.6.12-rc4/arch/x86_64/Kconfig	2005-05-17 05:20:49.000000000 +0000
@@ -410,6 +410,8 @@
 
 	  If unsure, say Y. Only embedded should say N here.
 
+source kernel/Kconfig.hz
+
 endmenu
 
 #
Index: linux-2.6.12-rc4/include/asm-x86_64/param.h
===================================================================
--- linux-2.6.12-rc4.orig/include/asm-x86_64/param.h	2005-05-17 05:08:52.000000000 +0000
+++ linux-2.6.12-rc4/include/asm-x86_64/param.h	2005-05-17 05:09:42.000000000 +0000
@@ -1,9 +1,11 @@
+#include <linux/config.h>
+
 #ifndef _ASMx86_64_PARAM_H
 #define _ASMx86_64_PARAM_H
 
 #ifdef __KERNEL__
-# define HZ            1000            /* Internal kernel timer frequency */
-# define USER_HZ       100          /* .. some user interfaces are in "ticks */
+# define HZ            CONFIG_HZ	/* Internal kernel timer frequency */
+# define USER_HZ       100		/* .. some user interfaces are in "ticks */
 #define CLOCKS_PER_SEC        (USER_HZ)       /* like times() */
 #endif
 
Index: linux-2.6.12-rc4/kernel/Kconfig.hz
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6.12-rc4/kernel/Kconfig.hz	2005-05-17 05:24:01.000000000 +0000
@@ -0,0 +1,46 @@
+#
+# Timer Interrupt Frequency Configuration
+#
+
+choice
+	prompt "Timer frequency"
+	default HZ_250
+	help
+	 Allows the configuration of the timer frequency. It is customary
+	 to have the timer interrupt run at 1000 HZ but 100 HZ may be more
+	 beneficial for servers and NUMA systems that do not need to have
+	 a fast response for user interaction and that may experience bus
+	 contention and cacheline bounces as a result of timer interrupts.
+	 Note that the timer interrupt occurs on each processor in an SMP
+	 environment leading to NR_CPUS * HZ number of timer interrupts
+	 per second.
+
+
+	config HZ_100
+		bool "100 HZ"
+	help
+	  100 HZ is a typical choice for servers, SMP and NUMA systems
+	  with lots of processors that may show reduced performance if
+	  too many timer interrupts are occurring.
+
+	config HZ_250
+		bool "250 HZ"
+	help
+	 250 HZ is a good compromise choice allowing server performance
+	 while also showing good interactive responsiveness even
+	 on SMP and NUMA systems.
+
+	config HZ_1000
+		bool "1000 HZ"
+	help
+	 1000 HZ is the preferred choice for desktop systems and other
+	 systems requiring fast interactive responses to events.
+
+endchoice
+
+config HZ
+	int
+	default 100 if HZ_100
+	default 250 if HZ_250
+	default 1000 if HZ_1000
+

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Joe Korty]

On Mon, May 16, 2005 at 10:31:25PM -0700, Christoph Lameter wrote:
> +	help
> +	  100 HZ is a typical choice for servers, SMP and NUMA systems
> +	  with lots of processors that may show reduced performance if
> +	  too many timer interrupts are occurring.

One of the options should mention the power savings benefit on laptops.
How about:

	help

	  100 HZ, the lowest setting, is the best choice for any system
	  where the servicing of interrupts is expensive.  This includes:
	  systems with so many processors that the mere execution of timer
	  interrupts on each and every processor degrades performance,
	  virtual systems, where Linux is not running on bare hardware
	  but is instead a guest operating system running on top of a
	  virtualization layer, and laptops, where each interrupt causes
	  a processor that is in low power mode to power up in order
	  to service the interrupt, and after the interrupt is complete,
	  might take up to one millisecond to power back down again.

Regards,
Joe
--
"Money can buy bandwidth, but latency is forever" -- John Mashey

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Andi Kleen]

> > +	help
> > +	  100 HZ is a typical choice for servers, SMP and NUMA systems
> > +	  with lots of processors that may show reduced performance if
> > +	  too many timer interrupts are occurring.
> 
> One of the options should mention the power savings benefit on laptops.
> How about:

Actually it is not 100% clear. The ACPI idle code relies on
the timer right now to go from C1 to C2/C3.  It basically
goes down in a staircase, first staying in C1, then when woken
up and still idle go down lower etc.

With HZ=100 the minimal latency (assuming no other interrupts) to go from C1 
to C2 is 10ms, not 1ms, which might be even a power loss in some workloads.

-Andi

P.S.: The SUSE 2.4 kernels had for some time variable HZ, settable at boot. 
It surprisingly didn't cause too much slowdown or code bloat and only
needed minor fixes over the tree. Might be worth considering at least
as a CONFIG

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Christoph Lameter]

On Tue, 17 May 2005, Joe Korty wrote:

> One of the options should mention the power savings benefit on laptops.
> How about:

I am not an expert on that. Submit a patch.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Pavel Machek]

Hi!

> 	config HZ
> 		int
> 		default 100 if HZ_100
> 		default 250 if HZ_250
> 		default 1000 if HZ_1000
> 
> and now you can just do
> 
> 	#define HZ CONFIG_HZ
> 
> or something. You can even maje the Kconfig parts be a separate Kconfig.HZ
> file, and have both the x86 and x86-64 Kconfig files just include the
> common part (since it's a generic issue, not even PC-related: we might
> want to allow things like 60Hz frequencies for CONFIG_EMBEDDED etc, and
> these choices are really valid on any system that allows for the timer to
> be reprogrammed)
> 
> The above is obviously totally untested, but it doesn't look any more
> complex than having a fairly ugly (and much less user-friendly) check at
> compile-time.

Please don't do this, CONFIG_NO_IDLE_HZ patches are better solution,
and they worked okay last time I tried them.
								Pavel

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Lee Revell]

On Wed, 2005-05-18 at 20:50 +0200, Pavel Machek wrote:
> Please don't do this, CONFIG_NO_IDLE_HZ patches are better solution,
> and they worked okay last time I tried them.

Last time the dynamic tick patches were posted, you reported they worked
fine.  The next question is, when do they get merged?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Tony Lindgren]

* Lee Revell <rlrevell@joe-job.com> [050518 12:06]:
> On Wed, 2005-05-18 at 20:50 +0200, Pavel Machek wrote:
> > Please don't do this, CONFIG_NO_IDLE_HZ patches are better solution,
> > and they worked okay last time I tried them.
> 
> Last time the dynamic tick patches were posted, you reported they worked
> fine.  The next question is, when do they get merged?

Uh, I've been meaning to do some clean-up on the x86 patch, but been
distracted every time I've tried... I'll try to do an updated patch
soon... But meanwhile, I believe the dyn-tick patch works reliably
on all machines if DYN_TICK_USE_APIC is not set in Kconfig.

Tony

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Linus Torvalds]

On Wed, 18 May 2005, Pavel Machek wrote:
> 
> Please don't do this, CONFIG_NO_IDLE_HZ patches are better solution,
> and they worked okay last time I tried them.

.. and they have nothing to do with this.

A number of people who want lower tick frequency are apparently _server_
people. Not because it makes any difference to idle time, but because it
can lessen the impact of the timer interrupt under load.

I don't know why, but I've actually gotten most of the complaints about
the 1kHz timer from ia64 people, who use a 1024Hz timer. Somebody from
Intel claimed a several percent reduction in performance between 1kHz and
100Hz under some load, apparently because of bad cache interaction.

At the same time, 100Hz really is too low for some desktop-like soft-RT
stuff, where you want to delay until the next frame (and humans notice
jitter at some fraction of a tenth of a second). With the 100Hz
granularity, and the uncertainty on where the jiffy tick ends up being,
you effectively have a ~50Hz clock you can depend on, which together with
worries about synchronizing with the video refresh rate etc seems to make
people unhappy.

So this thing has nothing to do with "idle". 

And the truly-variable-HZ stuff just makes me nervous, but regardless of 
that, you actually do want a "limit HZ to some value" configuration option 
anyway.

Even with fully variable HZ, you need a limit just to say "this is the
highest precision we'll ever use", because otherwise you'll just be
wasting a lot of time on timers.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Tony Lindgren]

* Linus Torvalds <torvalds@osdl.org> [050518 12:28]:
> 
> 
> On Wed, 18 May 2005, Pavel Machek wrote:
> > 
> > Please don't do this, CONFIG_NO_IDLE_HZ patches are better solution,
> > and they worked okay last time I tried them.
> 
> .. and they have nothing to do with this.
> 
> A number of people who want lower tick frequency are apparently _server_
> people. Not because it makes any difference to idle time, but because it
> can lessen the impact of the timer interrupt under load.
> 
> I don't know why, but I've actually gotten most of the complaints about
> the 1kHz timer from ia64 people, who use a 1024Hz timer. Somebody from
> Intel claimed a several percent reduction in performance between 1kHz and
> 100Hz under some load, apparently because of bad cache interaction.
> 
> At the same time, 100Hz really is too low for some desktop-like soft-RT
> stuff, where you want to delay until the next frame (and humans notice
> jitter at some fraction of a tenth of a second). With the 100Hz
> granularity, and the uncertainty on where the jiffy tick ends up being,
> you effectively have a ~50Hz clock you can depend on, which together with
> worries about synchronizing with the video refresh rate etc seems to make
> people unhappy.
> 
> So this thing has nothing to do with "idle". 

Yes, that's right. Setting HZ would just limit the max frequency
with dyn-tick patch when system is busy. On OMAP, we're using HZ=128
with dyn-tick.

> And the truly-variable-HZ stuff just makes me nervous, but regardless of 
> that, you actually do want a "limit HZ to some value" configuration option 
> anyway.

The dyn-tick patch skips ticks only during idle, and the system is
not doing anything at that point, so it should be safe. When the
system is under load, there is normal HZ tick and timer is not being
reprogrammed.

> Even with fully variable HZ, you need a limit just to say "this is the
> highest precision we'll ever use", because otherwise you'll just be
> wasting a lot of time on timers.

Yeah.

Tony

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt.

[Coywolf Qi Hunt]

On 5/17/05, Christoph Lameter <christoph@lameter.com> wrote:
> On Mon, 16 May 2005, Andrew Morton wrote:
> 
> > So yes, the time has come around again to work out what we're going to do
> > about this.  I'd be a bit worried about allowing users to set HZ=724,
> > simply because nobody tests with that, and it might expose odd timing
> > relationships and unfortunate arithmetic rounding artifacts.  So if we're
> > going to do this thing it might be better to just offer 100, 250 and 1000.
> 
> Ok. Here is the patch allowing 100, 250 and 1000 HZ for i386 and x86_64:
> 
> -----
> 
> Make the timer frequency selectable. The timer interrupt may cause bus
> and memory contention in large NUMA systems since the interrupt occurs
> on each processor HZ times per second.
> 
> Signed-off-by: Christoph Lameter <christoph@lameter.com>
> Signed-off-by: Shai Fultheim <shai@scalex86.org>
> 
> Index: linux-2.6.12-rc4/arch/i386/Kconfig
> ===================================================================
> --- linux-2.6.12-rc4.orig/arch/i386/Kconfig     2005-05-17 02:19:55.000000000 +0000
> +++ linux-2.6.12-rc4/arch/i386/Kconfig  2005-05-17 02:27:12.000000000 +0000
> @@ -1133,6 +1133,20 @@
>           a work-around for a number of buggy BIOSes. Switch this option on if
>           your computer crashes instead of powering off properly.
> 
> +config HZ
> +       int "Frequency of the Timer Interrupt (100, 250 or 1000 per second)"
> +       range 100 1000
> +       default 1000
> +       help
> +         Allows the configuration of the timer frequency. It is customary
> +         to have the timer interrupt run at 1000 HZ but 100 HZ may be more
> +         beneficial for servers and NUMA systems that do not need to have
> +         a fast response for user interaction and that may experience bus
> +         contention and cacheline bounces as a result of timer interrupts.
> +         Note that the timer interrupt occurs on each processor in an SMP
> +         environment leading to NR_CPUS * HZ number of timer interrupts
> +         per second.
> +
>  endmenu
> 
>  source "arch/i386/kernel/cpu/cpufreq/Kconfig"
> Index: linux-2.6.12-rc4/include/asm-i386/param.h
> ===================================================================
> --- linux-2.6.12-rc4.orig/include/asm-i386/param.h      2005-05-17 02:15:57.000000000 +0000
> +++ linux-2.6.12-rc4/include/asm-i386/param.h   2005-05-17 02:30:22.000000000 +0000
> @@ -1,8 +1,19 @@
> +#include <linux/config.h>
> +
>  #ifndef _ASMi386_PARAM_H
>  #define _ASMi386_PARAM_H
> 
>  #ifdef __KERNEL__
> -# define HZ            1000            /* Internal kernel timer frequency */
> +#if CONFIG_HZ == 1000
> +#define HZ 1000                                /* Internal kernel timer frequency */
> +#elif CONFIG_HZ == 250
> +#define CONFIG_HZ 250

You mean #define HZ 250 here.
-- 
Coywolf Qi Hunt
http://sosdg.org/~coywolf/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:

> [PATCH] i386: Selectable Frequency of the Timer Interrupt

[...]

> +choice
> +	prompt "Timer frequency"
> +	default HZ_250

WHAT?

The previous value here i386 is 1000 --- so why is the default 250.

Changing the *default* time interrupt frequency in a stable series is
*really* bad idea if you ask me.


Now that this has hit mainline please consider applying:


--- kernel/Kconfig.hz~	2005-06-24 22:16:35.023986593 -0700
+++ kernel/Kconfig.hz	2005-07-08 14:46:35.428917597 -0700
@@ -4,7 +4,7 @@
 
 choice
 	prompt "Timer frequency"
-	default HZ_250
+	default HZ_1000
 	help
 	 Allows the configuration of the timer frequency. It is customary
 	 to have the timer interrupt run at 1000 HZ but 100 HZ may be more

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andrew Morton]

Chris Wedgwood <cw@f00f.org> wrote:
>
> On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:
          ^^^^^^

It's been over two weeks and nobody has complained about anything.

> 
> > [PATCH] i386: Selectable Frequency of the Timer Interrupt
> 
> [...]
> 
> > +choice
> > +	prompt "Timer frequency"
> > +	default HZ_250
> 
> WHAT?
> 
> The previous value here i386 is 1000 --- so why is the default 250.

Because 1000 is too high.

> Now that this has hit mainline please consider applying:

We might indeed do that.  But if nobody continues to notice anything, we
might not.  We'll see.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Fri, Jul 08, 2005 at 02:59:53PM -0700, Andrew Morton wrote:

> > On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:
>           ^^^^^^

> It's been over two weeks and nobody has complained about anything.

Two weeks isn't that long IMO (I only just noticed myself).

> Because 1000 is too high.

How so?  There have been comparatively few complaints about this since
we switched quite some time ago.

Strictly speaking I agree 1000 might be too high --- but we've had it
for so long now, almost all over 2.5.x (I think?) and all of 2.6.x.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Fri, 8 Jul 2005, Andrew Morton wrote:
> > 
> > The previous value here i386 is 1000 --- so why is the default 250.
> 
> Because 1000 is too high.

Yes. I chose 1000 originally partly as a way to make sure that people that
assumed HZ was 100 would get a swift kick in the pants. That meant making
a _big_ change, not a small subtle one. For example, people tend to react
if "uptime" suddenly says the machine has been up for a hundred days (even
if it's really only been up for ten), but if it is off by just a factor of
two, it might be overlooked.

So 1kHz was a bit of an overkill, but it worked well enough that we never 
really got around to changing it. 

			Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Denis Vlasenko]

On Saturday 09 July 2005 01:08, Linus Torvalds wrote:
> 
> On Fri, 8 Jul 2005, Andrew Morton wrote:
> > > 
> > > The previous value here i386 is 1000 --- so why is the default 250.
> > 
> > Because 1000 is too high.
> 
> Yes. I chose 1000 originally partly as a way to make sure that people that
> assumed HZ was 100 would get a swift kick in the pants. That meant making
> a _big_ change, not a small subtle one. For example, people tend to react
> if "uptime" suddenly says the machine has been up for a hundred days (even
> if it's really only been up for ten), but if it is off by just a factor of
> two, it might be overlooked.
> 
> So 1kHz was a bit of an overkill, but it worked well enough that we never 
> really got around to changing it. 

There are lots of HZ/100 in the kernel. With either 100 or 1000 it divides exactly,
but with 250 it's wrong by 20%.
--
vda

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Fri, 2005-07-08 at 14:59 -0700, Andrew Morton wrote:
> Chris Wedgwood <cw@f00f.org> wrote:
> >
> > On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:
>           ^^^^^^
> 
> It's been over two weeks and nobody has complained about anything.

Wrong, I complained loudly about this as soon as it was posted, about
two weeks ago.  There was an LKML thread about it and lwn.net linked to
it.

In case you missed it my reaction was almost word for word the same as
Chris's, along the lines of "Are you insane?"

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andrew Morton]

Lee Revell <rlrevell@joe-job.com> wrote:
>
> On Fri, 2005-07-08 at 14:59 -0700, Andrew Morton wrote:
> > Chris Wedgwood <cw@f00f.org> wrote:
> > >
> > > On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:
> >           ^^^^^^
> > 
> > It's been over two weeks and nobody has complained about anything.
> 
> Wrong, I complained loudly about this as soon as it was posted, about
> two weeks ago.

You know what I mean.  Nobody has yet reported adverse effects upon real
applications.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

> It's been over two weeks and nobody has complained about anything.
> 
>> 
>> > [PATCH] i386: Selectable Frequency of the Timer Interrupt
>> 
>> [...]
>> 
>> > +choice
>> > +	prompt "Timer frequency"
>> > +	default HZ_250
>> 
>> WHAT?
>> 
>> The previous value here i386 is 1000 --- so why is the default 250.
> 
> Because 1000 is too high.
> 
>> Now that this has hit mainline please consider applying:
> 
> We might indeed do that.  But if nobody continues to notice anything, we
> might not.  We'll see.

I noticed something ... the fact that elapsed times on kernel compiles
etc got significantly faster with lower HZ values ... see for example

http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/kernbench.elm3b67.png

Sorry, it's getting a bit scrunched in there ... I need to make it bigger
and filter old details stuff out better.

I think we're talking between 2.6.12-git5 and 2.6.12-git6 right? I can
confirm more explicitly if really need be. 48s -> 45.5s elapsed.

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Fri, Jul 08, 2005 at 03:59:35PM -0700, Martin J. Bligh wrote:

> I think we're talking between 2.6.12-git5 and 2.6.12-git6 right? I
> can confirm more explicitly if really need be. 48s -> 45.5s elapsed.

That's a huge difference (5%) --- what hardware is that on?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--On Friday, July 08, 2005 16:03:03 -0700 Chris Wedgwood <cw@f00f.org> wrote:

> On Fri, Jul 08, 2005 at 03:59:35PM -0700, Martin J. Bligh wrote:
> 
>> I think we're talking between 2.6.12-git5 and 2.6.12-git6 right? I
>> can confirm more explicitly if really need be. 48s -> 45.5s elapsed.
> 
> That's a huge difference (5%) --- what hardware is that on?

16 CPU x440 (about a 2-3 year old NUMA box). flat 4x seems about a 1% 
gain, still statistically significant, but definitely smaller.

I'm not saying there isn't data supporting higher HZ ... I just haven't
seen it published. I get the feeling what people really want is high-res
timers anyway ... high HZ is just concealing the issue and making it
slightly less crap, not actually fixing it.

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[David S. Miller]

From: "Martin J. Bligh" <mbligh@mbligh.org>
Date: Fri, 08 Jul 2005 16:14:59 -0700

> I'm not saying there isn't data supporting higher HZ ... I just haven't
> seen it published. I get the feeling what people really want is high-res
> timers anyway ... high HZ is just concealing the issue and making it
> slightly less crap, not actually fixing it.

We very much want sub-HZ timers, especially in the networking.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Fri, 2005-07-08 at 16:29 -0700, David S. Miller wrote:
> From: "Martin J. Bligh" <mbligh@mbligh.org>
> Date: Fri, 08 Jul 2005 16:14:59 -0700
> 
> > I'm not saying there isn't data supporting higher HZ ... I just haven't
> > seen it published. I get the feeling what people really want is high-res
> > timers anyway ... high HZ is just concealing the issue and making it
> > slightly less crap, not actually fixing it.
> 
> We very much want sub-HZ timers, especially in the networking.

ALSA would also benefit greatly from this.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

On Fri, 2005-07-08 at 14:59 -0700, Andrew Morton wrote:
> Chris Wedgwood <cw@f00f.org> wrote:

> > WHAT?
> > 
> > The previous value here i386 is 1000 --- so why is the default 250.
> 
> Because 1000 is too high.
> 

What happened to 300 as default, as that is divisible by both 50 and 60
(or something like that) ?  Wanted to remember somebody suggested rather
using that ...  Curiosity sake.


Thanks,

-- 
Martin Schlemmer


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 19:08 +0200, Martin Schlemmer wrote:
> On Fri, 2005-07-08 at 14:59 -0700, Andrew Morton wrote:
> > Chris Wedgwood <cw@f00f.org> wrote:
> 
> > > WHAT?
> > > 
> > > The previous value here i386 is 1000 --- so why is the default 250.
> > 
> > Because 1000 is too high.
> > 
> 
> What happened to 300 as default, as that is divisible by both 50 and 60
> (or something like that) ?

I still think you're absolutely insane to change the default in the
middle of a stable kernel series.  People WILL complain about it.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Arjan van de Ven]

On Sat, 2005-07-09 at 14:16 -0400, Lee Revell wrote:
> On Sat, 2005-07-09 at 19:08 +0200, Martin Schlemmer wrote:
> > On Fri, 2005-07-08 at 14:59 -0700, Andrew Morton wrote:
> > > Chris Wedgwood <cw@f00f.org> wrote:
> > 
> > > > WHAT?
> > > > 
> > > > The previous value here i386 is 1000 --- so why is the default 250.
> > > 
> > > Because 1000 is too high.
> > > 
> > 
> > What happened to 300 as default, as that is divisible by both 50 and 60
> > (or something like that) ?
> 
> I still think you're absolutely insane to change the default in the
> middle of a stable kernel series.  People WILL complain about it.

why?

it's a config option. Some distros ship 100 already, others 1000, again
others will do 250. What does it matter?
(Although I still prefer 300 over 250 due to the 50/60 thing)

This is not a userspace visible thing really with few exceptions, and
well people can select the one they want, right?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Sat, Jul 09, 2005 at 08:31:55PM +0200, Arjan van de Ven wrote:

> it's a config option. Some distros ship 100 already, others 1000,
> again others will do 250.

Who does anything other than 1000 for a 2.6.x kernel?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 20:31 +0200, Arjan van de Ven wrote:
> why?

Because the minimum poll/select timeout is now 4ms rather than 1ms.  An
app that has a soft RT constraint somewhere in the middle that worked on
2.6.12 will break on 2.6.13.

> it's a config option. Some distros ship 100 already, others 1000, again
> others will do 250. What does it matter?
> (Although I still prefer 300 over 250 due to the 50/60 thing)
> 
> This is not a userspace visible thing really with few exceptions, and
> well people can select the one they want, right?

Then why not leave the default at 1000?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andrew Morton]

Lee Revell <rlrevell@joe-job.com> wrote:
>
>  > This is not a userspace visible thing really with few exceptions, and
>  > well people can select the one they want, right?
> 
>  Then why not leave the default at 1000?

Because some machines exhibit appreciable latency in entering low power
state via ACPI, and 1000Hz reduces their battery life.  By about half,
iirc.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 12:12 -0700, Andrew Morton wrote:
> Lee Revell <rlrevell@joe-job.com> wrote:
> >
> >  > This is not a userspace visible thing really with few exceptions, and
> >  > well people can select the one they want, right?
> > 
> >  Then why not leave the default at 1000?
> 
> Because some machines exhibit appreciable latency in entering low power
> state via ACPI, and 1000Hz reduces their battery life.  By about half,
> iirc.
> 

Then the owners of such machines can use HZ=250 and leave the default
alone.  Why should everyone have to bear the cost?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[randy_dunlap]

On Sat, 09 Jul 2005 15:16:01 -0400 Lee Revell wrote:

| On Sat, 2005-07-09 at 12:12 -0700, Andrew Morton wrote:
| > Lee Revell <rlrevell@joe-job.com> wrote:
| > >
| > >  > This is not a userspace visible thing really with few exceptions, and
| > >  > well people can select the one they want, right?
| > > 
| > >  Then why not leave the default at 1000?
| > 
| > Because some machines exhibit appreciable latency in entering low power
| > state via ACPI, and 1000Hz reduces their battery life.  By about half,
| > iirc.
| > 
| 
| Then the owners of such machines can use HZ=250 and leave the default
| alone.  Why should everyone have to bear the cost?

indeed, why should everyone have to have 1000 timer interrupts per second?

---
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 13:30 -0700, randy_dunlap wrote:
> | Then the owners of such machines can use HZ=250 and leave the default
> | alone.  Why should everyone have to bear the cost?
> 
> indeed, why should everyone have to have 1000 timer interrupts per second?

So why waste everyone's time with CONFIG_HZ when there are working
dynamic tick solutions out there?  It's just bad release engineering.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Randy Dunlap]

Lee Revell said:
> On Sat, 2005-07-09 at 13:30 -0700, randy_dunlap wrote:
>> | Then the owners of such machines can use HZ=250 and leave the default
>> | alone.  Why should everyone have to bear the cost?
>>
>> indeed, why should everyone have to have 1000 timer interrupts per
>> second?
>
> So why waste everyone's time with CONFIG_HZ when there are working
> dynamic tick solutions out there?  It's just bad release engineering.

hey, that seems to expect some top-level release (or project)
management.  ;)

anyway, I was just trying to point out more than one side to this,
and you have now done the same.  thanks.

-- 
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--On Saturday, July 09, 2005 17:25:58 -0400 Lee Revell <rlrevell@joe-job.com> wrote:

> On Sat, 2005-07-09 at 13:30 -0700, randy_dunlap wrote:
>> | Then the owners of such machines can use HZ=250 and leave the default
>> | alone.  Why should everyone have to bear the cost?
>> 
>> indeed, why should everyone have to have 1000 timer interrupts per second?
> 
> So why waste everyone's time with CONFIG_HZ when there are working
> dynamic tick solutions out there?  It's just bad release engineering.

So on the flip side of this ... why are you complaining about it, instead
of working on the real fix? ;-) 

Having HZ=1000 seems to just be a band-aid for not having sub-hz timers ...
it causes unnecessary overhead for other subsytems.

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

> > Because some machines exhibit appreciable latency in entering low power
> > state via ACPI, and 1000Hz reduces their battery life.  By about half,
> > iirc.
> > 
> Then the owners of such machines can use HZ=250 and leave the default
> alone.  Why should everyone have to bear the cost?

They need 100 really it seems, 250-500 have no real effect and on the
Dell I tried 250 didn't stop the wild clock slew from the APM bios
either. I played with this a fair bit on a couple of laptops. I've not
seen anything > 20% saving however so I've no idea who/why someone saw
50%

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Theodore Ts'o]

On Mon, Jul 11, 2005 at 02:23:08PM +0100, Alan Cox wrote:
> > > Because some machines exhibit appreciable latency in entering low power
> > > state via ACPI, and 1000Hz reduces their battery life.  By about half,
> > > iirc.
> > > 
> > Then the owners of such machines can use HZ=250 and leave the default
> > alone.  Why should everyone have to bear the cost?
> 
> They need 100 really it seems, 250-500 have no real effect and on the
> Dell I tried 250 didn't stop the wild clock slew from the APM bios
> either. I played with this a fair bit on a couple of laptops. I've not
> seen anything > 20% saving however so I've no idea who/why someone saw
> 50%

The real answer here is for the tickless patches to cleaned up to the
point where they can be merged, and then we won't waste battery power
entering the timer interrupt in the first place.  :-)

						- Ted

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Arjan van de Ven]

> The real answer here is for the tickless patches to cleaned up to the
> point where they can be merged, and then we won't waste battery power
> entering the timer interrupt in the first place.  :-)

one big step forward for that is to have a

mod_timer_relative() and add_timer_relative()

which 
1) take a relative delay as argument (and thus kill 99% of the jiffies
use in the kernel right there and then)
2) takes it in miliseconds, killing 99% of all the HZ conversions/uses
3) takes a "accuracy" argument

3) is tricky I guess, it's designed for cases that are like "I want a
timer 1 second from now, but it's ok to be also at 1.5 seconds if that
suits you better". Those cases are far less rare than you might think at
first, most watchdog kind things are of this type. This accuracy thing
will allow the kernel to save many wakeups by grouping them I suspect.

Alan: you worked on this before, where did you end up with ?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

> 3) is tricky I guess, it's designed for cases that are like "I want a
> timer 1 second from now, but it's ok to be also at 1.5 seconds if that
> suits you better". Those cases are far less rare than you might think at
> first, most watchdog kind things are of this type. This accuracy thing
> will allow the kernel to save many wakeups by grouping them I suspect.
> 
> Alan: you worked on this before, where did you end up with ?

For #1/#2 add_timer_relative is an easy wrapper around add_timer for the
moment and I did play with that a bit. Never looked at the accuracy
stuff. In theory its a case of picking existing timeout points for low
resolution timers or tacking them onto an existing timeout that is near
enough. On the pratical side there area few problems with timer
performanc eon insert to consider

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Eric St-Laurent]

On Mon, 2005-07-11 at 16:08 +0200, Arjan van de Ven wrote:

> Alan: you worked on this before, where did you end up with ?
> 

The last patch i've seen is 1 year old.

http://www.ussg.iu.edu/hypermail/linux/kernel/0407.3/0643.html

Eric St-Laurent

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Mon, Jul 11, 2005 at 10:05:10AM -0400, Theodore Ts'o wrote:

> The real answer here is for the tickless patches to cleaned up to
> the point where they can be merged, and then we won't waste battery
> power entering the timer interrupt in the first place.  :-)

Whilst conceptually this is a nice idea I've yet to see any viable
code that overall has a lower cost.  Tickless is a really nice idea
for embedded devices and also paravirtualized hardware but I don't
think anyone has it working well enough yet do they?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Pavel Machek]

Hi!

> > The real answer here is for the tickless patches to cleaned up to
> > the point where they can be merged, and then we won't waste battery
> > power entering the timer interrupt in the first place.  :-)
> 
> Whilst conceptually this is a nice idea I've yet to see any viable
> code that overall has a lower cost.  Tickless is a really nice idea
> for embedded devices and also paravirtualized hardware but I don't
> think anyone has it working well enough yet do they?

Actually for power managment uses, "NO_IDLE_HZ" seems to be enough,
and that is both implemented and working.
								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Theodore Ts'o wrote:
> On Mon, Jul 11, 2005 at 02:23:08PM +0100, Alan Cox wrote:
> 
>>>>Because some machines exhibit appreciable latency in entering low power
>>>>state via ACPI, and 1000Hz reduces their battery life.  By about half,
>>>>iirc.
>>>>
>>>
>>>Then the owners of such machines can use HZ=250 and leave the default
>>>alone.  Why should everyone have to bear the cost?
>>
>>They need 100 really it seems, 250-500 have no real effect and on the
>>Dell I tried 250 didn't stop the wild clock slew from the APM bios
>>either. I played with this a fair bit on a couple of laptops. I've not
>>seen anything > 20% saving however so I've no idea who/why someone saw
>>50%
> 
> 
> The real answer here is for the tickless patches to cleaned up to the
> point where they can be merged, and then we won't waste battery power
> entering the timer interrupt in the first place.  :-)

And that does seem to be the long term solution. Most (not all) modern 
hardware has a readable timer as accurate as the tick, so doing a timer 
to clock conversion as needed would be possible.

Unfortunately the interest in tickless operation seems to be mostly in 
the power saving possibilities of laptops. If you could make it part of 
some really sexy high interest area, like real time premption, it might 
get done sooner ;-)

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 11:59 -0400, Bill Davidsen wrote:
> Theodore Ts'o wrote:
> > The real answer here is for the tickless patches to cleaned up to the
> > point where they can be merged, and then we won't waste battery power
> > entering the timer interrupt in the first place.  :-)
> 
> And that does seem to be the long term solution. Most (not all) modern 
> hardware has a readable timer as accurate as the tick, so doing a timer 
> to clock conversion as needed would be possible.
> 
> Unfortunately the interest in tickless operation seems to be mostly in 
> the power saving possibilities of laptops. If you could make it part of 
> some really sexy high interest area, like real time premption, it might 
> get done sooner ;-)
> 

Actually, there is quite a bit of interest already in the same circles
that are using RT preempt.  "How do I get a timer with better than 1ms
resolution" is an FAQ on linux-audio-dev, and once you have dynamic tick
the next logical step is high res timers.

Currently we refer these users to George's site.  I suspect the only
reason there has not been more interest is that not many users are up to
integrating the HRT and the RT preempt patch.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[pmarques]

Quoting Arjan van de Ven <arjan@infradead.org>:
> [...]
> it's a config option. Some distros ship 100 already, others 1000, again
> others will do 250. What does it matter?
> (Although I still prefer 300 over 250 due to the 50/60 thing)

I just want to point out that while a frequency of 300Hz has good dividers, it
has a lousy period of 3.33... ms. It seems easier to make calculations with a
period of 4 ms per jiffy (250Hz).

The rest of the discussion I'll leave to others as I have no strong feelings
either way.

--
Paulo Marques

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Diego Calleja]

El Sat, 09 Jul 2005 14:16:31 -0400,
Lee Revell <rlrevell@joe-job.com> escribió:

> I still think you're absolutely insane to change the default in the
> middle of a stable kernel series.  People WILL complain about it.


Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
ps).

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 20:39 +0200, Diego Calleja wrote:
> El Sat, 09 Jul 2005 14:16:31 -0400,
> Lee Revell <rlrevell@joe-job.com> escribió:
> 
> > I still think you're absolutely insane to change the default in the
> > middle of a stable kernel series.  People WILL complain about it.
> 
> 
> Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
> stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
> ps).
> 

Yes, that's called "progress" so no one complained.  Going back is
called a "regression".  People don't like those as much.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-09 at 14:41 -0400, Lee Revell wrote:
> Yes, that's called "progress" so no one complained.  Going back is
> called a "regression".  People don't like those as much.

Sorry for the tone of this message, I really sound like a jerk.

Anyway, I've said all I have on this topic.  I don't want a flame war
over this.

BTW, Christoph Lameter, if you're seeing this, your mail is bouncing...

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Sat, Jul 09, 2005 at 02:49:43PM -0400, Lee Revell wrote:

> BTW, Christoph Lameter, if you're seeing this, your mail is bouncing...

my bad, i typoed it when i first send the original email

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

>> Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
>> stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
>> ps).
>> 
> 
> Yes, that's called "progress" so no one complained.  Going back is
> called a "regression".  People don't like those as much.

That's a very subjective viewpoint. Realize that this is a balancing
act between latency and overhead ... and you're firmly only looking
at one side of the argument, instead of taking a comprimise in the
middle ...

If I start arguing for 100HZ on the grounds that it's much more efficient,
will that make 250/300 look much better to you? ;-)

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 11:38 -0700, Martin J. Bligh wrote:
> That's a very subjective viewpoint. Realize that this is a balancing
> act between latency and overhead ... and you're firmly only looking
> at one side of the argument, instead of taking a comprimise in the
> middle ...
> 
> If I start arguing for 100HZ on the grounds that it's much more efficient,
> will that make 250/300 look much better to you? ;-)

Mostly my argument is that all technical arguments aside, it's crazy to
change this in the middle of a stable kernel series.

My other objection is that 90% of the arguments for HZ=250 are based on
battery life.  But most Linux systems still don't run on batteries, so I
object to having to take a performance hit (a latency hit, which is the
same as performance for multimedia apps) for their sake.

Tickless + sub HZ timers is a win for everyone, the multimedia people
get better latency, and the laptop people get to run longer.

I guess CONFIG_HZ makes sense if the tickless solutions are not going to
be ready anytime soon.  But I don't see the problem with leaving the
default at 1000HZ and letting the laptop users lower it.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Friesen]

Lee Revell wrote:

> Tickless + sub HZ timers is a win for everyone, the multimedia people
> get better latency, and the laptop people get to run longer.

IIRC it's not a win for many systems.  Throughput goes down due to timer 
manipulation overhead.

Chris

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 14:39 -0600, Chris Friesen wrote:
> Lee Revell wrote:
> 
> > Tickless + sub HZ timers is a win for everyone, the multimedia people
> > get better latency, and the laptop people get to run longer.
> 
> IIRC it's not a win for many systems.  Throughput goes down due to timer 
> manipulation overhead.

Makes sense.  Anyway, this whole thread has been pretty hand wavey, I
propose that until we see some numbers from the HZ=250 advocates, we
leave the default alone.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--Lee Revell <rlrevell@joe-job.com> wrote (on Monday, July 11, 2005 20:30:59 -0400):

> On Mon, 2005-07-11 at 14:39 -0600, Chris Friesen wrote:
>> Lee Revell wrote:
>> 
>> > Tickless + sub HZ timers is a win for everyone, the multimedia people
>> > get better latency, and the laptop people get to run longer.
>> 
>> IIRC it's not a win for many systems.  Throughput goes down due to timer 
>> manipulation overhead.
> 
> Makes sense.  Anyway, this whole thread has been pretty hand wavey, I
> propose that until we see some numbers from the HZ=250 advocates, we
> leave the default alone.

Odd. Since I showed you some numbers already ... and nobody from the latency
side of the argument has come up with any?

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 21:07 -0700, Martin J. Bligh wrote:
> 
> --Lee Revell <rlrevell@joe-job.com> wrote (on Monday, July 11, 2005 20:30:59 -0400):
> 
> > On Mon, 2005-07-11 at 14:39 -0600, Chris Friesen wrote:
> >> Lee Revell wrote:
> >> 
> >> > Tickless + sub HZ timers is a win for everyone, the multimedia people
> >> > get better latency, and the laptop people get to run longer.
> >> 
> >> IIRC it's not a win for many systems.  Throughput goes down due to timer 
> >> manipulation overhead.
> > 
> > Makes sense.  Anyway, this whole thread has been pretty hand wavey, I
> > propose that until we see some numbers from the HZ=250 advocates, we
> > leave the default alone.
> 
> Odd. Since I showed you some numbers already ... and nobody from the latency
> side of the argument has come up with any?

Sorry, I have not seen any.  Got a link?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 00:13:21 -0400):

> On Mon, 2005-07-11 at 21:07 -0700, Martin J. Bligh wrote:
>> 
>> --Lee Revell <rlrevell@joe-job.com> wrote (on Monday, July 11, 2005 20:30:59 -0400):
>> 
>> > On Mon, 2005-07-11 at 14:39 -0600, Chris Friesen wrote:
>> >> Lee Revell wrote:
>> >> 
>> >> > Tickless + sub HZ timers is a win for everyone, the multimedia people
>> >> > get better latency, and the laptop people get to run longer.
>> >> 
>> >> IIRC it's not a win for many systems.  Throughput goes down due to timer 
>> >> manipulation overhead.
>> > 
>> > Makes sense.  Anyway, this whole thread has been pretty hand wavey, I
>> > propose that until we see some numbers from the HZ=250 advocates, we
>> > leave the default alone.
>> 
>> Odd. Since I showed you some numbers already ... and nobody from the latency
>> side of the argument has come up with any?
> 
> Sorry, I have not seen any.  Got a link?

Look back in the thread. It made kernel compiles about 5% faster on a
fairly large box. I think the SGI people did it originally because it
caused them even larger problems.

I'm not saying their aren't arguments on both sides ... there are. I just
agree with you there's a lot of hand-waving going on ... but probably not
agreeing as to who it's coming from ;-)

Some sort of comprimise has to be struck for now, until we get sub-HZ
timers. I'd prefer 100, personally (I had that set as default in my tree
for a long time). Some people would prefer 1000 or even more, maybe.
250/300 seems like a reasonable comprimise to me. Exactly what problems
*does* it cause (in visible effect, not "timers are less granular").
Jittery audio/video? How much worse is it?

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
> Look back in the thread. It made kernel compiles about 5% faster on a
> fairly large box. I think the SGI people did it originally because it
> caused them even larger problems.
> 

Right, I saw those, but you don't expect to change the default HZ based
on that one test, do you?  How about some numbers for a regular desktop?

> I'm not saying their aren't arguments on both sides ... there are. I
> just agree with you there's a lot of hand-waving going on ... but
> probably not agreeing as to who it's coming from ;-)
> 

Well, I think the burden of proof is on those who are proposing radical
changes, IOW I don't think I should be required to produce any numbers
to justify the status quo.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
> Exactly what problems
> *does* it cause (in visible effect, not "timers are less granular").
> Jittery audio/video? How much worse is it?

Yes, exactly.  Say you need to deliver a frame of audio or video every
5ms.  You have a rendering thread and a display thread that communicate
via FIFOs.  The main thread waits in select() for the next frame to
complete rendering or for the deadline to expire.  That's next to
impossible with HZ=100, because the best you can do is the deadline
+-10ms.  With HZ=1000 it's no problem.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 10:24:59 -0400):

> On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
>> Exactly what problems
>> *does* it cause (in visible effect, not "timers are less granular").
>> Jittery audio/video? How much worse is it?
> 
> Yes, exactly.  Say you need to deliver a frame of audio or video every
> 5ms. 

Ummm. that's a 200HZ refresh rate, is it not? That seems unreasonable 
(to a lay-person, as far as video goes).

> You have a rendering thread and a display thread that communicate
> via FIFOs.  The main thread waits in select() for the next frame to
> complete rendering or for the deadline to expire.  That's next to
> impossible with HZ=100, because the best you can do is the deadline
> +-10ms.  With HZ=1000 it's no problem.

So if we have a 50HZ refresh rate, and a HZ rate of 250 or 300, it'll
work fine then, right? I know that's actually some error in the timers,
so it may be 2 or 3 ticks, not 1, but if we're running HZ at 5 or 6
times the frequency of video, presumably that'd still work fine?

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Tue, 2005-07-12 at 07:56 -0700, Martin J. Bligh wrote:
> --Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 10:24:59 -0400):
> 
> > On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
> >> Exactly what problems
> >> *does* it cause (in visible effect, not "timers are less granular").
> >> Jittery audio/video? How much worse is it?
> > 
> > Yes, exactly.  Say you need to deliver a frame of audio or video every
> > 5ms. 
> 
> Ummm. that's a 200HZ refresh rate, is it not? That seems unreasonable 
> (to a lay-person, as far as video goes).
> 

Seems unreasonable now but things like HDTV actually have much tighter
constraints than this.  And frame times this small are quite often used
for audio, which admittedly is less affected by HZ because those devices
generate their own interrupts.

> > You have a rendering thread and a display thread that communicate
> > via FIFOs.  The main thread waits in select() for the next frame to
> > complete rendering or for the deadline to expire.  That's next to
> > impossible with HZ=100, because the best you can do is the deadline
> > +-10ms.  With HZ=1000 it's no problem.
> 
> So if we have a 50HZ refresh rate, and a HZ rate of 250 or 300, it'll
> work fine then, right? I know that's actually some error in the timers,
> so it may be 2 or 3 ticks, not 1, but if we're running HZ at 5 or 6
> times the frequency of video, presumably that'd still work fine?

Yes, for PAL/NTSC, but it's nice to be forward looking...

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 11:00:02 -0400):

> On Tue, 2005-07-12 at 07:56 -0700, Martin J. Bligh wrote:
>> --Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 10:24:59 -0400):
>> 
>> > On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
>> >> Exactly what problems
>> >> *does* it cause (in visible effect, not "timers are less granular").
>> >> Jittery audio/video? How much worse is it?
>> > 
>> > Yes, exactly.  Say you need to deliver a frame of audio or video every
>> > 5ms. 
>> 
>> Ummm. that's a 200HZ refresh rate, is it not? That seems unreasonable 
>> (to a lay-person, as far as video goes).
>> 
> 
> Seems unreasonable now but things like HDTV actually have much tighter
> constraints than this.  And frame times this small are quite often used
> for audio, which admittedly is less affected by HZ because those devices
> generate their own interrupts.
> 
>> > You have a rendering thread and a display thread that communicate
>> > via FIFOs.  The main thread waits in select() for the next frame to
>> > complete rendering or for the deadline to expire.  That's next to
>> > impossible with HZ=100, because the best you can do is the deadline
>> > +-10ms.  With HZ=1000 it's no problem.
>> 
>> So if we have a 50HZ refresh rate, and a HZ rate of 250 or 300, it'll
>> work fine then, right? I know that's actually some error in the timers,
>> so it may be 2 or 3 ticks, not 1, but if we're running HZ at 5 or 6
>> times the frequency of video, presumably that'd still work fine?
> 
> Yes, for PAL/NTSC, but it's nice to be forward looking...

Well, looking forward, you'll have sub-HZ timers, so none of this will
matter. Actually, looking at the above, 150 seems perfectly reasonable
to me, but 300 seems to be close enough. I'll run some numbers on both.

>From your above email, I'm more convinced than ever that lowering HZ is
the right thing to do ...

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Tue, 2005-07-12 at 08:08 -0700, Martin J. Bligh wrote:
> Well, looking forward, you'll have sub-HZ timers, so none of this will
> matter. Actually, looking at the above, 150 seems perfectly reasonable
> to me, but 300 seems to be close enough. I'll run some numbers on both.
> 
> >From your above email, I'm more convinced than ever that lowering HZ is
> the right thing to do ...

Which we can live with of course, I just wanted to make sure people were
aware of the multimedia side of the argument.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--Lee Revell <rlrevell@joe-job.com> wrote (on Tuesday, July 12, 2005 11:10:13 -0400):

> On Tue, 2005-07-12 at 08:08 -0700, Martin J. Bligh wrote:
>> Well, looking forward, you'll have sub-HZ timers, so none of this will
>> matter. Actually, looking at the above, 150 seems perfectly reasonable
>> to me, but 300 seems to be close enough. I'll run some numbers on both.
>> 
>> > From your above email, I'm more convinced than ever that lowering HZ is
>> the right thing to do ...
> 
> Which we can live with of course, I just wanted to make sure people were
> aware of the multimedia side of the argument.

OK, thanks. and I'm full of shit about 150, too early in the morning. 
300 is the lowest common multiple of 50 and 60.

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Tue, 2005-07-12 at 08:08 -0700, Martin J. Bligh wrote:
> Well, looking forward, you'll have sub-HZ timers, so none of this will
> matter. Actually, looking at the above, 150 seems perfectly reasonable
> to me, but 300 seems to be close enough. I'll run some numbers on
> both.
> 
> From your above email, I'm more convinced than ever that lowering HZ
> is the right thing to do ...

Another example is sending MIDI clock.  This is not possible at HZ=100
as the jitter is too much for external devices to lock on to.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
> Some sort of comprimise has to be struck for now, until we get sub-HZ
> timers. I'd prefer 100, personally (I had that set as default in my tree
> for a long time). Some people would prefer 1000 or even more, maybe.
> 250/300 seems like a reasonable comprimise to me. Exactly what problems
> *does* it cause (in visible effect, not "timers are less granular").
> Jittery audio/video? How much worse is it?

OK, here's a real world example, taken straight from the linux-audio-dev
list today.

Lee

Lee Revell <rlrevell@joe-job.com> :
> On Tue, 2005-07-12 at 20:57 +0200, Kjetil Svalastog Matheussen wrote:
>> E-radium has been tested with both the 2.4 kernel and the 2.6 kernel
>> and with a ~1GhZ machine and a ~2ghz machine. (A 2.4 kernel with a
>> 100hz resolution timer will proably not work very nice though.)
>
> Can you please explain why 100HZ would be a problem for your app?  Right
> now the kernel people are trying to change the default HZ for 2.6 to
> 250.  I have told them that this is insane but they seem inclined to do
> it anyway.
>

The program use poll to sleep. If the resolution of the kernel is 100Hz,
there would sometimes be a too long delay of up to 10ms (and probably beyond)
before the program is woken up,  and before a midi message is sent,
which can cause music to stutter.

Simple as that. :-)

> If you can provide more examples of apps that would be broken by this
> change maybe we can convince them not to change it.
>

Hmm, mplayer I guess...
Don't know how muse, rosegarden, seq24 etc. handles timing...
But all midi-sequencers that doesn't use /dev/rtc could suffer. (?)

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--On Tuesday, July 12, 2005 16:58:44 -0400 Lee Revell <rlrevell@joe-job.com> wrote:

> On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
>> Some sort of comprimise has to be struck for now, until we get sub-HZ
>> timers. I'd prefer 100, personally (I had that set as default in my tree
>> for a long time). Some people would prefer 1000 or even more, maybe.
>> 250/300 seems like a reasonable comprimise to me. Exactly what problems
>> *does* it cause (in visible effect, not "timers are less granular").
>> Jittery audio/video? How much worse is it?
> 
> OK, here's a real world example, taken straight from the linux-audio-dev
> list today.

OK, what level causes Midi stuttering to stop then, under some fairly
reasonable load? Of course ... if we set HZ to 100000, we'll get higher
res still ... the question is how high it *needs* to be ;-)

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Tue, 2005-07-12 at 15:22 -0700, Martin J. Bligh wrote:
> 
> --On Tuesday, July 12, 2005 16:58:44 -0400 Lee Revell <rlrevell@joe-job.com> wrote:
> 
> > On Mon, 2005-07-11 at 21:30 -0700, Martin J. Bligh wrote:
> >> Some sort of comprimise has to be struck for now, until we get sub-HZ
> >> timers. I'd prefer 100, personally (I had that set as default in my tree
> >> for a long time). Some people would prefer 1000 or even more, maybe.
> >> 250/300 seems like a reasonable comprimise to me. Exactly what problems
> >> *does* it cause (in visible effect, not "timers are less granular").
> >> Jittery audio/video? How much worse is it?
> > 
> > OK, here's a real world example, taken straight from the linux-audio-dev
> > list today.
> 
> OK, what level causes Midi stuttering to stop then, under some fairly
> reasonable load? Of course ... if we set HZ to 100000, we'll get higher
> res still ... the question is how high it *needs* to be ;-)

I don't remember the details, but they can be found in the list
archives.  I only remember that 100HZ is not good enough and 1000HZ is
OK.

Anyway it looks like this is a done deal.  If it breaks a lot of apps
then we'll have to deal with it.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jan Engelhardt]

>OK, what level causes Midi stuttering to stop then, under some fairly
>reasonable load? Of course ... if we set HZ to 100000, we'll get higher
>res still ... the question is how high it *needs* to be ;-)

No, some kernel code causes a triple-fault-and-reboot when the HZ is >=
10KHz. Maybe the highest possible value is 8192 Hz, not sure.


Jan Engelhardt
-- 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[George Anzinger]

[-- Attachment #1: Type: text/plain, Size: 2035 bytes --]

Martin J. Bligh wrote:
>>>Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
>>>stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
>>>ps).
>>>
>>
>>Yes, that's called "progress" so no one complained.  Going back is
>>called a "regression".  People don't like those as much.
> 
> 
> That's a very subjective viewpoint. Realize that this is a balancing
> act between latency and overhead ... and you're firmly only looking
> at one side of the argument, instead of taking a compromise in the
> middle ...
> 
> If I start arguing for 100HZ on the grounds that it's much more efficient,
> will that make 250/300 look much better to you? ;-)

I would like to interject an addition data point, and I will NOT be subjective. 
  The nature of the PIT is that it can _hit_ some frequencies better than 
others.  We have had complaints about repeating timers not keeping good time. 
These are not jitter issues, but drift issues.  The standard says we may not 
return early from a timer so any timer will either be on time or late.  The 
amount of lateness depends very much on the HZ value.  Here is what the values 
are for the standard CLOCK_TICK_RATE:

HZ  	TICK RATE	jiffie(ns)	second(ns)	 error (ppbillion)
  100	 1193180	10000000	1000000000	       0
  200	 1193180	 5000098	1000019600	   19600
  250	 1193180	 4000250	1000062500	   62500
  500	 1193180	 1999703	1001851203	 1851203
1000	 1193180	  999848	1000847848	  847848

The jiffie values here are exactly what the kernel uses and are based on the 
best one can do with the PIT hardware.

I am not suggesting any given default HZ, but rather an argumentation of the 
help text that goes with it.  For those who want timers to repeat at one second 
(or multiples there of) this is useful info.

For you enjoyment I have attached the program used to print this.  It allows you 
to try additional values...


-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

[-- Attachment #2: as.c --]
[-- Type: text/x-csrc, Size: 1430 bytes --]



#define NSEC_PER_SEC  1000000000
//#define CLOCK_TICK_RATE /*100000000 */ 1193180
#define LATCH(CLOCK_TICK_RATE,HZ)  ((CLOCK_TICK_RATE + HZ/2) / HZ)
#define SH_DIV(NOM,DEN,LSH) (	((NOM / DEN) << LSH)			\
			     + (((NOM % DEN) << LSH) + DEN / 2) / DEN)
#define ACTHZ (SH_DIV (CLOCK_TICK_RATE, LATCH(CLOCK_TICK_RATE,HZ), 8))
#define TICK_NSEC (SH_DIV (1000000UL * 1000, ACTHZ, 8))


struct {
	int hz;
	int clocktickrate;
} vals[] = {{100, 1193180}, {200, 1193180}, {250, 1193180}, {500, 1193180}, {1000, 1193180},{0,0}};

void do_it(int hz,int tickrate)
{
	int HZ = hz;
	int CLOCK_TICK_RATE = tickrate;
	int tick_nsec = TICK_NSEC;
	int ticks_per_sec = NSEC_PER_SEC/tick_nsec;
	int sec_size = ticks_per_sec * tick_nsec;
	int one_sec_p;
	int err;

	if (sec_size < NSEC_PER_SEC)
		sec_size += tick_nsec;
	one_sec_p = sec_size;
	err = one_sec_p - NSEC_PER_SEC;
	printf( "%4d\t%8d\t%8d\t%10d\t%8d\n",hz, tickrate, tick_nsec, 
		one_sec_p, err);
}
	
void bail(void)
{
	printf("run as: as [hz [clock_tick_rate]]\n");
	exit(1);
}

main(int argc, char** argv)
{
	int i = 0;
	int phz = 0;
	int pcr = vals[0].clocktickrate;

	if (argc > 1) { 
		phz = atoi(argv[1]);
		if (!phz)
			bail();
	}
	if (argc > 2) {
		pcr = atoi(argv[2]);
		if (!pcr)
			bail();
	}

	printf("HZ  \tTICK RATE\tjiffie(ns)\tsecond(ns)\t error (ppbillion)\n");
	while(vals[i].hz) {
		do_it(vals[i].hz, vals[i].clocktickrate);
		i++;
	}
	if (phz)
		do_it(phz, pcr);
}

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Mon, Jul 11, 2005 at 05:38:05PM -0700, George Anzinger wrote:

> I would like to interject an addition data point, and I will NOT be 
> subjective. The nature of the PIT is that it can _hit_ some frequencies 
> better than others.  We have had complaints about repeating timers not 
> keeping good time. These are not jitter issues, but drift issues.  The 
> standard says we may not return early from a timer so any timer will either 
> be on time or late.  The amount of lateness depends very much on the HZ 
> value.  Here is what the values are for the standard CLOCK_TICK_RATE:
> 
> HZ  	TICK RATE	jiffie(ns)	second(ns)	 error (ppbillion)
>  100	 1193180	10000000	1000000000	       0
>  200	 1193180	 5000098	1000019600	   19600
>  250	 1193180	 4000250	1000062500	   62500
>  500	 1193180	 1999703	1001851203	 1851203
> 1000	 1193180	  999848	1000847848	  847848
> 
> The jiffie values here are exactly what the kernel uses and are based on 
> the best one can do with the PIT hardware.

The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
and is divided by 12 to get PIT tick rate

	14.3181818 MHz / 12 = 1193182 Hz

The reality is that the crystal is usually off by 50-100 ppm from the
standard value, depending on temperature. 

    HZ   ticks/jiffie  1 second      error (ppm)
---------------------------------------------------
   100      11932      1.000015238      15.2
   200       5966      1.000015238      15.2
   250       4773      1.000057143      57.1
   300       3977      0.999931429     -68.6
   333       3583      0.999964114     -35.9
   500       2386      0.999847619    -152.4
  1000       1193      0.999847619    -152.4

Some HZ values indeed fit the tick frequency better than others, up to
333 the error is lost in the physical error of the crystal, for 500 and
1000, it definitely is larger, and thus noticeable.

Some (less round and nice) values of HZ would fit even better:

    HZ   ticks/jiffie  1 second      error (ppm)
---------------------------------------------------
    82      14551      1.000000152       0.2
    96      12429      1.000001829       1.8
   209       5709      0.999999314      -0.7
   363       3287      0.999999314      -0.7
   519       2299      0.999999314      -0.7
   864       1381      1.000001829       1.8

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

On Tue, 12 Jul 2005 22:10, Vojtech Pavlik wrote:
> The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> and is divided by 12 to get PIT tick rate
>
> 	14.3181818 MHz / 12 = 1193182 Hz
>
> The reality is that the crystal is usually off by 50-100 ppm from the
> standard value, depending on temperature.
>
>     HZ   ticks/jiffie  1 second      error (ppm)
> ---------------------------------------------------
>    100      11932      1.000015238      15.2
>    200       5966      1.000015238      15.2
>    250       4773      1.000057143      57.1
>    300       3977      0.999931429     -68.6
>    333       3583      0.999964114     -35.9
>    500       2386      0.999847619    -152.4
>   1000       1193      0.999847619    -152.4
>
> Some HZ values indeed fit the tick frequency better than others, up to
> 333 the error is lost in the physical error of the crystal, for 500 and
> 1000, it definitely is larger, and thus noticeable.
>
> Some (less round and nice) values of HZ would fit even better:
>
>     HZ   ticks/jiffie  1 second      error (ppm)
> ---------------------------------------------------
>     82      14551      1.000000152       0.2


Most interesting... Would 838 Hz be a much better choice than 1000 then? 
(apart from the ugliness).

Cheers,
COn

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1515 bytes --]

On Tue, 12 Jul 2005 22:39, Con Kolivas wrote:
> On Tue, 12 Jul 2005 22:10, Vojtech Pavlik wrote:
> > The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> > and is divided by 12 to get PIT tick rate
> >
> > 	14.3181818 MHz / 12 = 1193182 Hz
> >
> > The reality is that the crystal is usually off by 50-100 ppm from the
> > standard value, depending on temperature.
> >
> >     HZ   ticks/jiffie  1 second      error (ppm)
> > ---------------------------------------------------
> >    100      11932      1.000015238      15.2
> >    200       5966      1.000015238      15.2
> >    250       4773      1.000057143      57.1
> >    300       3977      0.999931429     -68.6
> >    333       3583      0.999964114     -35.9
> >    500       2386      0.999847619    -152.4
> >   1000       1193      0.999847619    -152.4
> >
> > Some HZ values indeed fit the tick frequency better than others, up to
> > 333 the error is lost in the physical error of the crystal, for 500 and
> > 1000, it definitely is larger, and thus noticeable.
> >
> > Some (less round and nice) values of HZ would fit even better:
> >
> >     HZ   ticks/jiffie  1 second      error (ppm)
> > ---------------------------------------------------
> >     82      14551      1.000000152       0.2
>
> Most interesting... Would 838 Hz be a much better choice than 1000 then?
> (apart from the ugliness).

Duh I should have looked lower where you suggested 864, sorry.

   864       1381      1.000001829       1.8

Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[George Anzinger]

Con Kolivas wrote:
> On Tue, 12 Jul 2005 22:39, Con Kolivas wrote:
> 
>>On Tue, 12 Jul 2005 22:10, Vojtech Pavlik wrote:
>>
>>>The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
>>>and is divided by 12 to get PIT tick rate
>>>
>>>	14.3181818 MHz / 12 = 1193182 Hz

Yes, but the current code uses 1193180.  Wonder why that is...

>>>
>>>The reality is that the crystal is usually off by 50-100 ppm from the
>>>standard value, depending on temperature.
>>>
>>>    HZ   ticks/jiffie  1 second      error (ppm)
>>>---------------------------------------------------
>>>   100      11932      1.000015238      15.2
>>>   200       5966      1.000015238      15.2
>>>   250       4773      1.000057143      57.1
>>>   300       3977      0.999931429     -68.6
>>>   333       3583      0.999964114     -35.9
>>>   500       2386      0.999847619    -152.4
>>>  1000       1193      0.999847619    -152.4

If we are following the standard and trying to set up a timer, the 1 second time 
MUST be >= 1 second.  Thus the values for 300 and above in this table don't fly. 
  If we are trying to keep system time, well we do just fine at that by using 
the actual value of the jiffie (NOT 1/HZ) when we update time (one of the 
reasons for going to nanoseconds in xtime).  The observable thing the user sees 
is best seen by setting up an itimer to repeat every second.  Then you will see 
the drift AND it will be against the system clock which itself is quite accurate 
(the 50-100ppm you mention), even without ntp.  And the error really is in the 
range of 848ppm for HZ=1000 BECAUSE we need to follow the standard.  You can 
easily see this with the current 2.6 kernel.  We even have a bug report on it:

http://bugzilla.kernel.org/show_bug.cgi?id=3289
~
-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Tue, Jul 12, 2005 at 08:57:06AM -0700, George Anzinger wrote:

> >>>The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> >>>and is divided by 12 to get PIT tick rate
> >>>
> >>>	14.3181818 MHz / 12 = 1193182 Hz
> 
> Yes, but the current code uses 1193180.  Wonder why that is...

Because it was stated so in some ancient DOS docs? I really don't know.
x86-64 uses 1193182. This is because when I was doing x86-64 timing
code, I had observably better results with this number.

> >>>   HZ   ticks/jiffie  1 second      error (ppm)
> >>>---------------------------------------------------
> >>>  100      11932      1.000015238      15.2
> >>>  200       5966      1.000015238      15.2
> >>>  250       4773      1.000057143      57.1
> >>>  300       3977      0.999931429     -68.6
> >>>  333       3583      0.999964114     -35.9
> >>>  500       2386      0.999847619    -152.4
> >>> 1000       1193      0.999847619    -152.4
> 
> If we are following the standard and trying to set up a timer, the 1
> second time MUST be >= 1 second.  Thus the values for 300 and above in
> this table don't fly. 

In that case, the table will look like this (with the CGA clock fixed to
14.31818000 MHz):

   HZ   ticks/jiffie  1 second      error (ppm)
---------------------------------------------------
   100      11932      1.000015365      15.4
   200       5966      1.000015365      15.4
   250       4773      1.000057270      57.3
   300       3978      1.000182984     183.0
   500       2387      1.000266794     266.8
  1000       1194      1.000685841     685.8
---------------------------------------------------
    82      14551      1.000000279       0.3
   216       5524      1.000001956       2.0
   432       2762      1.000001956       2.0
   864       1381      1.000001956       2.0
  1381        864      1.000001956       2.0


> If we are trying to keep system time, we'll we do just fine at 
> that by using the actual value of the jiffie (NOT 1/HZ) when we update time 
> (one of the reasons for going to nanoseconds in xtime).

Yes, adding (12.0/14318180)*LATCH to xtime in each timer interrupt,
instead of (1.0/HZ) indeed eliminates the error entirely.

> The observable thing the user sees is best seen by setting up an
> itimer to repeat every second.  Then you will see the drift AND it
> will be against the system clock which itself is quite accurate (the
> 50-100ppm you mention), even without ntp.  And the error really is in
> the range of 848ppm for HZ=1000 BECAUSE we need to follow the
> standard.  You can easily see this with the current 2.6 kernel.  We
> even have a bug report on it:
> 
> http://bugzilla.kernel.org/show_bug.cgi?id=3289
 
Going to HZ=864 would fix this problem. It would likely cause other
problems in places that expect 1/HZ to be a sane number, though.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Vojtech Pavlik wrote:
> On Tue, Jul 12, 2005 at 08:57:06AM -0700, George Anzinger wrote:
> 
> 
>>>>>The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
>>>>>and is divided by 12 to get PIT tick rate
>>>>>
>>>>>	14.3181818 MHz / 12 = 1193182 Hz
>>
>>Yes, but the current code uses 1193180.  Wonder why that is...
> 
> 
> Because it was stated so in some ancient DOS docs? I really don't know.
> x86-64 uses 1193182. This is because when I was doing x86-64 timing
> code, I had observably better results with this number.
> 
> 
>>>>>  HZ   ticks/jiffie  1 second      error (ppm)
>>>>>---------------------------------------------------
>>>>> 100      11932      1.000015238      15.2
>>>>> 200       5966      1.000015238      15.2
>>>>> 250       4773      1.000057143      57.1
>>>>> 300       3977      0.999931429     -68.6
>>>>> 333       3583      0.999964114     -35.9
>>>>> 500       2386      0.999847619    -152.4
>>>>>1000       1193      0.999847619    -152.4
>>
>>If we are following the standard and trying to set up a timer, the 1
>>second time MUST be >= 1 second.  Thus the values for 300 and above in
>>this table don't fly. 
> 
> 
> In that case, the table will look like this (with the CGA clock fixed to
> 14.31818000 MHz):
> 
>    HZ   ticks/jiffie  1 second      error (ppm)
> ---------------------------------------------------
>    100      11932      1.000015365      15.4
>    200       5966      1.000015365      15.4
>    250       4773      1.000057270      57.3
>    300       3978      1.000182984     183.0
>    500       2387      1.000266794     266.8
>   1000       1194      1.000685841     685.8
> ---------------------------------------------------
>     82      14551      1.000000279       0.3
>    216       5524      1.000001956       2.0
>    432       2762      1.000001956       2.0
>    864       1381      1.000001956       2.0
>   1381        864      1.000001956       2.0
> 
> 
> 
>>If we are trying to keep system time, we'll we do just fine at 
>>that by using the actual value of the jiffie (NOT 1/HZ) when we update time 
>>(one of the reasons for going to nanoseconds in xtime).
> 
> 
> Yes, adding (12.0/14318180)*LATCH to xtime in each timer interrupt,
> instead of (1.0/HZ) indeed eliminates the error entirely.
> 
> 
>>The observable thing the user sees is best seen by setting up an
>>itimer to repeat every second.  Then you will see the drift AND it
>>will be against the system clock which itself is quite accurate (the
>>50-100ppm you mention), even without ntp.  And the error really is in
>>the range of 848ppm for HZ=1000 BECAUSE we need to follow the
>>standard.  You can easily see this with the current 2.6 kernel.  We
>>even have a bug report on it:
>>
>>http://bugzilla.kernel.org/show_bug.cgi?id=3289
> 
>  
> Going to HZ=864 would fix this problem. It would likely cause other
> problems in places that expect 1/HZ to be a sane number, though.
> 
But if you are going to an "odd" value, would 1381 would be a better 
choice, given the interest in minimizing latency currently evident? The 
overhead would be a little higher, of course, but the people who want to 
drop to 250 for battery life would probably want 216 anyway.

How serious is the 1/HZ = sane problem, and more to the point how many 
programs get the HZ value with a system call as opposed to including a 
header or building it in? I know some of my older programs use header 
files, that was part of the planning for the future even before 2.5 
started. At the time I didn't expect to have to use the system call.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 12:26 -0400, Bill Davidsen wrote:
> >  
> > Going to HZ=864 would fix this problem. It would likely cause other
> > problems in places that expect 1/HZ to be a sane number, though.
> > 
> But if you are going to an "odd" value, would 1381 would be a better 
> choice, given the interest in minimizing latency currently evident?

No, I think 864 is quite good enough.  1381 seems overkill even for
multimedia, people who need that can use the RTC anyway.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[David Lang]

On Wed, 13 Jul 2005, Bill Davidsen wrote:

> How serious is the 1/HZ = sane problem, and more to the point how many 
> programs get the HZ value with a system call as opposed to including a header 
> or building it in? I know some of my older programs use header files, that 
> was part of the planning for the future even before 2.5 started. At the time 
> I didn't expect to have to use the system call.

in binary 1/100 or 1/1000 are not sane values to start with so I don't 
think that that this is likly to be that critical (remembering that the 
kernel doesn't do floating point math)

David Lang

-- 
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
  -- C.A.R. Hoare

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Wed, Jul 13, 2005 at 10:24:10AM -0700, David Lang wrote:

> >How serious is the 1/HZ = sane problem, and more to the point how many 
> >programs get the HZ value with a system call as opposed to including a 
> >header or building it in? I know some of my older programs use header 
> >files, that was part of the planning for the future even before 2.5 
> >started. At the time I didn't expect to have to use the system call.
> 
> in binary 1/100 or 1/1000 are not sane values to start with so I don't 
> think that that this is likly to be that critical (remembering that the 
> kernel doesn't do floating point math)
 
No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
a counter that counts the ticks in nanoseconds (xtime ...), the first
will be exact, the second will be accumulating an error.

It's a tradeoff.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Wed, 13 Jul 2005, Vojtech Pavlik wrote:
>
> No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
> a counter that counts the ticks in nanoseconds (xtime ...), the first
> will be exact, the second will be accumulating an error.

It's not even that we have a counter like that, it's the simple fact that
we have a standard interface to user space that is based on milli-, micro-
and nanoseconds.

(For "poll()", "struct timeval" and "struct timespec" respectively).

It's totally pointless saying that we can do 864 Hz "exactly", when the
fact is that all the timeouts we ever get from user space aren't in that 
format. So the only thing that matters is how close to a millisecond we 
can get, not how close to some random number.

So we do a lot of conversions from "struct timeval" to "jiffies", and if
you don't take the error in that conversion into account, then you're
ignoring what is likely a _bigger_ error.

Long-term time drift is a known issue, and is unavoidable since you don't 
even know the exact frequency of the crystal, since that is not only not 
that exact in the first place, it depends on temperature etc. So long-term 
time drift is something that we inevitably have to use things like NTP to 
handle, if you want an exact clock.

And in short-term things, the timeval/jiffie conversion is likely to be a 
_bigger_ issue than the crystal frequency conversion.

So we should aim for a HZ value that makes it easy to convert to and from
the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
good values for that reason. 864 is not.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> So we should aim for a HZ value that makes it easy to convert to and from
> the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> good values for that reason. 864 is not.

How about 500?  This might be good enough to solve the MIDI problem.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Dmitry Torokhov]

Hi,

On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
> On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> > So we should aim for a HZ value that makes it easy to convert to and from
> > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> > good values for that reason. 864 is not.
> 
> How about 500?  This might be good enough to solve the MIDI problem.
>

I would expect number of laptop users significatly outnumber ones
driving MIDI so as a default entry 250 makes more sense IMHO.
 
-- 
Dmitry

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--On Wednesday, July 13, 2005 14:32:02 -0500 Dmitry Torokhov <dmitry.torokhov@gmail.com> wrote:

> Hi,
> 
> On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
>> On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
>> > So we should aim for a HZ value that makes it easy to convert to and from
>> > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
>> > good values for that reason. 864 is not.
>> 
>> How about 500?  This might be good enough to solve the MIDI problem.
>> 
> 
> I would expect number of laptop users significatly outnumber ones
> driving MIDI so as a default entry 250 makes more sense IMHO.

Could someone actually test it, rather than randomly guessing? ;-)

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Fernando Lopez-Lezcano]

On Wed, 2005-07-13 at 12:33, Martin J. Bligh wrote:
> --On Wednesday, July 13, 2005 14:32:02 -0500 Dmitry Torokhov <dmitry.torokhov@gmail.com> wrote:
> 
> > Hi,
> > 
> > On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
> >> On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> >> > So we should aim for a HZ value that makes it easy to convert to and from
> >> > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> >> > good values for that reason. 864 is not.
> >> 
> >> How about 500?  This might be good enough to solve the MIDI problem.
> >> 
> > 
> > I would expect number of laptop users significatly outnumber ones
> > driving MIDI so as a default entry 250 makes more sense IMHO.
> 
> Could someone actually test it, rather than randomly guessing? ;-)

A Midi NoteOn command (ie: the equivalent of pressing a note on a
musical keyboard) transmitted through a physical Midi interface is
around 1 millisecond long with no running status, or 0.64 msec long with
running status going on (that means one note in between other note
on's)[*]. 

Scheduling of midi output should rely on a timing source that has a
better timing resolution than that. So, HZ=1000 let's us sort of
correctly schedule note-on (and similar sized events). It already
introduces jitter in two byte critical messages like MTC (Midi Time
code) Quarter Frame messages.

-- Fernando

[*] basic midi bit rate is 31250, one midi byte is 10 bits long,
commands can be one, two, three or more bytes long. Midi commands are
made of one status byte and optional data bytes, the status byte can be
ommited on multibyte messages for consecutive messages that are of the
same type (called "running status"). Real time commands such as Midi
Clock can be sent at any time, even in the middle of an already ongoing
midi message. 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 12:33 -0700, Martin J. Bligh wrote:
> 
> --On Wednesday, July 13, 2005 14:32:02 -0500 Dmitry Torokhov <dmitry.torokhov@gmail.com> wrote:
> 
> > Hi,
> > 
> > On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
> >> On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> >> > So we should aim for a HZ value that makes it easy to convert to and from
> >> > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> >> > good values for that reason. 864 is not.
> >> 
> >> How about 500?  This might be good enough to solve the MIDI problem.
> >> 
> > 
> > I would expect number of laptop users significatly outnumber ones
> > driving MIDI so as a default entry 250 makes more sense IMHO.
> 
> Could someone actually test it, rather than randomly guessing? ;-)
> 

It's not just random guessing, the relationship between HZ and MIDI
clock is well known.  But it's not trivial to give hard numbers because
of the range of hardware possibilities.  Here's another linux-audio-dev
excerpt:

On Thu, 2004-07-01 at 16:32 +0200, Tim Goetze wrote: 
> [Paul Davis]
> 
> >until the high resolution clock timers patch is solid enough to be
> >used by any system, there is no way to schedule MIDI output with this
> >kind of resolution, and if you can't schedule it, then the receiver of
> >your MIDI clock signal will see a lot of jitter and may refuse to lock
> >to it. Even if it locks, its not clear what it will do with the jitter.
> 
> fwiw, i'm achieving quite satisfying results driving MIDI out from a
> 1024 Hz RTC thread, with external hardware locking steadily onto the
> output MIDI clock stream, even at tempi up to 240 bpm.
> 
> MIDI out jitter is about the audio block size at max. DSP load (~1.3
> ms) during audio processing cycles, a fraction of a millisecond for
> the difference between 1024 Hz and the wanted MIDI clock frequency
> otherwise; however this seems to be no problem for the hardware
> attached (a fairly recent synthesizer and infrequently an aging cheap
> drum machine).
> 
> at lower RTC frequencies, the jitter effect on the MIDI h/w becomes
> noticeable (erratic rubato) but it still locks on.
> 

I guess there's a good argument that we should have stuck with the RTC
like in 2.4, but sleep() based timing is a much nicer interface, and it
did work in 2.6.0 - 2.6.12 so the MIDI people have grown used to it.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 14:32 -0500, Dmitry Torokhov wrote:
> Hi,
> 
> On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
> > On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> > > So we should aim for a HZ value that makes it easy to convert to and from
> > > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> > > good values for that reason. 864 is not.
> > 
> > How about 500?  This might be good enough to solve the MIDI problem.
> >
> 
> I would expect number of laptop users significatly outnumber ones
> driving MIDI so as a default entry 250 makes more sense IMHO.
>  

Alan tested it and said that 250HZ does not save much power anyway.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andrew Morton]

Lee Revell <rlrevell@joe-job.com> wrote:
>
> On Wed, 2005-07-13 at 14:32 -0500, Dmitry Torokhov wrote:
> > Hi,
> > 
> > On 7/13/05, Lee Revell <rlrevell@joe-job.com> wrote:
> > > On Wed, 2005-07-13 at 12:10 -0700, Linus Torvalds wrote:
> > > > So we should aim for a HZ value that makes it easy to convert to and from
> > > > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> > > > good values for that reason. 864 is not.
> > > 
> > > How about 500?  This might be good enough to solve the MIDI problem.
> > >
> > 
> > I would expect number of laptop users significatly outnumber ones
> > driving MIDI so as a default entry 250 makes more sense IMHO.
> >  
> 
> Alan tested it and said that 250HZ does not save much power anyway.
> 

Len Brown, a year ago: "The bottom line number to laptop users is battery
lifetime.  Just today somebody complained to me that Windows gets twice the
battery life that Linux does."

And "Maybe I can get Andy Grover over in the moble lab to get some time on
that fancy power measurement setup they have...

"My expectation is if we want to beat the competition, we'll want the
ability to go *under* 100Hz."

But then, power consumption of the display should preponderate, so it's not
clear.

Len, any updates on the relationship between HZ and power consumption?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Wed, Jul 13, 2005 at 01:48:57PM -0700, Andrew Morton wrote:

> Len Brown, a year ago: "The bottom line number to laptop users is
> battery lifetime.  Just today somebody complained to me that Windows
> gets twice the battery life that Linux does."

It seems the motivation for lower HZ is really:

   (1) ACPI/SMM suckage in laptops

   (2) NUMA systems with *horrible* remote memory latencies

Both can be detected from you .config and we could see HZ as needed
there and everyone else could avoid this surely?

The above one year old comment where it says "someone complained"
makes me wonder if we ever had an original report with hard facts or
just some internal/private comment/bug about how things seem.

Surely we can test this accurately?

> "My expectation is if we want to beat the competition, we'll want
> the ability to go *under* 100Hz."

What does Windows do here?

> Len, any updates on the relationship between HZ and power
> consumption?

I can measure this[1] for some *old* hardware but as it has unusable
behavior for ACPI I'm not sure how useful that is.


[1] I was just going to remove batteries and run it from a bench
    supply in the lab and measure how much current was being drawn.
    Can anyone suggest something more sensible than that?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 14:16 -0700, Chris Wedgwood wrote:
> Both can be detected from you .config and we could see HZ as needed
> there and everyone else could avoid this surely?

Does anyone object to setting HZ at boot?  I suspect nothing else will
make everyone happy.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

--On Wednesday, July 13, 2005 17:24:41 -0400 Lee Revell <rlrevell@joe-job.com> wrote:

> On Wed, 2005-07-13 at 14:16 -0700, Chris Wedgwood wrote:
>> Both can be detected from you .config and we could see HZ as needed
>> there and everyone else could avoid this surely?
> 
> Does anyone object to setting HZ at boot?  I suspect nothing else will
> make everyone happy.

Having the option is good ... the same arguments about creating a sensible
default still applies though (whatever that is). Creating yet more frigging
tunables for users to manage is not really a good way out ...

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Friesen]

Lee Revell wrote:

> Does anyone object to setting HZ at boot?  I suspect nothing else will
> make everyone happy.

Makes it harder to optimize the conversions, doesn't it?  Knowing the HZ 
value at compile-time would allow the compiler to do it's thing better 
in some cases.

Chris

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Wed, Jul 13, 2005 at 05:24:41PM -0400, Lee Revell wrote:

> Does anyone object to setting HZ at boot?  I suspect nothing else
> will make everyone happy.

Does it bloat the code or slow things measurably?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[George Anzinger]

Lee Revell wrote:
> On Wed, 2005-07-13 at 14:16 -0700, Chris Wedgwood wrote:
> 
>>Both can be detected from you .config and we could see HZ as needed
>>there and everyone else could avoid this surely?

> 
> Does anyone object to setting HZ at boot?  I suspect nothing else will
> make everyone happy.
> 
This will really mess up the jiffie_to_* and *_to_jiffie conversions.  They rely 
in a rather large way on the complier doing all the heavy lifting.  If HZ is a 
variable we introduce a LOT of runtime overhead here.  (Try make kernel/itimer.i 
and look for jiffies_to_t* and friends.)


-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

>> Len Brown, a year ago: "The bottom line number to laptop users is
>> battery lifetime.  Just today somebody complained to me that Windows
>> gets twice the battery life that Linux does."
> 
> It seems the motivation for lower HZ is really:
> 
>    (1) ACPI/SMM suckage in laptops
> 
>    (2) NUMA systems with *horrible* remote memory latencies

It makes a difference on more normal SMP systems as well, just not as
much. See earlier in the thread. The NUMA system I used as an example
was actually a newer one with something like a 4:1 ratio, not an older
one with 20:1 or so. I have a feeling it's more to do with the number
of procs and the scheduler being invoked more than it is really to do
with NUMA ratios.
 
It seems people are agreed we want sub-HZ timers, and eventually go
to tickless ... the question is more what to do in the meantime.

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 14:16 -0700, Chris Wedgwood wrote:
>    (1) ACPI/SMM suckage in laptops

Anything that loses ticks at 1000HZ is unsuitable for serious multimedia
use anyway, so I think this part of the argument isn't as important.
Also, I don't know that anyone has a list of machines with these
ACPI/SMM bugs.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[dean gaudet]

On Wed, 13 Jul 2005, Chris Wedgwood wrote:

> On Wed, Jul 13, 2005 at 01:48:57PM -0700, Andrew Morton wrote:
> > "My expectation is if we want to beat the competition, we'll want
> > the ability to go *under* 100Hz."
> 
> What does Windows do here?

windows xp base rate is 100Hz... but multimedia apps can ask for almost 
any rate they want (depends on the hw capabilities).  i recall seeing 
rates >1200Hz when you launch some of the media player apps -- sorry i 
forget the exact number.

i don't think it goes slower than 100Hz.

-dean

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Petr Vandrovec]

dean gaudet wrote:
> On Wed, 13 Jul 2005, Chris Wedgwood wrote:
> 
> 
>>On Wed, Jul 13, 2005 at 01:48:57PM -0700, Andrew Morton wrote:
>>
>>>"My expectation is if we want to beat the competition, we'll want
>>>the ability to go *under* 100Hz."
>>
>>What does Windows do here?
> 
> 
> windows xp base rate is 100Hz... but multimedia apps can ask for almost 

83Hz

> any rate they want (depends on the hw capabilities).  i recall seeing 
> rates >1200Hz when you launch some of the media player apps -- sorry i 
> forget the exact number.

When you have (exactly one) VMware's VM with WindowsXP running on the box,
you can track guest's tick frequency changes in `dmesg` as vmmon reports
frequency guest wants while reprogramming /dev/rtc.

Highest frequency I ever saw is 2.6 Linux with local APIC - it needs 2kHz, as
1kHz timer from 8253 and from local APIC are shifted appart by exactly half
of their period.
							Petr Vandrovec

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jan Engelhardt]

>>>> "My expectation is if we want to beat the competition, we'll want
>>>> the ability to go *under* 100Hz."
>>> 
>>> What does Windows do here?
>>
>> windows xp base rate is 100Hz... but multimedia apps can ask for almost 
>
> 83Hz

Well, Windoes 98 (vmmon) shows very different ones:

/dev/vmmon[4355]: host clock rate change request 0 -> 19
/dev/vmmon[4355]: host clock rate change request 19 -> 0
/dev/vmmon[4355]: host clock rate change request 0 -> 19
/dev/vmmon[4355]: host clock rate change request 19 -> 63
/dev/vmmon[4355]: host clock rate change request 63 -> 200
/dev/vmmon[4355]: host clock rate change request 200 -> 201
/dev/vmmon[4355]: host clock rate change request 201 -> 1001

>> any rate they want (depends on the hw capabilities).  i recall seeing
>> rates >1200Hz when you launch some of the media player apps -- sorry i
>> forget the exact number.

I have seen some apps which seem to schedule themselves using some kind of
SCHED_FIFO and therefore seem to get good RT:

from an ini file...
  # This option determines the multi-tasking capabilities of WinDEU.
  # The priority determines the minimum number of milliseconds WinDEU
  # will work before giving control back to Windows.
  # For example, if you set it to 20, it means WinDEU will gives
  # back control to Windows approximately (at most) 50 times a second.
  # A value of 0 means WinDEU WON'T multi-task.
  # (Can be changed in the preferences dialog box.)
  BuildPriority=25



Jan Engelhardt
-- 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 11:24 +0200, Jan Engelhardt wrote:
> >>>> "My expectation is if we want to beat the competition, we'll want
> >>>> the ability to go *under* 100Hz."
> >>> 
> >>> What does Windows do here?
> >>
> >> windows xp base rate is 100Hz... but multimedia apps can ask for almost 
> >
> > 83Hz
> 
> Well, Windoes 98 (vmmon) shows very different ones:

Wow.  Windows has been doing this since *98*?

So that's what Paul meant by "the stupidity of a fixed HZ, which is so
early '90s that its embarrassing".

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jan Engelhardt]

>> >>> What does Windows do here?
>> >> windows xp base rate is 100Hz... but multimedia apps can ask for almost 
>> > 83Hz
>> Well, Windoes 98 (vmmon) shows very different ones:
>Wow.  Windows has been doing this since *98*?

...since Windows does multitask scheduling I suppose, which is since 95 or
NT I suppose.



Jan Engelhardt
-- 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Jan Engelhardt wrote:
>
> >> >>> What does Windows do here?
> >> >> windows xp base rate is 100Hz... but multimedia apps can ask for almost 
> >> > 83Hz
> >> Well, Windoes 98 (vmmon) shows very different ones:
> >Wow.  Windows has been doing this since *98*?
> 
> ...since Windows does multitask scheduling I suppose, which is since 95 or
> NT I suppose.

I wouldn't be surprised if windows did it from day 1.

Windows started out as a program loader and a somewhat graphical shell. I
think people forget the early trials, ie before Win-3.0, which ended up
mostly running DOS programs. With a timer frequency of 18.2Hz or something
like that.

So I wouldn't be surprised if Win-1.0 used to switch between different 
frequencies..

			Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Wed, Jul 13, 2005 at 04:41:41PM -0700, dean gaudet wrote:

> windows xp base rate is 100Hz... but multimedia apps can ask for
> almost any rate they want (depends on the hw capabilities).  i
> recall seeing rates >1200Hz when you launch some of the media player
> apps -- sorry i forget the exact number.

Windows starts an additional high-speed timer as needed for this?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[dean gaudet]

On Wed, 13 Jul 2005, Chris Wedgwood wrote:

> On Wed, Jul 13, 2005 at 04:41:41PM -0700, dean gaudet wrote:
> 
> > windows xp base rate is 100Hz... but multimedia apps can ask for
> > almost any rate they want (depends on the hw capabilities).  i
> > recall seeing rates >1200Hz when you launch some of the media player
> > apps -- sorry i forget the exact number.
> 
> Windows starts an additional high-speed timer as needed for this?

i don't think so -- i think it reprograms the divisor... but don't trust 
me, check this doc:

http://www.microsoft.com/whdc/system/CEC/mm-timer.mspx

it goes into detail on the current and future timer support in winxp.

-dean

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 18:13 -0700, dean gaudet wrote:
> On Wed, 13 Jul 2005, Chris Wedgwood wrote:
> 
> > On Wed, Jul 13, 2005 at 04:41:41PM -0700, dean gaudet wrote:
> > 
> > > windows xp base rate is 100Hz... but multimedia apps can ask for
> > > almost any rate they want (depends on the hw capabilities).  i
> > > recall seeing rates >1200Hz when you launch some of the media player
> > > apps -- sorry i forget the exact number.
> > 
> > Windows starts an additional high-speed timer as needed for this?
> 
> i don't think so -- i think it reprograms the divisor... but don't trust 
> me, check this doc:
> 
> http://www.microsoft.com/whdc/system/CEC/mm-timer.mspx
> 
> it goes into detail on the current and future timer support in winxp.

Interesting.  First they say it's impractical to reprogram the PIT, then
they later imply that's exactly what Windows does, though for some
reason they don't come out and say it.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Wed, 13 Jul 2005, Lee Revell wrote:
> 
> Interesting.  First they say it's impractical to reprogram the PIT, then
> they later imply that's exactly what Windows does, though for some
> reason they don't come out and say it.

I suspect that it is impractical to reprogram the PIT on a very fine
granularity.

Btw, if somebody really gets excited about all this, let me say (once 
again) what I think might be an acceptable situation.

First off, I'm _not_ a believer in "sub-HZ ticks". Quite the reverse. I 
think we should have HZ be some high value, but we would _slow_down_ the 
tick when not needed, and count by 2's, 3's or even 10's when there's not 
a lot going on.

In other words, I don't think we want a _highfrequency_ timer, I want a
_lower_ frequency mode.

So let's say that we raise HZ to 2000, or somethign that we decide is the
upper limit of sanity. We then have some timer logic entity that notices
that nothing is going to care for the next 100 ticks, so we go into "slow
mode", and reprogram the timer to tick at a frequency of 100Hz, but when
it does tick, we just count it as 20.

IOW, nothing ever sees any "variable frequency", and there's never any 
question about what the timer tick is: the timer tick is 2kHz as far as 
everybody is concerned. It's just that the ticks sometimes come in 
"bunches of 20".

This also means that there is never any issue of the timer running wild. 
The _most_ it will ever run at is limited quite naturally, and some crazy 
user asking for a 1ns itimer won't make any difference at all to the 
system.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Arjan van de Ven]

> IOW, nothing ever sees any "variable frequency", and there's never any 
> question about what the timer tick is: the timer tick is 2kHz as far as 
> everybody is concerned. It's just that the ticks sometimes come in 
> "bunches of 20".

btw we can hide all of this a lot nicer from just about the entire
kernel by reducing the usage of both HZ and jiffies in drivers/non
platform code. That isn't hard; msleep() is a good step forward there
already; the next step is a nicer api for add_timer/mod_timer that is
both relative and in miliseconds; with those 2 the majority of code that
has "knowledge" about this shrinks to near zero. Once we have that the
actual implementation of this in the background matters a whole lot
less.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Thu, Jul 14, 2005 at 09:42:18AM +0200, Arjan van de Ven wrote:

> > IOW, nothing ever sees any "variable frequency", and there's never any 
> > question about what the timer tick is: the timer tick is 2kHz as far as 
> > everybody is concerned. It's just that the ticks sometimes come in 
> > "bunches of 20".
> 
> btw we can hide all of this a lot nicer from just about the entire
> kernel by reducing the usage of both HZ and jiffies in drivers/non
> platform code. That isn't hard; msleep() is a good step forward there
> already; the next step is a nicer api for add_timer/mod_timer that is
> both relative and in miliseconds; with those 2 the majority of code that
> has "knowledge" about this shrinks to near zero. Once we have that the
> actual implementation of this in the background matters a whole lot
> less.
 
A note on the relaive timer API: There needs to be a way to say
"x milliseconds from the time this timer should have triggered" instead
of "x milliseconds from now", to avoid skew in timers that try to be
strictly periodic.

But other than that - such an API would be a great thing for drivers.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Vojtech Pavlik wrote:
>  
> A note on the relaive timer API: There needs to be a way to say
> "x milliseconds from the time this timer should have triggered" instead
> of "x milliseconds from now", to avoid skew in timers that try to be
> strictly periodic.

I disagree.

There should be an _absolute_ interface, and a driver that wants that 
should just have calculated when in time the timeout finishes - and then 
keep on using the absolute value.

Btw, this is exactly why the jiffy-based thing is _good_. The kernel 
timers _are_ absolute, and you make them relative by adding "jiffies".

The fact is, the current timers are better than people give them credit 
for, and converting them away from a jiffies-based interface (to a 
usleep-like one) is STUPID.

There's absolutely nothing wrong with "jiffies", and anybody who thinks 
that

	msleep(20);

is fundamentally better than

	timeout = jiffies + HZ/50;

just doesn't realize that the latter is a bit more complicated exactly 
because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
jiffies for some religious reason is _stupid_.

I have to say, this whole thread has been pretty damn worthless in general 
in my not-so-humble opinion.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Arjan van de Ven]

On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:


> There should be an _absolute_ interface

I'm not arguing there shouldn't be an absolute interface. I'm arguing
that *most* uses are relative, and as such a relative interface makes
sense for those cases.


> Btw, this is exactly why the jiffy-based thing is _good_. The kernel 
> timers _are_ absolute, and you make them relative by adding "jiffies".

again there is absolutely nothing wrong with having absolute timers and
a general notion of absolute time. Jiffies is one way of achieving that,
and it's the current linux way. I see the "absolute timers are good"
argument sort of separate from "jiffies / HZ are good" argument; there
is no principal reason why such an interface couldn't be in say usec.


> There's absolutely nothing wrong with "jiffies", and anybody who thinks 
> that
> 
> 	msleep(20);
> 
> is fundamentally better than
>
> 	timeout = jiffies + HZ/50;

I *will* argue that for relative delays in drivers, msleep() is better.
The reason is different than you think of; the argument why I consider
msleep() better as interface for relative delays in drivers is that it
is harder for a driver writer to get wrong, by virtue of it being
simpler. jiffies and HZ conversion is one of those areas that driver
writers very often get wrong. (multiply by HZ not divide for example,
but there's a few dozen ways it can and does go wrong). A relative msec
based interface is a LOT harder to get wrong, and also often is closer
to what the datasheet of the hardware says. I'm not going to say "all
driver writers are stupid" because they're not; however too many of them
just act like they are too much of the time. That doesn't mean that
there is no room for a "powerful interface" next to a simple one, and I
hope you're not fully against adding a simple interface on top of a more
powerful one if that simple interface is a way to reduce mistakes and
thus bugs in drivers.


> just doesn't realize that the latter is a bit more complicated exactly 
> because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
> jiffies for some religious reason is _stupid_.

I have nothing religious against jiffies per se. My argument however is
that with a few simple, relative interfaces *in addition* to an absolute
interface, almost all drivers suddenly are isolated from jiffies and HZ
because they simply don't care. Because they really DON'T care about
absolute time. At all. 

Doing this will in turn open up flexibility in experimenting with how
one implements the timer stuff; there's suddenly a lot less code to
touch in doing so. Also such relative interface can match the intent a
lot better and separated from the actual implementation. 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Arjan van de Ven wrote:
> 
> I *will* argue that for relative delays in drivers, msleep() is better.

Oh, I agree.

I think we should continue the simplification of the simple stuff. If 
somebody just wants to sleep a while, do it.

But if somebody is doing this because they think "jiffies" are evil, then 
that somebody is being a total idiot. Jiffies aren't evil. They are 
basically the most efficient way to handle any kind of absolute timeouts, 
and any "repeating timeout" _needs_ to be written in absolute terms.

And quite frankly, in this discussion I have several times seen the 
argument that we should be getting rid of jiffies because somebody thinks 
jiffies are "hard" and somehow incompatible with "tickless".

THAT is the thinking I want to squash.

We're not getting rid of jiffies, and we're not going to be "tickless". We 
might have a variable clock, but that has _nothing_ to do with the basic 
fact that even a variable clock needs a fundamental baseline, and that the 
kernel _needs_ a heartbeat.

And that baseline and heartbeat is HZ and "jiffies". End of story. Anybody
who argues for getting rid of it just isn't thinking things through.

> I have nothing religious against jiffies per se. My argument however is
> that with a few simple, relative interfaces *in addition* to an absolute
> interface, almost all drivers suddenly are isolated from jiffies and HZ
> because they simply don't care. Because they really DON'T care about
> absolute time. At all. 

That's not even true.

A _lot_ of drivers end up caring about absolute time, because a _lot_ of 
drivers have a very simple issue like:

 - poll this port every 10ms until it returns "ready", or until we time
   out after 500ms.

And the thing is, you can do it the stupid way:

	for (i = 0; i < 50; i++) {
		if (ready())
			return 0;
		msleep(10);
	}
	.. timeout ..

or you can do it the _right_ way. The stupid way is simpler, but anybody 
who doesn't see what the problem is has some serious problems in kernel 
programming. Hint: it might not be polling for half a second, it might be 
polling for half a _minute_ for all you know.

In other words, the _right_ way to do this is literally

	unsigned long timeout = jiffies + HZ/2;
	for (;;) {
		if (ready())
			return 0;
		if (time_after(timeout, jiffies))
			break;
		msleep(10);
	}

which is unquestionably more complex, yes, but it's more complex because 
it is CORRECT!

And yes, we have had people "simplifying" drivers and screwing things like 
this up. And you don't even notice, until the machine is under heavy load, 
and then the "simplified" driver ends up being very very broken, and 
people wonder why performance sucks.

So stop saying that drivers not needing absolute timeouts. They need it
qutie often.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Russell King]

On Thu, Jul 14, 2005 at 10:21:41AM -0700, Linus Torvalds wrote:
> In other words, the _right_ way to do this is literally
> 
> 	unsigned long timeout = jiffies + HZ/2;
> 	for (;;) {
> 		if (ready())
> 			return 0;
> 		if (time_after(timeout, jiffies))
> 			break;
> 		msleep(10);
> 	}
> 
> which is unquestionably more complex, yes, but it's more complex because 
> it is CORRECT!

Umm.  Except, according to your description of what it's supposed to
do, the above code can have an accumulating error.

	unsigned long timeout, max_timeout, now;

	now = jiffies;
	timeout = now + HZ/100;
	max_timeout = now + HZ/2;

	for (;;) {
		if (ready())
			return 0;
		if (time_after(timeout, jiffies))
			break;
		sleep_until(timeout);
		timeout += HZ/100;
	}

would be even more correct, if we had an absolute schedule_timeout()
called sleep_until().  If we woke up late, we will schedule the next
wakeup exactly 10ms after the previous wakeup _should_ have happened.

Quick!  Convert all the kernel interfaces back to absolute time! 8)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Russell King wrote:
> 
> Umm.  Except, according to your description of what it's supposed to
> do, the above code can have an accumulating error.

No. It can have a local drift, but the point is, the error never gets 
worse - it _stays_ local.

There's no point in polling twice in immediate succession just because a 
sleep overslept. That's like a security guard testing each door twice for 
being locked, just because he overslept one round. Pointless.

But what matters is that you don't let your local errors accumulate into 
the big picture.

Now, if somebody wants to make nicer helper functions so that you can say

	timeout = ms_from_now(500);

or something instead of saying "timeout = jiffies + HZ/2", then hey, go 
wild. At that point it's just syntactic sugar, and maybe it's worth it.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Russell King]

On Thu, Jul 14, 2005 at 01:06:00PM -0700, Linus Torvalds wrote:
> On Thu, 14 Jul 2005, Russell King wrote:
> > Umm.  Except, according to your description of what it's supposed to
> > do, the above code can have an accumulating error.
> 
> No. It can have a local drift, but the point is, the error never gets 
> worse - it _stays_ local.
> 
> There's no point in polling twice in immediate succession just because a 
> sleep overslept. That's like a security guard testing each door twice for 
> being locked, just because he overslept one round. Pointless.

That depends what you're trying to achieve.  If you're trying to achieve
an average interval of 10ms over half a second, your solution won't get
that, whereas mine will.

> But what matters is that you don't let your local errors accumulate into 
> the big picture.

Precisely - it's all about "the big picture" and what the requirements
there are.  My requirements for "once every 10ms" may be different from
yours.

An example may be best.  You need to toggle an IO output at 50Hz for
half a second but you don't care at all about the duty cycle.  You
do care about it being as close as possible 50Hz, and not 40Hz
because for whatever reason it's taking you a hypothetical (*) 2ms
latency between timer expiry either time.

(* - yes, I know we don't like that word here... 8) but I'm trying
to make a point through exaggeration, so I'm covering my ass when
someone says "but 2ms is a bloody stupid latency!".)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Gerd Knorr]

Linus Torvalds <torvalds@osdl.org> writes:

> Now, if somebody wants to make nicer helper functions so that you can say
> 
> 	timeout = ms_from_now(500);

We already have something very simliar:
        timeout = jiffies + msecs_to_jiffies(500);

;)

  Gerd

-- 
panic("it works"); /* avoid being flooded with debug messages */

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Johannes Stezenbach]

On Thu, Jul 14, 2005 Linus Torvalds wrote:
> In other words, the _right_ way to do this is literally
> 
> 	unsigned long timeout = jiffies + HZ/2;
> 	for (;;) {
> 		if (ready())
> 			return 0;
> 		if (time_after(timeout, jiffies))
> 			break;
> 		msleep(10);
> 	}
> 
> which is unquestionably more complex, yes, but it's more complex because 
> it is CORRECT!

Since you emphasised on correctness, your code is actually buggy
for the preemptible kernel. It could get preempted after the ready() test,
but before the time_after(), for quite a whie if a high priority process
keeps the system busy. This code is better:

 	unsigned long timeout = jiffies + HZ/2;
	int err;
 	for (;;) {
 		err = time_after(timeout, jiffies);
 		if (ready())
 			return 0;
 		if (err)
 			break;
 		msleep(10);
 	}

This way the condition is always re-tested before reporting timeout.

Johannes

RE: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Al Boldi]

On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
>
> There's absolutely nothing wrong with "jiffies", and anybody who 
> thinks that
> 
> 	msleep(20);
> 
> is fundamentally better than
>
> 	timeout = jiffies + HZ/50;
>

What's wrong with structured programming?

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[john stultz]

On Thu, 2005-07-14 at 19:02 +0200, Arjan van de Ven wrote:
> On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
> > just doesn't realize that the latter is a bit more complicated exactly 
> > because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
> > jiffies for some religious reason is _stupid_.
> 
> I have nothing religious against jiffies per se. My argument however is
> that with a few simple, relative interfaces *in addition* to an absolute
> interface, almost all drivers suddenly are isolated from jiffies and HZ
> because they simply don't care. Because they really DON'T care about
> absolute time. At all. 

We'll I'd probably put it as: "they do care about absolute time, but
they do not care about ticks or timer interrupt frequency"

thanks
-john

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, john stultz wrote:
> 
> We'll I'd probably put it as: "they do care about absolute time, but
> they do not care about ticks or timer interrupt frequency"

Well, the thing is, you have to count time some sane way.

You can do it by having very expensive data structures that say "real
time", but then you have some serious confusion when it comes to things
like whether it's wallclock time (which might have shifts and other
interesting issues) or some "virtual cpu time". You also end up having a
much much more expensive interface, ie "time_after()" ends up being a much
more complicated test.

So the _sane_ way to do timeouts is to define an _arbitrary_ clock that is 
just an integer counter. None of this "nanoseconds + full seconds" crap. 
None of this stupid confusion with "real time". You select something that 
is conceptually _clearly_ somethign else, and that will never get confused 
when root sets the time backwards or anything like that.

In other words, you select the thing we call "jiffies".

Face it, it is just _superior_ to the alternatives. The alternative is to
have some "fake time" aka "struct timespec", and always have to worry
about normalization and complicated comparisons, and using more memory 
too, btw.

There is no way to avoid having some kind of counter to specify time.  
NONE. The only choice is what you base your notion of time on, and how you
represent it. Do you represent it as two separate counters and try to make
it look like "fractions of seconds", or do you represent it as a single
counter, and make it look like "ticks".

And the single counter is _simpler_. The alternatives have absolutely 
_zero_ upsides. Name _one_. 

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Christoph Lameter]

On Thu, 14 Jul 2005, Linus Torvalds wrote:

> So the _sane_ way to do timeouts is to define an _arbitrary_ clock that is 
> just an integer counter. None of this "nanoseconds + full seconds" crap. 
> None of this stupid confusion with "real time". You select something that 
> is conceptually _clearly_ somethign else, and that will never get confused 
> when root sets the time backwards or anything like that.
> 
> In other words, you select the thing we call "jiffies".

AFAIK John simply wants to change jiffies to count in nanoseconds since 
bootup and then call it "clock_monotonic". One 64 bit value no splitting 
into seconds and nanoseconds anymore. This allows arbitrary length 
intervals between timer interrupts.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Thu, Jul 14, 2005 at 01:41:44PM -0700, Christoph Lameter wrote:

> AFAIK John simply wants to change jiffies to count in nanoseconds
> since bootup and then call it "clock_monotonic".

Clocks and counter drift so calling it <prefix>seconds would be
misleading.  It would really only be good for approximate timing.

I think call it something arbitrary and work towards have a separate
mechanism for time of day (which could end up being much more
expensive to use but less frrequently needed).

> One 64 bit value no splitting into seconds and nanoseconds anymore.

Using a 64-bit value is a pain on some (many?) 32-bit CPUs.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[john stultz]

On Thu, 2005-07-14 at 13:13 -0700, Linus Torvalds wrote:
> 
> On Thu, 14 Jul 2005, john stultz wrote:
> > 
> > We'll I'd probably put it as: "they do care about absolute time, but
> > they do not care about ticks or timer interrupt frequency"
> 
> Well, the thing is, you have to count time some sane way.
> 
> You can do it by having very expensive data structures that say "real
> time", but then you have some serious confusion when it comes to things
> like whether it's wallclock time (which might have shifts and other
> interesting issues) or some "virtual cpu time". You also end up having a
> much much more expensive interface, ie "time_after()" ends up being a much
> more complicated test.

I think you might be mis-characterizing Nish's and my idea. To me it
sounds very close to what you are suggesting w/ the HZ=2k idea.

> So the _sane_ way to do timeouts is to define an _arbitrary_ clock that is 
> just an integer counter. None of this "nanoseconds + full seconds" crap. 
> None of this stupid confusion with "real time". You select something that 
> is conceptually _clearly_ somethign else, and that will never get confused 
> when root sets the time backwards or anything like that.

But confusion is the problem we have now. 

People think in time, so they try to convert it into jiffies using HZ,
but then they're surprised when things happen early since the interrupt
freq isn't quite HZ but ACTHZ. Then if you fix that, you still have the
issue that if they use clock_gettime(CLOCK_MONOTONIC) to measure your
timer request it may still be early because jiffies aren't NTP adjusted.

We're not suggesting using timespecs for timers internally, just
do_monotonic_clock() which in Nish's patch returns 64 bits of
nanoseconds since the system booted and doesn't go backwards.

> In other words, you select the thing we call "jiffies".

In Nish's patch, do_monotonic_clock() is used in almost exactly the same
way as jiffies is used. 

Imagine HZ is a billion we have perfect timer hardware (which increments
jiffies exactly each nanosecond)and replace jiffies w/
do_monotonic_clock(). 

That's effectively what you get only without having the timer interrupt
go off a billion times a second. Instead, the timer interrupt can go off
at whatever frequency and soft-timers don't have to care. All interrupt
frequency changes is worse case latency.
 

> Face it, it is just _superior_ to the alternatives. The alternative is to
> have some "fake time" aka "struct timespec", and always have to worry
> about normalization and complicated comparisons, and using more memory 
> too, btw.
>
> There is no way to avoid having some kind of counter to specify time.  
> NONE. The only choice is what you base your notion of time on, and how you
> represent it. Do you represent it as two separate counters and try to make
> it look like "fractions of seconds", or do you represent it as a single
> counter, and make it look like "ticks".

I agree we have to keep time somehow, but we already do and we already
keep nanosecond precision. So why don't we just use it instead of some
low-res interrupt counter?

> And the single counter is _simpler_. The alternatives have absolutely 
> _zero_ upsides. Name _one_. 

In my mind, consistency. 

I mean, if I understand your suggestion w/ HZ=2k. Effectively you are
just redefining jiffies to be some fixed unit (effectively 500
microseconds). Then when timer interrupt arrive at some frequency lower
then 2kHZ you have to add more then 500usecs to jiffies. Right?

This is what we already do for timekeeping. The only difference is that
timekeeping deals with the HZ/ACTHZ issue and NTP adjustments and we
keep nanosecond precision.

thanks
-john

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

On Iau, 2005-07-14 at 21:13, Linus Torvalds wrote:
> There is no way to avoid having some kind of counter to specify time.  
> NONE. The only choice is what you base your notion of time on, and how you
> represent it. Do you represent it as two separate counters and try to make
> it look like "fractions of seconds", or do you represent it as a single
> counter, and make it look like "ticks".

I suspect the problem for some of this is that people think of jiffies
as incrementing by 1. If HZ is right then jiffies can be in nS, it just
won't increment by 1. Its also why jiffies() is better on some platforms
because many machines can answer "what time is it" far more accurately
than they can set interrupts.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Alan Cox wrote:
> 
> I suspect the problem for some of this is that people think of jiffies
> as incrementing by 1. If HZ is right then jiffies can be in nS, it just
> won't increment by 1.

No, jiffies _cannot_ be in nS, because of the fact that then it doesn't 
fit in a word any more. A lot of things want timeouts in the tens of 
minutes, and a jiffy clock that tries to ne in nS just screws that up 
entirely, and forces people to use u64.

Which is much more expensive to compare on 32-bit architectures due to 
nasty atomicity issues. 

So you want to keep the "normal" timeout 32-bit. In ten years we may not 
care any more. For the forseeable future we definitely do.

> Its also why jiffies() is better on some platforms
> because many machines can answer "what time is it" far more accurately
> than they can set interrupts.

That's not what "jiffies" are about. If you want accurate time, use
something else, like gettimeofday. The timeouts are _only_ relevant on the
scale of a timer interrupt, since by definition that's what we're waiting
for.

So accuracy is a total non-issue. The only kind of accuracy we care about 
is "how often can the timer ticks happen".

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

On Gwe, 2005-07-15 at 00:19, Linus Torvalds wrote:
> That's not what "jiffies" are about. If you want accurate time, use
> something else, like gettimeofday. The timeouts are _only_ relevant on the
> scale of a timer interrupt, since by definition that's what we're waiting
> for.

Ok makes sense - thats the same reason I wanted jiffies() - the timer
interrupt resolution might be useless at the given time (eg seconds). It
does mean people using while loops testing against jiffies are generally
wrong still.

Alan

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Friesen]

Linus Torvalds wrote:

> There's absolutely nothing wrong with "jiffies", and anybody who thinks 
> that
> 
> 	msleep(20);
> 
> is fundamentally better than
> 
> 	timeout = jiffies + HZ/50;
> 
> just doesn't realize that the latter is a bit more complicated exactly 
> because the latter is a hell of a lot more POWERFUL.

But if all I really want is to sleep for 20ms, what does the additional 
power actually buy me?

Chris

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Chris Friesen wrote:
> 
> But if all I really want is to sleep for 20ms, what does the additional 
> power actually buy me?

If you _only_ want to sleep for 20ms, it doesn't buy you anything.

But the sleep is often part of a bigger picture, where the 20ms might be 
part of a bigger loop that wants to run for at most a second, after which 
it will error out.

At which point it's not just about sleeping 20ms any more. 

I'm not saying that we should get rid of msleep(). I'm saying that anybody 
who thinks the jiffies-based stuff should always be rewritten as msleep() 
simply doesn't know what the hell he is talking about. At ALL.

Jiffies are here to stay, and they are here to stay for some very very 
fundamental reasons. If you hear somebody arguing for removing jiffies, 
you should piss in their general direction, and realize that they don't 
know what they are talking about.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[john stultz]

On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
> 
> On Thu, 14 Jul 2005, Vojtech Pavlik wrote:
> >  
> > A note on the relaive timer API: There needs to be a way to say
> > "x milliseconds from the time this timer should have triggered" instead
> > of "x milliseconds from now", to avoid skew in timers that try to be
> > strictly periodic.
> 
> I disagree.
> 
> There should be an _absolute_ interface, and a driver that wants that 
> should just have calculated when in time the timeout finishes - and then 
> keep on using the absolute value.
> 
> Btw, this is exactly why the jiffy-based thing is _good_. The kernel 
> timers _are_ absolute, and you make them relative by adding "jiffies".
> 
> The fact is, the current timers are better than people give them credit 
> for, and converting them away from a jiffies-based interface (to a 
> usleep-like one) is STUPID.

Yes, I strongly agree that absolute references are required in order to
avoid the accumulating time error that can happen when folks use
relative interfaces alone. 

That said, I still think there is a benefit to moving away from jiffies
and using absolute time units. Now keep your badder in control for just
a second, let me try to explain myself :)

> There's absolutely nothing wrong with "jiffies", and anybody who thinks 
> that
> 
> 	msleep(20);
> 
> is fundamentally better than
> 
> 	timeout = jiffies + HZ/50;
> 
> just doesn't realize that the latter is a bit more complicated exactly 
> because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
> jiffies for some religious reason is _stupid_.


Agreed, but we preserve the same correctness if we do something like:
	now_ns = do_monotonic_clock()
	timeout_ns = now_ns + 20000000;

And I'd argue we get something even more powerful. First, we have
something that is more easily understandable because we're working with
units people do not have to convert. Secondly we have a fixed point in
TIME rather then ticks, which allows us to avoid all the issues caused
by lost timer interrupts and the fact that (jiffies * HZ) != (xtime +
wall_to_monotonic).

Again, I agree that that it is necessary to keep absolute units similar
to what you posted about setting HZ to 2000 and just varying the
interrupt frequency as needed. Only rather then keeping some ticks as
our absolute unit, why not nanoseconds? Then we can make changes to the
interrupt frequency without affecting our absolute references.

Nish has some code, which I hope he'll be sending out shortly that does
just this, converting the soft-timer subsystem to use absolute time
instead of ticks for expiration. I feel it both simplifies the code and
makes it easier to changing the timer interrupt frequency while the
system is running.

We'll also be talking more about it at OLS, where apparently I may have
to hide behind plastic sheeting. :)

thanks
-john

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

> just doesn't realize that the latter is a bit more complicated exactly 
> because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
> jiffies for some religious reason is _stupid_.

Getting rid of jiffies in its current form is a huge win for very
non-religious reasons. Jiffies is expensive in power management and
virtualisation because you have to maintain it.

Swap jiffies for jiffies() and the world gets a lot better. 

In actual fact you also want to fix users of

	while(time_before(foo, jiffies)) { whack(mole); }

to become

	init_timeout(&timeout);
	timeout.expires = jiffies + n
	add_timeout(&timeout);
	while(!timeout_expired(&timeout)) {}

Which is a trivial wrapper around timers as we have them now

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Alan Cox wrote:
>
> > just doesn't realize that the latter is a bit more complicated exactly 
> > because the latter is a hell of a lot more POWERFUL. Trying to get rid of 
> > jiffies for some religious reason is _stupid_.
> 
> Getting rid of jiffies in its current form is a huge win for very
> non-religious reasons. Jiffies is expensive in power management and
> virtualisation because you have to maintain it.

No, you're now confusing "interrupts" with "jiffies".

There is no conceptual 1:1 thing between those two.

It so happens that traditionally we've kept them 1:1, but there's nothing 
that says that we can't slow down the interrupt source and just increment 
jiffies by a factor of the slowdown when the interrupt _does_ happen.

But no, that does NOT mean that "jiffies" should just count nanoseconds, 
and the problem would be solved. The fact is, most users of jiffies only 
care about the low 32 bits on 32-bit architectures, and that's fine as 
long as jiffies are in the millisecond range, since it still leaves a 
useful timeout value for almost everything (and then only long-range stuff 
needs to use "u64" for their timeouts).

In other words, we want a clock that is _known_ to not be very accurate,
but that is easy to just read from a memory location, and that has some
relationship to a timer tick in the sense that it should be at least in
the order-of-magnitude range for what a timer tick can cause.

Anybody who asks for nanoseconds is confused. That just forces you to use 
a 64-bit value, where no such value is needed. Things like TCP 
retransmission timeouts would be totally _idiotic_ to be made in 
nanoseconds: it would just make the socket data structures larger, and it 
has zero relevance, since the actual timer tick doesn't have that kind of 
resolution _anyway_.

The current "jiffies" actually fits all of these problems _wonderfully_
well. Yes, it needs to be converted from "struct timeval" and friends, but
it needs to be converted exactly _because_ of the good properties it has,
namely that it fits in a word, and is _relevant_ to what a timer interrupt
ends up being.

Look at 99% of the use of jiffies: it uses _jiffies_. It doesn't use
"jiffies_64", even though that's actually what gets updated. And it does
that _exactly_ because almost _nobody_ cares to pay the price of 64 bit
issues (both structure memory usage, and atomicity costs on 32-bit
architectures).

And I claim that you _cannot_ do better.

But what you can do is to have HZ at some reasonably high value (ie in the 
kHz range), and then slow down the system clock to conserve energy, and 
increment jiffies by 16 or 32 when in "slow clock mode". And then, when 
there is a multimedia app or somethign that asks for high-precision 
timers, you speed the interrupts up again, and you increment jiffies by 1.

It's that simple. And it really _is_ simple. 

And guys, the fact is, jiffies works _today_. Making this change won't 
break anything, and won't introduce any new concepts, and won't break any 
existing drivers. In contrast, introducing _yet_ another timekeeping 
mechanism is not only going to be objectively _worse_ than jiffies from a 
technical standpoint, it's going to be a total disaster from a transition 
standpoint too. 

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Linus Torvalds wrote:
> 
> But what you can do is to have HZ at some reasonably high value (ie in the 
> kHz range), and then slow down the system clock to conserve energy, and 
> increment jiffies by 16 or 32 when in "slow clock mode".

Btw, it doesn't have to even be a slow-down due to the kernels decision.

In a VM environment, the timer interrupt might be erratic, and the timer 
interrupt might read some hardware register (TSC or something) and use 
_that_ to increment jiffies by the "proper" amount.

See? The point is that "jiffies" is useful exactly because it's very cheap 
to read portably (there are no portable high-performance alternatives) and 
because it has the right resolution to be useful in 32 bits.

That doesn't mean that the code that updates it can't be clever. We
already have code that updates it that is a lot more intelligent than 99%
of the code that reads it:  we update it in 64 bits under xtime_lock, even
though most readers use a lock-less 32-bit read and only get a partial
value - the part they care about.

And this is a wonderful property that everybody seems to be ignoring, even 
though we have absolutely tons of code that just takes all of this 
goodness for granted.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Eric St-Laurent]

On Thu, 2005-07-14 at 23:37 +0100, Alan Cox wrote:

> In actual fact you also want to fix users of
> 
> 	while(time_before(foo, jiffies)) { whack(mole); }
> 
> to become
> 
> 	init_timeout(&timeout);
> 	timeout.expires = jiffies + n
> 	add_timeout(&timeout);
> 	while(!timeout_expired(&timeout)) {}
> 
> Which is a trivial wrapper around timers as we have them now

Or something like this:

struct timeout_timer {
	unsigned long expires;
};

static inline void timeout_set(struct timeout_timer *timer,
	unsigned int msecs)
{
	timer->expires = jiffies + msecs_to_jiffies(msecs);
}

static inline int timeout_expired(struct timeout_timer *timer)
{
	return (time_after(jiffies, timer->expires));
}

It provides a nice API for relative timeouts without adding overhead.


- Eric St-Laurent

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
> I have to say, this whole thread has been pretty damn worthless in
> general in my not-so-humble opinion.
> 

This thread has really gone OT, but to revisit the original issue for a
bit, are you still unwilling to consider leaving the default HZ at 1000
for 2.6.13?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Lee Revell wrote:
>
> On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
> > I have to say, this whole thread has been pretty damn worthless in
> > general in my not-so-humble opinion.
> > 
> 
> This thread has really gone OT, but to revisit the original issue for a
> bit, are you still unwilling to consider leaving the default HZ at 1000
> for 2.6.13?

Yes. I see absolutely no point to it until I actually hear people who have 
actually tried some real load that doesn't work. Dammit, I want a real 
user who says that he can noticeable see his DVD stuttering, not some 
theory.

I'm incredibly fed up with the theoretical whining. 

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 16:25 -0700, Linus Torvalds wrote:
> Yes. I see absolutely no point to it until I actually hear people who have 
> actually tried some real load that doesn't work. Dammit, I want a real 
> user who says that he can noticeable see his DVD stuttering, not some 
> theory.
> 
> I'm incredibly fed up with the theoretical whining. 
> 

And I'm incredibly frustrated by this insistence on hard data when it's
completely obvious to anyone who knows the first thing about MIDI that
HZ=250 will fail in situations where HZ=1000 succeeds.

Do you really consider this "theoretical whining"?

http://lkml.org/lkml/2005/7/13/229

It's straight from the MIDI spec.  Your argument is pretty close to "the
MIDI spec is wrong, no one can hear the difference between 1ms and 4ms".

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Lee Revell wrote:
> 
> And I'm incredibly frustrated by this insistence on hard data when it's
> completely obvious to anyone who knows the first thing about MIDI that
> HZ=250 will fail in situations where HZ=1000 succeeds.

Ok, guys. How many people have this MIDI thing? How many of you can't be 
bothered to set the default to suit your usage?

> It's straight from the MIDI spec.  Your argument is pretty close to "the
> MIDI spec is wrong, no one can hear the difference between 1ms and 4ms".

No.

YOUR argument is "nobody else matters, only I do".

MY argument is that this is a case of give and take. 

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 16:49 -0700, Linus Torvalds wrote:
> YOUR argument is "nobody else matters, only I do".
> 
> MY argument is that this is a case of give and take. 

I wouldn't say that.  I do agree with you that HZ=1000 for everyone is
problematic, I just feel that a reasonable compromise is CONFIG_HZ with
the default left at 1000.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Lee Revell wrote:

>On Thu, 2005-07-14 at 16:49 -0700, Linus Torvalds wrote:
>  
>
>>YOUR argument is "nobody else matters, only I do".
>>
>>MY argument is that this is a case of give and take. 
>>    
>>
>
>I wouldn't say that.  I do agree with you that HZ=1000 for everyone is
>problematic, I just feel that a reasonable compromise is CONFIG_HZ with
>the default left at 1000.
>

I would just say that changing something like this now is probably not a 
great idea, while allowing a config option for 100/250/1000 and maybe 
even 2000 won't make everyone happy, but seems to allow everyone to make 
themselves happy.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 16:49 -0700, Linus Torvalds wrote:
> 
> On Thu, 14 Jul 2005, Lee Revell wrote:
> > 
> > And I'm incredibly frustrated by this insistence on hard data when it's
> > completely obvious to anyone who knows the first thing about MIDI that
> > HZ=250 will fail in situations where HZ=1000 succeeds.
> 
> Ok, guys. How many people have this MIDI thing?

>  How many of you can't be 
> bothered to set the default to suit your usage?

Very few, and even fewer, respectively.  But, we'd still like to be able
to use the same kernel image as everyone else if possible.

I guess we'll have to deal with it until a variable tick solution is
ready.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Fernando Lopez-Lezcano]

On Thu, 2005-07-14 at 16:49, Linus Torvalds wrote:
> On Thu, 14 Jul 2005, Lee Revell wrote:
> > 
> > And I'm incredibly frustrated by this insistence on hard data when it's
> > completely obvious to anyone who knows the first thing about MIDI that
> > HZ=250 will fail in situations where HZ=1000 succeeds.
> 
> Ok, guys. How many people have this MIDI thing? How many of you can't be 
> bothered to set the default to suit your usage?
> 
> > It's straight from the MIDI spec.  Your argument is pretty close to "the
> > MIDI spec is wrong, no one can hear the difference between 1ms and 4ms".
> 
> No.
> 
> YOUR argument is "nobody else matters, only I do".
> 
> MY argument is that this is a case of give and take. 

Take from "few" multimedia users, give to "many" laptop users. Where
"few" and "many" are not very well defined quantities, but obviously
"many" > "few" :-) 

As to how few is few. I don't claim to know, but users that bother to
subscribe to the Planet CCRMA[*] mailing list number 750+, so that's one
datapoint. Total users of Planet CCRMA, I have no idea. Most of them
will use MIDI, either externally through hardware interfaces or
internally through the ALSA sequencer api. 

Planet CCRMA includes custom kernels with Ingo's patches for low
latency, so I will have to configure them with HZ=1000 (or 500 or
whatever) in 2.6.13+. Oh well. 

HZ=250 is a setback anyway, as many advances had been made recently in
the stock kernel that made it more and more suitable to multimedia work
(_GREAT_ work BTW). That raised my hopes that, eventually, I would not
have to build kernels, just apps, as stock kernels would be good enough.
This will make the wait longer. 

Sigh, I'll be patient and dream about high resolution timers or other
technically elegant solutions that will not penalize multimedia apps or
laptops... 

-- Fernando

[*] http://ccrma.stanford.edu/planetccrma/software/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Fernando Lopez-Lezcano wrote:

>On Thu, 2005-07-14 at 16:49, Linus Torvalds wrote:
>  
>
>>On Thu, 14 Jul 2005, Lee Revell wrote:
>>    
>>
>>>And I'm incredibly frustrated by this insistence on hard data when it's
>>>completely obvious to anyone who knows the first thing about MIDI that
>>>HZ=250 will fail in situations where HZ=1000 succeeds.
>>>      
>>>
>>Ok, guys. How many people have this MIDI thing? How many of you can't be 
>>bothered to set the default to suit your usage?
>>
>>    
>>
>>>It's straight from the MIDI spec.  Your argument is pretty close to "the
>>>MIDI spec is wrong, no one can hear the difference between 1ms and 4ms".
>>>      
>>>
>>No.
>>
>>YOUR argument is "nobody else matters, only I do".
>>
>>MY argument is that this is a case of give and take. 
>>    
>>
>
>Take from "few" multimedia users, give to "many" laptop users. Where
>"few" and "many" are not very well defined quantities, but obviously
>"many" > "few" :-) 
>
Of course that assumes that these are not the same users, which clearly 
isn't true in all cases.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Fernando Lopez-Lezcano]

On Fri, 2005-07-15 at 05:46, Bill Davidsen wrote:
> Fernando Lopez-Lezcano wrote:
> >On Thu, 2005-07-14 at 16:49, Linus Torvalds wrote:
> >>On Thu, 14 Jul 2005, Lee Revell wrote:
> >>>And I'm incredibly frustrated by this insistence on hard data when it's
> >>>completely obvious to anyone who knows the first thing about MIDI that
> >>>HZ=250 will fail in situations where HZ=1000 succeeds.
> >>>
> >>Ok, guys. How many people have this MIDI thing? How many of you can't be 
> >>bothered to set the default to suit your usage?
> >>
> >>>It's straight from the MIDI spec.  Your argument is pretty close to "the
> >>>MIDI spec is wrong, no one can hear the difference between 1ms and 4ms".
> >>>
> >>No.
> >>
> >>YOUR argument is "nobody else matters, only I do".
> >>
> >>MY argument is that this is a case of give and take. 
> >
> >Take from "few" multimedia users, give to "many" laptop users. Where
> >"few" and "many" are not very well defined quantities, but obviously
> >"many" > "few" :-) 
> >
> Of course that assumes that these are not the same users, which clearly 
> isn't true in all cases.

Yes, indeed, the two sets do intersect, I belong to both. I can't speak
for everybody in the "multimedia" set but in my musical work (and I
suspect many others share this view) I would rather not sacrifice timing
precision for battery life. Of course there are scenarios where the
opposite can be true, but are fewer by far (IMHO). 

-- Fernando

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 16:25 -0700, Linus Torvalds wrote:
> On Thu, 14 Jul 2005, Lee Revell wrote:
> > This thread has really gone OT, but to revisit the original issue for a
> > bit, are you still unwilling to consider leaving the default HZ at 1000
> > for 2.6.13?
> 
> Yes. I see absolutely no point to it until I actually hear people who have 
> actually tried some real load that doesn't work. Dammit, I want a real 
> user who says that he can noticeable see his DVD stuttering, not some 
> theory.

Well, on the plus side, this will probably drive the development of a
mergeable variable tick solution, as I can't imagine the distros will
want to have to ship a separate HZ=1000 kernel for multimedia use.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 3403 bytes --]

On Fri, 15 Jul 2005 09:25, Linus Torvalds wrote:
> On Thu, 14 Jul 2005, Lee Revell wrote:
> > On Thu, 2005-07-14 at 09:37 -0700, Linus Torvalds wrote:
> > > I have to say, this whole thread has been pretty damn worthless in
> > > general in my not-so-humble opinion.
> >
> > This thread has really gone OT, but to revisit the original issue for a
> > bit, are you still unwilling to consider leaving the default HZ at 1000
> > for 2.6.13?
>
> Yes. I see absolutely no point to it until I actually hear people who have
> actually tried some real load that doesn't work. Dammit, I want a real
> user who says that he can noticeable see his DVD stuttering, not some
> theory.

Disclaimer - This is not proof of a real world dvd stuttering, simply a 
benchmarked result. My code may be crap, but then the real apps out there may 
also be.

Results from interbench v0.21 
(http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2)

2.6.13-rc1 on a pentium4 3.06

HZ=1000:
--- Benchmarking Audio in the presence of loads ---
	Latency +/- SD (ms)  Max Latency   % Desired CPU  % Deadlines Met
None	  0.012 +/- 0.00196    0.021		 100	        100
Video	   1.28 +/- 0.509       2.01		 100	        100
X	  0.289 +/- 0.578          2		 100	        100
Burn	  0.014 +/- 0.002      0.023		 100	        100
Write	  0.025 +/- 0.0349      0.49		 100	        100
Read	   0.02 +/- 0.00383    0.052		 100	        100
Compile	  0.023 +/- 0.00752    0.054		 100	        100
Memload	  0.222 +/- 0.892       9.04		 100	        100

--- Benchmarking Video in the presence of loads ---
	Latency +/- SD (ms)  Max Latency   % Desired CPU  % Deadlines Met
None	  0.012 +/- 0.00169    0.023		 100	        100
X	   2.55 +/- 2.37        18.7		 100	       75.8
Burn	   1.08 +/- 1.06        16.7		 100	       88.2
Write	  0.224 +/- 0.215       16.7		 100	       97.8
Read	  0.019 +/- 0.00354    0.059		 100	        100
Compile	   4.55 +/- 4.53        17.6		 100	       57.5
Memload	    1.3 +/- 1.34        51.5		 100	         88


HZ=250:
--- Benchmarking Audio in the presence of loads ---
	Latency +/- SD (ms)  Max Latency   % Desired CPU  % Deadlines Met
None	  0.011 +/- 0.00152    0.022		 100	        100
Video	  0.157 +/- 0.398       3.62		 100	        100
X	    1.3 +/- 1.82        4.01		 100	        100
Burn	  0.014 +/- 0.00142    0.026		 100	        100
Write	  0.022 +/- 0.0125     0.092		 100	        100
Read	  0.021 +/- 0.00366    0.048		 100	        100
Compile	   0.03 +/- 0.0469     0.559		 100	        100
Memload	  0.144 +/- 0.681       8.05		 100	        100

--- Benchmarking Video in the presence of loads ---
	Latency +/- SD (ms)  Max Latency   % Desired CPU  % Deadlines Met
None	      5 +/- 4.99        16.7		 100	         54
X	   9.98 +/- 8.94        20.7		 100	       31.2
Burn	   16.6 +/- 16.6        16.7		 100	      0.167
Write	   4.11 +/- 4.08        16.7		 100	       60.8
Read	   2.55 +/- 2.53        16.7		 100	       73.8
Compile	   15.6 +/- 15.6        17.7		 100	        3.5
Memload	   2.91 +/- 2.92        45.4		 100	       72.5


Audio did show slightly larger max latencies but nothing that would be of 
significance.

On video, maximum latencies are only slightly larger at HZ 250, all the 
desired cpu was achieved, but the average latency and number of missed 
deadlines was significantly higher.

Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Fri, 2005-07-15 at 14:08 +1000, Con Kolivas wrote:
> Audio did show slightly larger max latencies but nothing that would be of 
> significance.
> 
> On video, maximum latencies are only slightly larger at HZ 250, all the 
> desired cpu was achieved, but the average latency and number of missed 
> deadlines was significantly higher.

Because audio timing is driven by the soundcard interrupt while video,
like MIDI, relies heavily on timers.

Lee 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Zwane Mwaikambo]

On Fri, 15 Jul 2005, Lee Revell wrote:

> On Fri, 2005-07-15 at 14:08 +1000, Con Kolivas wrote:
> > Audio did show slightly larger max latencies but nothing that would be of 
> > significance.
> > 
> > On video, maximum latencies are only slightly larger at HZ 250, all the 
> > desired cpu was achieved, but the average latency and number of missed 
> > deadlines was significantly higher.
> 
> Because audio timing is driven by the soundcard interrupt while video,
> like MIDI, relies heavily on timers.

In interbench it's not driven by a soundcard interrupt.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 22:54 -0600, Zwane Mwaikambo wrote:
> On Fri, 15 Jul 2005, Lee Revell wrote:
> 
> > On Fri, 2005-07-15 at 14:08 +1000, Con Kolivas wrote:
> > > Audio did show slightly larger max latencies but nothing that would be of 
> > > significance.
> > > 
> > > On video, maximum latencies are only slightly larger at HZ 250, all the 
> > > desired cpu was achieved, but the average latency and number of missed 
> > > deadlines was significantly higher.
> > 
> > Because audio timing is driven by the soundcard interrupt while video,
> > like MIDI, relies heavily on timers.
> 
> In interbench it's not driven by a soundcard interrupt.
> 
> 

OK.  Con, any idea why video is so much more affected than audio?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[kernel]

Quoting Lee Revell <rlrevell@joe-job.com>:

> On Thu, 2005-07-14 at 22:54 -0600, Zwane Mwaikambo wrote:
> > On Fri, 15 Jul 2005, Lee Revell wrote:
> > 
> > > On Fri, 2005-07-15 at 14:08 +1000, Con Kolivas wrote:
> > > > Audio did show slightly larger max latencies but nothing that would be
> of 
> > > > significance.
> > > > 
> > > > On video, maximum latencies are only slightly larger at HZ 250, all the
> 
> > > > desired cpu was achieved, but the average latency and number of missed
> 
> > > > deadlines was significantly higher.
> > > 
> > > Because audio timing is driven by the soundcard interrupt while video,
> > > like MIDI, relies heavily on timers.
> > 
> > In interbench it's not driven by a soundcard interrupt.
> > 
> > 
> 
> OK.  Con, any idea why video is so much more affected than audio?

In the emulation, video vs audio is 40% cpu vs 5% cpu, 16.7ms frames instead of
50ms frames. When your cpu requirements are higher and your frames are shorter
the likelihood of dropping a frame, especially under load, will skyrocket as
your timing granularity decreases. Clearly 250HZ is not as good as 1000HZ for
this, and I assume your midi example.

Cheers,
Con

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andrew Morton]

Vojtech Pavlik <vojtech@suse.cz> wrote:
>
> A note on the relaive timer API: There needs to be a way to say
>  "x milliseconds from the time this timer should have triggered" instead
>  of "x milliseconds from now", to avoid skew in timers that try to be
>  strictly periodic.

	mod_timer(timer, timer->expires + N) ?

(add_timer() will treat a negative `expires' as "right now", so we'll dtrt
if the time now has overrun (timer->expires + N))

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Ingo Molnar]

* Linus Torvalds <torvalds@osdl.org> wrote:

> I suspect that it is impractical to reprogram the PIT on a very fine 
> granularity.

yes - reprogramming the PIT can take up to 10 usecs even on recent PCs.  
(in fact the cost is pretty much system-independent due to PIO.) On 
modern, PIT-less timesources (e.g. HPET) it can be faster.

> Btw, if somebody really gets excited about all this, let me say (once 
> again) what I think might be an acceptable situation.
> 
> First off, I'm _not_ a believer in "sub-HZ ticks". Quite the reverse. 
> I think we should have HZ be some high value, but we would _slow_down_ 
> the tick when not needed, and count by 2's, 3's or even 10's when 
> there's not a lot going on.

i think that would be an acceptable solution for high-precision timers, 
as long as two other problems are solved too:

 - there are real-time applications (robotic environments: fast rotating
   tools, media and mobile/phone applications, etc.) that want 10
   usecs precision. If such users increased HZ to 100,000 or even
   1000,000, the current timer implementation would start to creek: e.g.
   jiffies on 32-bit systems would wrap around in 11 hours or 1.1 hours.
   (To solve this cleanly, pretty much the only solution seems to be to
   increase the timeout to a 64 bit value. A non-issue for 64-bit
   systems, that's why i think we could eventually look at this 
   possibility, once all the other problems are hashed out.)

 - at very high HZ values the clustering of e.g. network timers is lost, 
   creating an artificially high number of timer interrupts. So likely 
   we'd still need some way to 'blur' timeouts and to round e.g. network 
   timers to the next 1 msec or 500 usecs boundary, to cluster up timers 
   for bulk processing. But in any case, such a solution does not sound 
   nearly as messy as the sub-jiffies method.

if the 'high precision' uses are not addressed [*] i fear the whole HRT 
game starts again: embedded folks trying to standardize on Linux for 
everything [**] will want HRT timers and will do addons and sub-jiffy 
approaches [***], and will push for inclusion. I think we could as well 
solve this whole problem area by making ridiculously high HZ values 
practical too!

	Ingo

[*]  there's also a third problem: timer prioritization. It's not 
     necessarily a problem the upstream kernel should care about, but 
     it's a problem for things that try to offer hard-real-time, like 
     PREEMPT_RT: HRT timers need to be prioritizable. If e.g. the system 
     is soaked handling network timers, it should still be possible for 
     that single mega-important HRT timer to run and wake up the 
     mega-high-priority RT task that will preempt all network activity 
     within 10 usecs worst-case. The sub-jiffies approach does this 
     prioritization in a natural way, because there HRT timers are 
     separate, so the prioritization of them is easy. With the grand 
     unified 'big HZ' scheme the HRT folks would have to implement a 
     mechanism to split off highprio timers from the stream of normal 
     timers. ]

[**] having high precision is also a perception and uniformity of 
     platform issue: most embedded developers will find 500 usecs 
     precision good enough for most uses, but it does not 'sound' good 
     enough, and there's no easy way out either. So _if_ there's the 
     occasional need for higher precision they'll have no easy solution 
     for Linux, and this prevents them from standardizing on Linux for 
    _everything_.


[***]
   a side-thought about sub-jiffies: the biggest conceptual problem 
   the sub-jiffy method has is the sorting needed when timers move from 
   the jiffy bucket into the HRT list which doesnt scale - but this is 
   really a HRT-timers-internal problem.  It could be further improved 
   by e.g.  dividing the last jiffy up into say 10 usec buckets - with a 
   1msec jiffy clock that's 100 buckets, and having a bitmap to see 
   which bucket is active. Then the 'get the next timer' act becomes a 
   matter of searching the bitmap for the next bit set - at pretty much 
   constant overhead. Even a 1 usec precision would mean only 1000 
   buckets for the last jiffy, with 1000 bits (128 bytes) to search, 
   still quite ok. But, in any case, it would be nice to avoid the 
   "conceptual dualness" of the HRT patch.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 10:38 +0200, Ingo Molnar wrote:
>  - there are real-time applications (robotic environments: fast rotating
>    tools, media and mobile/phone applications, etc.) that want 10
>    usecs precision. If such users increased HZ to 100,000 or even
>    1000,000, the current timer implementation would start to creek: e.g.
>    jiffies on 32-bit systems would wrap around in 11 hours or 1.1 hours.
>    (To solve this cleanly, pretty much the only solution seems to be to
>    increase the timeout to a 64 bit value. A non-issue for 64-bit
>    systems, that's why i think we could eventually look at this 
>    possibility, once all the other problems are hashed out.)
> 

Those types of systems will not be 64 bit for many, many years, if
ever...

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Christoph Lameter]

On Thu, 14 Jul 2005, Lee Revell wrote:

> On Thu, 2005-07-14 at 10:38 +0200, Ingo Molnar wrote:
> >  - there are real-time applications (robotic environments: fast rotating
> >    tools, media and mobile/phone applications, etc.) that want 10
> >    usecs precision. If such users increased HZ to 100,000 or even
> >    1000,000, the current timer implementation would start to creek: e.g.
> >    jiffies on 32-bit systems would wrap around in 11 hours or 1.1 hours.
> >    (To solve this cleanly, pretty much the only solution seems to be to
> >    increase the timeout to a 64 bit value. A non-issue for 64-bit
> >    systems, that's why i think we could eventually look at this 
> >    possibility, once all the other problems are hashed out.)
> > 
> 
> Those types of systems will not be 64 bit for many, many years, if
> ever...

Linux can already provide a response time within < 3 usecs from user space 
using f.e. the Altix RTC driver which can generate an interrupt that then 
sends a signal to an application. The Altix RTC clock is supported via POSIX
timer syscalls and can be accessed using CLOCK_SGI_CYCLE. This has been 
available in Linux since last fall and events can be specified with 50 
nanoseconds accurary.

Other clock sources like  HPET could do the same if someone would be 
willing to provide the hookup to the posix layer.

I doubt that increasing the timer frequency is the way to go to solve 
these issues. HZ should be as low as possible and we should strive for a 
tickless system.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Thu, 2005-07-14 at 08:02 -0700, Christoph Lameter wrote:
> I doubt that increasing the timer frequency is the way to go to solve 
> these issues. HZ should be as low as possible and we should strive for
> a tickless system.

Agreed.  Most of those applications are driven by their own interrupt
source anyway.

I do think Linus' proposal, or even copying what Windows does, would be
a big improvement over the fixed tick rate.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Wed, 2005-07-13 at 18:13 -0700, dean gaudet wrote:
> http://www.microsoft.com/whdc/system/CEC/mm-timer.mspx

Did anyone else find this strange:

"The RTC is used in periodic mode to provide the system profiling
interrupt on uni-processor systems and the clock interrupt on
multi-processor systems."

We just take NR_CPUS * HZ timer interrupts per second, what's the
advantage of using the RTC?

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Wed, 13 Jul 2005, Lee Revell wrote:

> Did anyone else find this strange:
> 
> "The RTC is used in periodic mode to provide the system profiling
> interrupt on uni-processor systems and the clock interrupt on
> multi-processor systems."
> 
> We just take NR_CPUS * HZ timer interrupts per second, what's the
> advantage of using the RTC?

 It tends to work in the APIC mode all the time (with all systems), unlike 
the PIT which has "interesting" routing problems with its IRQ0, which 
you've probably already noticed.  Have a look at all the hassle in 
check_timer() if you want to double-check it.

 Of course using APIC internal timers is generally the best idea on SMP, 
but they may have had reasons to avoid them (it's not an ISA interrupt, so 
it could have been simply out of question in the initial design).

  Maciej

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/13/05, Chris Wedgwood <cw@f00f.org> wrote:
> On Wed, Jul 13, 2005 at 01:48:57PM -0700, Andrew Morton wrote:
> 
> > Len Brown, a year ago: "The bottom line number to laptop users is
> > battery lifetime.  Just today somebody complained to me that Windows
> > gets twice the battery life that Linux does."
> 
> It seems the motivation for lower HZ is really:
> 
>    (1) ACPI/SMM suckage in laptops
> 
>    (2) NUMA systems with *horrible* remote memory latencies
> 
> Both can be detected from you .config and we could see HZ as needed
> there and everyone else could avoid this surely?
> 

While reading this thread it occoured to me that perhaps what we
really want (besides sub HZ timers) might be for the kernel to
auto-tune HZ?

Would it make sense to introduce a new config option (say
CONFIG_HZ_AUTO) that when selected does something like this at boot:

if (running_on_a_laptop()) {
    set_HZ_to(250);
} else if (running_on_large_NUMA_box()) {
    set_HZ_to_100();
} else if (running_on_multimedia_box() {
    set_HZ_to_1000();
} else {
    set_HZ_to_some_other_sane_default();
}

and if user wants to not use the auto detection they can select a
certain HZ in their .config instead of CONFIG_HZ_AUTO.


Just wanted to throw the idea up in the air in case it made sense.
Feel free to pick it apart or simply ignore it. :-)


-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Fri, 2005-07-15 at 02:04 +0200, Jesper Juhl wrote:
> While reading this thread it occoured to me that perhaps what we
> really want (besides sub HZ timers) might be for the kernel to
> auto-tune HZ?
> 
> Would it make sense to introduce a new config option (say
> CONFIG_HZ_AUTO) that when selected does something like this at boot:
> 
> if (running_on_a_laptop()) {
>     set_HZ_to(250);
> }

I don't think this will fly because we take a big performance hit by
calculating HZ at runtime.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/15/05, Lee Revell <rlrevell@joe-job.com> wrote:
> On Fri, 2005-07-15 at 02:04 +0200, Jesper Juhl wrote:
> > While reading this thread it occoured to me that perhaps what we
> > really want (besides sub HZ timers) might be for the kernel to
> > auto-tune HZ?
> >
> > Would it make sense to introduce a new config option (say
> > CONFIG_HZ_AUTO) that when selected does something like this at boot:
> >
> > if (running_on_a_laptop()) {
> >     set_HZ_to(250);
> > }
> 
> I don't think this will fly because we take a big performance hit by
> calculating HZ at runtime.
> 
Even if we only have to do it once at boot?  The thought was to detect
what type of machine we are booting on, figure out what a good HZ
would be for that type of box, then set that HZ value and treat it as
a constant from that point forward.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Fri, 15 Jul 2005, Jesper Juhl wrote:
>
> Even if we only have to do it once at boot?  The thought was to detect
> what type of machine we are booting on, figure out what a good HZ
> would be for that type of box, then set that HZ value and treat it as
> a constant from that point forward.

No, it really should be a compile-time constant, or a lot of things get a 
lot more expensive. There's a HZ embedded in a lot of places, and some of 
them are divides, for example. Others do optimized special cases based on 
static knowledge of what HZ is.

So this is why I so strongly argue that we should have a constant HZ, but 
a dynamic _increment_ of "jiffies". Nobody (obviously) depends on jiffies 
being constant, so it's ok to increment jiffies by pretty much any value.

Yeah, yeah, there might be some _very_ few code-paths (bogomips, I think)  
that may look at when "jiffies" changes, and actually measure one tick 
that way. They would need to be taught that they don't measure "one" tick 
any more, they measure "jiffies_increment" ticks or something.

But I really wouldn't be surprised if the bogomips calibration loop was 
really the only thing that needed some small tweaking for increments of 
other than one.

(Oh, we'll find other things we want to fix up, and such a change would
result in other changes down the line, no question about that.  But I
don't think it would be very much at all, and I don't think it would 
turn out at all traumatic).

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Russell King]

On Thu, Jul 14, 2005 at 05:42:15PM -0700, Linus Torvalds wrote:
> So this is why I so strongly argue that we should have a constant HZ, but 
> a dynamic _increment_ of "jiffies". Nobody (obviously) depends on jiffies 
> being constant, so it's ok to increment jiffies by pretty much any value.

I agree.  Isn't this exactly what HZ=1000 with VST achieves?  We
know this works already...

> But I really wouldn't be surprised if the bogomips calibration loop was 
> really the only thing that needed some small tweaking for increments of 
> other than one.

Having run VST on ARM, VST must be disabled while the bogomips
calibrations have completed - I suspect VST requires some sort of
enable/disable counted system like we do for interrupts and the hlt
thing, so that the hotplug CPU code can do it's bogomips calibration
appropriately.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Thu, 14 Jul 2005, Lee Revell wrote:
> 
> I don't think this will fly because we take a big performance hit by
> calculating HZ at runtime.

I think it might be an acceptable solution for a distribution that really 
needed it, since it should be fairly simple. However, it's definitely not 
the right solution.

HOWEVER. I bet that somebody who really really cares (hint hint) could
easily make HZ be 1000, and then dynamically tweak the divisor at bootup
to be either 1000, 250, or 100, and then increment "jiffies" by 1, 4 or
10.

My wild guess is that this is 20 lines of code, plus another 20 for 
"setup", so that you can choose between 100/250/1000 Hz with a kernel 
command line.

It wouldn't be "dynamic" at first - you'd just set it up at bootup, and 
set a "jiffies_increment" variable, and change the

	jiffies_64++;

into

	jiffies_64 += jiffies_increment;

and you'd be done. 

Really. I dare you guys. First one to send me a tested patch gets a gold 
star.

Then, a year from now, people will realize how _easy_ it is to change the
jiffies_increment on the fly, and add a /sys/kernel/timer_frequency file, 
and then you can switch it around at run-time.

Trust me. When I say that the right thing to do is to just have a fixed 
(but high) HZ value, and just changing the timer rate, I'm -right-.

I'm always right. This time I'm just even more right than usual.

			Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/15/05, Linus Torvalds <torvalds@osdl.org> wrote:
> 
> 
> On Thu, 14 Jul 2005, Lee Revell wrote:
> >
> > I don't think this will fly because we take a big performance hit by
> > calculating HZ at runtime.
> 
> I think it might be an acceptable solution for a distribution that really
> needed it, since it should be fairly simple. However, it's definitely not
> the right solution.
> 
> HOWEVER. I bet that somebody who really really cares (hint hint) could
> easily make HZ be 1000, and then dynamically tweak the divisor at bootup
> to be either 1000, 250, or 100, and then increment "jiffies" by 1, 4 or
> 10.
> 
[...]
> 
> Really. I dare you guys. First one to send me a tested patch gets a gold
> star.
> 

Testing a patch right now, I'll send it to you as soon as it doesn't
blow up on boot (which it currently does).

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Eric St-Laurent]

On Thu, 2005-07-14 at 17:24 -0700, Linus Torvalds wrote:
> 
> On Thu, 14 Jul 2005, Lee Revell wrote:
> 
> Trust me. When I say that the right thing to do is to just have a fixed 
> (but high) HZ value, and just changing the timer rate, I'm -right-.
> 
> I'm always right. This time I'm just even more right than usual.

Of course you are, jiffies are simple and efficient.

But it may be worthwhile to provide better/simpler API for relative
timeouts and also better hide the implementation details of the tick
system.


If i sum-up the discussion from my POV:

- use a 32-bit tick counter on 32-bit platforms and use a 64-bit counter
on 64-bit platforms

- keep the constant HZ=1000 (mS resolution) on 32-bit platforms

- remove the assumption that timer interrupts and jiffies are 1:1 thing
(jiffies may be incremented by >1 ticks at timer interrupt)

- determine jiffies_increment at boot

- have a slow clock mode to help power management (adjust
jiffies_increment by the slowdown factor)

- it may be useful to bump up HZ to 1e6 (uS res.) or 1e9 (nS res.) on
64-bit platforms, if there are benefits such as better accuracy during
time units conversions or if a higher frequency timer hardware is
available/viable.

- it may be also useful to bump HZ on -RT (Real-time) kernels, or with
-HRT (High-resolution timers support). Users of those kernel are willing
to pay the cost of the overhead to have better resolution

- avoid direct usage of the jiffies variable, instead use jiffies()
(inline or MACRO), IMO monotonic_clock() would be a better name

- provide a relative timeout API (see my previous post, or Alan's
suggestions)

- remove most of the direct use of jiffies through the code and replace
them with msleep(), relative timer, etc

- use human units for those APIs


- Eric St-Laurent

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Stephen Pollei]

On 7/14/05, Eric St-Laurent <ericstl34@sympatico.ca> wrote:
> On Thu, 2005-07-14 at 17:24 -0700, Linus Torvalds wrote:
> > Trust me. When I say that the right thing to do is to just have a fixed
> > (but high) HZ value, and just changing the timer rate, I'm -right-.

> Of course you are, jiffies are simple and efficient.

> If i sum-up the discussion from my POV:

> - use a 32-bit tick counter on 32-bit platforms and use a 64-bit counter
> on 64-bit platforms
If the 64bit counter doesn't have any overhead then sure.

> - keep the constant HZ=1000 (mS resolution) on 32-bit platforms
Which HZ Is that? CONFIG_JIFFIES_HZ or CONFIG_FIXED_PIT_HZ ?
I think you meant CONFIG_JIFFIES_HZ which I think for even 32bit
counters could go up to 1e4 to 5e4 , with some patching going on in
some places of course.

> - remove the assumption that timer interrupts and jiffies are 1:1 thing
> (jiffies may be incremented by >1 ticks at timer interrupt)
Yes maybe nuke CONFIG_HZ and replace it with CONFIG_JIFFIES_HZ and
CONFIG_(FIXED|DEFAULT|DYNAMIC)_PIT_HZ . Starting with just
CONFIG_FIXED_PIT_HZ, add others as needed.

Extreme might be to also just nuke HZ and replace it with JHZ and PHZ,
or whatever so that people are *crystal* clear about the difference.

> - determine jiffies_increment at boot
So CONFIG_<foo>_PIT_HZ could be a per boot time thing maybe.
So you'd have CONFIG_DEFAULT_PIT_HZ if it was a per per boot or runtime thing.
CONFIG_DYNAMIC_PIT_HZ if it was changable as the system is running --
like windows.
CONFIG_FIXED_PIT_HZ if it is a compile time constant.
Or something like the that?

> - have a slow clock mode to help power management (adjust
> jiffies_increment by the slowdown factor)
CONFIG_DYNAMIC_PIT_HZ unless it's overhead is so low that everyone
just wants it by default.

> - it may be useful to bump up HZ to 1e6 (uS res.) or 1e9 (nS res.) on
> 64-bit platforms, if there are benefits such as better accuracy during
> time units conversions or if a higher frequency timer hardware is
> available/viable.
Too high starts to cause other troubles. I think that the real time
people want 10uS scheduling, but even the ipipe and rt-preempt has
18us-70uS delays at times IIRC. So 5e4 to 1e5 is about the extreme end
of the road for CONFIG_JIFFIES_HZ . I think even long term that 1e5 to
1e6 would be extreme because of speed of light issues, etc. Hpet is
only 1.4e7 IIRC.

I think that you should start with:
1) CONFIG_FIXED_PIT_HZ=50 CONFIG_JIFFIES_HZ=2000
2) try it out and fix any bugs, send the fixes to Linus to see if how
much he bitches.
3) if you still need CONFIG_JIFFIES_HZ to be larger, double it and then goto 2.
4) enjoy your higher frequency jiffies

I bet that even that going to somewhere between 2e3 through 1e5 will
make you want to change a few things for performance and sanity
reasons. So I'd focus on that before I even thought about 1e6 through
1e10 . Plus I think the interest level really fails off to go that
extreme.

Just making JIFFIES_HZ != PIT_HZ will require patches.
Dynamic pit hz or lazy update of jiffies based on tsc/hpet/other are
other patches.

> - it may be also useful to bump HZ on -RT (Real-time) kernels,
yes they sound like they want JIFFIES_HZ to be 1e3 through 1e5
depending on task. They also want hpet(or other), vertical retrace
interrupts(so xsync works for video), perhaps a nist mini atomic
clock, and a few other goodies AFAIK.
> -HRT (High-resolution timers support).
Yes tsc or hpet or whatever users might benefit in several ways.
1) both tsc and hpet might be able to bump up to a more accurate value
on entry to idle and then test to see if anything got scheduled.
2) hpet can set set one shot timers for the next up coming event on
idle if it's sooner than when the PIT interrupt is suppose to come in.
Of course update the jiffies when that hpet interrupt comes.

>Users of those kernel are willing
> to pay the cost of the overhead to have better resolution
Yes realtime users with something like hpet might not vary the pit
timer, but place hooks to update the jiffies between pit interrupts
like idle, scheduler(task switch), etc. And use the hpet one shot
interrupts as well.

> - avoid direct usage of the jiffies variable, instead use jiffies()
> (inline or MACRO), IMO monotonic_clock() would be a better name
I don't know I think it could remain a variable you usual just want it
to be a light-weight memory read not a call out to an hpet and then a
math conversion, or a call out to tsc that then has to known about if
the tsc represents work or time, and if the cpu has been slowed for
power save reasons etc etc etc. I think you want a symbol exported gpl
of something like void force_update_jiffies(void); that you can call
in different hook locations to force the update of jiffies from
non-interupt sources. Actually you might want more than one version of
that function or have it take an argument, becuase some people might
want to be super lazy and only update it when the enter or leave idle,
while others(real timers) might want it updated on every interupt and
scheduling decision point. Hook placement might have strong affinity
with where the preemption hooks are placed for the various levels of
preemption(voluntary,standard,rt).


But If I understand Linus's points he wants jiffies to remain a memory
fetch, and make sure it doesn't turn into a singing dancing christmas
tree.

Sounds like it might be a a bunch of config options or some of it
might be low enough overhead that the non-lazy approach might work for
everyone.

P.S.
http://www.nist.gov/public_affairs/releases/miniclock.htm
[[The clock's inner workings are about the size of a grain of rice
(1.5 millimeters on a side and 4 millimeters high), consume less than
75 thousandths of a watt (enabling the clock to be operated on
batteries) and are stable to one part in 10 billion, equivalent to
gaining or losing just one second every 300 years. ...  measuring time
by the natural vibrations of cesium atoms, at 9.2 billion "ticks" per
second ... ]]
This thing measures time on the order of 1e10 .

http://keithp.com/~keithp/talks/xarch_ols2004/xarch-ols2004-html/
mentions xsync.

-- 
http://dmoz.org/profiles/pollei.html
http://sourceforge.net/users/stephen_pollei/
http://www.orkut.com/Profile.aspx?uid=2455954990164098214
http://stephen_pollei.home.comcast.net/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Eric St-Laurent]

On Fri, 2005-07-15 at 12:58 -0700, Stephen Pollei wrote:
> But If I understand Linus's points he wants jiffies to remain a memory
> fetch, and make sure it doesn't turn into a singing dancing christmas
> tree.

It seems it relatively easy to support dynamic tick, the ARM
architecture has it. But with the numerous users of jiffies through the
code, it seems to me that it's hard to ensure that everyone of them will
continue to work correctly if the jiffies_increment is changed during
runtime.

As Linus noted, the current tick code is flexible and powerful, but it
can be hard to get it right in all case. 

WinCE developers have similar problems/concerns:

http://blogs.msdn.com/ce_base/archive/2005/06/08/426762.aspx

With the previous cleanup like time_after()/time_before(), msleep() and
friends, unit conversion helpers, etc. it's a step in the right
direction.

I just wanted to point out that while it's good to preserve the current
efficient tick implementation, it may be worthwhile to add a relative
timeout API like Alan Cox proposed a year ago to better hide the
implementation details.


- Eric St-Laurent

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Linus Torvalds wrote:

>On Thu, 14 Jul 2005, Lee Revell wrote:
>  
>
>>I don't think this will fly because we take a big performance hit by
>>calculating HZ at runtime.
>>    
>>
>
>I think it might be an acceptable solution for a distribution that really 
>needed it, since it should be fairly simple. However, it's definitely not 
>the right solution.
>
>HOWEVER. I bet that somebody who really really cares (hint hint) could
>easily make HZ be 1000, and then dynamically tweak the divisor at bootup
>to be either 1000, 250, or 100, and then increment "jiffies" by 1, 4 or
>10.
>
>My wild guess is that this is 20 lines of code, plus another 20 for 
>"setup", so that you can choose between 100/250/1000 Hz with a kernel 
>command line.
>
>It wouldn't be "dynamic" at first - you'd just set it up at bootup, and 
>set a "jiffies_increment" variable, and change the
>
>	jiffies_64++;
>
>into
>
>	jiffies_64 += jiffies_increment;
>
>and you'd be done. 
>
>Really. I dare you guys. First one to send me a tested patch gets a gold 
>star.
>
>Then, a year from now, people will realize how _easy_ it is to change the
>jiffies_increment on the fly, and add a /sys/kernel/timer_frequency file, 
>and then you can switch it around at run-time.
>
>Trust me. When I say that the right thing to do is to just have a fixed 
>(but high) HZ value, and just changing the timer rate, I'm -right-.
>
>I'm always right. This time I'm just even more right than usual.
>

And humble, too ;-)

Do you actually have something against tickless, or just don't think it 
can be done in reasonable time? I can see this needing very careful 
thought on SMP.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Matthias Urlichs]

Hi, Bill Davidsen wrote:

> Do you actually have something against tickless, or just don't think it 
> can be done in reasonable time?

You can do it in small steps.

When you have that jiffies_increment variable, you can add code to
dynamically adjust it at runtime -- just reprogram the system timer
(which may not be cheap).

After you've done *that*, you can teach the add_timer code to optionally
adjust jiffies_increment when demand changes; add an estimate on timer
tick cost vs. reprogramming cost (which could return "always" when you're
running UML); you might want to take user prefs into account ("always
reprogram if the timeout would arrive more than 10 msec late, because
otherwise my Doom3 game lags too much").

There you are. Tickless, and nobody even notices.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Caesar had his Brutus -- charles the First, his Cromwell -- and George the
Third ("Treason!" cried the Speaker) -- may profit by their example. If this
be treason, make the most of it.
					-- Patrick Henry

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

[-- Attachment #1: Type: text/plain, Size: 6110 bytes --]

Linus Torvalds wrote:
> 
> On Thu, 14 Jul 2005, Lee Revell wrote:
> 
>>I don't think this will fly because we take a big performance hit by
>>calculating HZ at runtime.
> 
> 
> I think it might be an acceptable solution for a distribution that really 
> needed it, since it should be fairly simple. However, it's definitely not 
> the right solution.
> 
> HOWEVER. I bet that somebody who really really cares (hint hint) could
> easily make HZ be 1000, and then dynamically tweak the divisor at bootup
> to be either 1000, 250, or 100, and then increment "jiffies" by 1, 4 or
> 10.
> 
> My wild guess is that this is 20 lines of code, plus another 20 for 
> "setup", so that you can choose between 100/250/1000 Hz with a kernel 
> command line.
> 
> It wouldn't be "dynamic" at first - you'd just set it up at bootup, and 
> set a "jiffies_increment" variable, and change the
> 
> 	jiffies_64++;
> 
> into
> 
> 	jiffies_64 += jiffies_increment;
> 
> and you'd be done. 
> 
> Really. I dare you guys. First one to send me a tested patch gets a gold 
> star.
> 

I don't know if this will earn me that gold star or not, but it's what I 
have at this point.
It's buggy, that I know. setting kernel_hz (the new boot parameter) to 
250 causes my system clock to run at something like 4-5 times normal 
speed, and key repeat is all funny (super fast) and various other nasty 
things - however, it does build and it does boot (i386 only as that's 
all I have) and apart from the kernels notion of time being way off it 
does seem to be a step in the right direction.
Me never having looked at the kernels timer code before is most likely 
the explanation for the bugs - that and the fact that I didn't try to 
tackle the bogomips calculation at all...  Ohh, and it'll probably bee 
even more strange if you build the kernel with CONFIG_HZ set to anything 
other than 1000 - if this is what we want to do, then I guess CONFIG_HZ 
needs to go away completely and just always be 1000 and then people 
should use the boot option to modify if needed/wanted.

Anyway, is this somewhere along the lines of what you were thinking?

The patch is below, but I've also attached it since I suspect 
thunderbird will mangle it.


diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/time.c 
linux-2.6.13-rc3/arch/i386/kernel/time.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/time.c	2005-07-14 
20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/time.c	2005-07-15 
04:02:50.000000000 +0200
@@ -75,6 +75,7 @@ int pit_latch_buggy;              /* ext
  #include "do_timer.h"

  u64 jiffies_64 = INITIAL_JIFFIES;
+u16 jiffies_increment = 1;

  EXPORT_SYMBOL(jiffies_64);

@@ -481,3 +482,27 @@ void __init time_init(void)

  	time_init_hook();
  }
+
+static int __init jiffies_increment_setup(char *str)
+{
+	printk(KERN_NOTICE "setting up jiffies_increment : ");
+	if (str) {
+		printk("kernel_hz = %s, ", str);
+	} else {
+		printk("kernel_hz is unset, ");
+	}
+	if (!strncmp("100", str, 3)) {
+		jiffies_increment = 10;
+		printk("jiffies_increment set to 10, effective HZ will be 100\n");
+	} else if (!strncmp("250", str, 3)) {
+		jiffies_increment = 4;
+		printk("jiffies_increment set to 4, effective HZ will be 250\n");
+	} else {
+		jiffies_increment = 1;
+		printk("jiffies_increment set to 1, effective HZ will be 1000\n");
+	}
+
+	return 1;
+}
+
+__setup("kernel_hz=", jiffies_increment_setup);
diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_pm.c 
linux-2.6.13-rc3/arch/i386/kernel/timers/timer_pm.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_pm.c	2005-07-14 
20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/timers/timer_pm.c	2005-07-15 
02:59:39.000000000 +0200
@@ -176,7 +176,7 @@ static void mark_offset_pmtmr(void)

  	/* compensate for lost ticks */
  	if (lost >= 2)
-		jiffies_64 += lost - 1;
+		jiffies_64 += (lost * jiffies_increment) - 1;

  	/* don't calculate delay for first run,
  	   or if we've got less then a tick */
diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_tsc.c 
linux-2.6.13-rc3/arch/i386/kernel/timers/timer_tsc.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_tsc.c	2005-07-14 
20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/timers/timer_tsc.c	2005-07-15 
02:59:13.000000000 +0200
@@ -193,7 +193,7 @@ static void mark_offset_tsc_hpet(void)
  	offset = hpet_readl(HPET_T0_CMP) - hpet_tick;
  	if (unlikely(((offset - hpet_last) > hpet_tick) && (hpet_last != 0))) {
  		int lost_ticks = (offset - hpet_last) / hpet_tick;
-		jiffies_64 += lost_ticks;
+		jiffies_64 += lost_ticks * jiffies_increment;
  	}
  	hpet_last = hpet_current;

@@ -415,7 +415,7 @@ static void mark_offset_tsc(void)
  	lost = delta/(1000000/HZ);
  	delay = delta%(1000000/HZ);
  	if (lost >= 2) {
-		jiffies_64 += lost-1;
+		jiffies_64 += (lost * jiffies_increment) - 1;

  		/* sanity check to ensure we're not always losing ticks */
  		if (lost_count++ > 100) {
@@ -448,7 +448,7 @@ static void mark_offset_tsc(void)
  	 * usec delta is > 90% # of usecs/tick)
  	 */
  	if (lost && abs(delay - delay_at_last_interrupt) > (900000/HZ))
-		jiffies_64++;
+		jiffies_64 += jiffies_increment;
  }

  static int __init init_tsc(char* override)
diff -upr linux-2.6.13-rc3-orig/include/linux/jiffies.h 
linux-2.6.13-rc3/include/linux/jiffies.h
--- linux-2.6.13-rc3-orig/include/linux/jiffies.h	2005-06-17 
21:48:29.000000000 +0200
+++ linux-2.6.13-rc3/include/linux/jiffies.h	2005-07-15 
03:42:57.000000000 +0200
@@ -83,6 +83,7 @@
   */
  extern u64 __jiffy_data jiffies_64;
  extern unsigned long volatile __jiffy_data jiffies;
+extern u16 jiffies_increment;

  #if (BITS_PER_LONG < 64)
  u64 get_jiffies_64(void);
diff -upr linux-2.6.13-rc3-orig/kernel/timer.c 
linux-2.6.13-rc3/kernel/timer.c
--- linux-2.6.13-rc3-orig/kernel/timer.c	2005-07-14 20:34:24.000000000 +0200
+++ linux-2.6.13-rc3/kernel/timer.c	2005-07-15 02:55:45.000000000 +0200
@@ -948,7 +948,7 @@ static inline void update_times(void)

  void do_timer(struct pt_regs *regs)
  {
-	jiffies_64++;
+	jiffies_64 += jiffies_increment;
  	update_times();
  }



-- 
Jesper Juhl


[-- Attachment #2: jiffies_interval.patch --]
[-- Type: text/x-patch, Size: 3847 bytes --]

diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/time.c linux-2.6.13-rc3/arch/i386/kernel/time.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/time.c	2005-07-14 20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/time.c	2005-07-15 04:02:50.000000000 +0200
@@ -75,6 +75,7 @@ int pit_latch_buggy;              /* ext
 #include "do_timer.h"
 
 u64 jiffies_64 = INITIAL_JIFFIES;
+u16 jiffies_increment = 1;
 
 EXPORT_SYMBOL(jiffies_64);
 
@@ -481,3 +482,27 @@ void __init time_init(void)
 
 	time_init_hook();
 }
+
+static int __init jiffies_increment_setup(char *str)
+{
+	printk(KERN_NOTICE "setting up jiffies_increment : ");
+	if (str) {
+		printk("kernel_hz = %s, ", str);
+	} else {
+		printk("kernel_hz is unset, ");
+	}
+	if (!strncmp("100", str, 3)) {
+		jiffies_increment = 10;
+		printk("jiffies_increment set to 10, effective HZ will be 100\n");
+	} else if (!strncmp("250", str, 3)) {
+		jiffies_increment = 4;
+		printk("jiffies_increment set to 4, effective HZ will be 250\n");
+	} else {
+		jiffies_increment = 1;
+		printk("jiffies_increment set to 1, effective HZ will be 1000\n");
+	}
+
+	return 1;
+}
+
+__setup("kernel_hz=", jiffies_increment_setup);
diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_pm.c linux-2.6.13-rc3/arch/i386/kernel/timers/timer_pm.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_pm.c	2005-07-14 20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/timers/timer_pm.c	2005-07-15 02:59:39.000000000 +0200
@@ -176,7 +176,7 @@ static void mark_offset_pmtmr(void)
 
 	/* compensate for lost ticks */
 	if (lost >= 2)
-		jiffies_64 += lost - 1;
+		jiffies_64 += (lost * jiffies_increment) - 1;
 
 	/* don't calculate delay for first run,
 	   or if we've got less then a tick */
diff -upr linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_tsc.c linux-2.6.13-rc3/arch/i386/kernel/timers/timer_tsc.c
--- linux-2.6.13-rc3-orig/arch/i386/kernel/timers/timer_tsc.c	2005-07-14 20:33:35.000000000 +0200
+++ linux-2.6.13-rc3/arch/i386/kernel/timers/timer_tsc.c	2005-07-15 02:59:13.000000000 +0200
@@ -193,7 +193,7 @@ static void mark_offset_tsc_hpet(void)
 	offset = hpet_readl(HPET_T0_CMP) - hpet_tick;
 	if (unlikely(((offset - hpet_last) > hpet_tick) && (hpet_last != 0))) {
 		int lost_ticks = (offset - hpet_last) / hpet_tick;
-		jiffies_64 += lost_ticks;
+		jiffies_64 += lost_ticks * jiffies_increment;
 	}
 	hpet_last = hpet_current;
 
@@ -415,7 +415,7 @@ static void mark_offset_tsc(void)
 	lost = delta/(1000000/HZ);
 	delay = delta%(1000000/HZ);
 	if (lost >= 2) {
-		jiffies_64 += lost-1;
+		jiffies_64 += (lost * jiffies_increment) - 1;
 
 		/* sanity check to ensure we're not always losing ticks */
 		if (lost_count++ > 100) {
@@ -448,7 +448,7 @@ static void mark_offset_tsc(void)
 	 * usec delta is > 90% # of usecs/tick)
 	 */
 	if (lost && abs(delay - delay_at_last_interrupt) > (900000/HZ))
-		jiffies_64++;
+		jiffies_64 += jiffies_increment;
 }
 
 static int __init init_tsc(char* override)
diff -upr linux-2.6.13-rc3-orig/include/linux/jiffies.h linux-2.6.13-rc3/include/linux/jiffies.h
--- linux-2.6.13-rc3-orig/include/linux/jiffies.h	2005-06-17 21:48:29.000000000 +0200
+++ linux-2.6.13-rc3/include/linux/jiffies.h	2005-07-15 03:42:57.000000000 +0200
@@ -83,6 +83,7 @@
  */
 extern u64 __jiffy_data jiffies_64;
 extern unsigned long volatile __jiffy_data jiffies;
+extern u16 jiffies_increment;
 
 #if (BITS_PER_LONG < 64)
 u64 get_jiffies_64(void);
diff -upr linux-2.6.13-rc3-orig/kernel/timer.c linux-2.6.13-rc3/kernel/timer.c
--- linux-2.6.13-rc3-orig/kernel/timer.c	2005-07-14 20:34:24.000000000 +0200
+++ linux-2.6.13-rc3/kernel/timer.c	2005-07-15 02:55:45.000000000 +0200
@@ -948,7 +948,7 @@ static inline void update_times(void)
 
 void do_timer(struct pt_regs *regs)
 {
-	jiffies_64++;
+	jiffies_64 += jiffies_increment;
 	update_times();
 }
 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Linus Torvalds]

On Fri, 15 Jul 2005, Jesper Juhl wrote:
>
> It's buggy, that I know. setting kernel_hz (the new boot parameter) to 
> 250 causes my system clock to run at something like 4-5 times normal 
> speed

4 times normal. You don't actually make the timer interrupt happen at 
250Hz, so the timer will be programmed to run at the full 1kHz.

You also need to actually change the LATCH define (in 
include/linux/jiffies.h) to take this into account (there might be 
something else too).

So 

	/* LATCH is used in the interval timer and ftape setup. */
	#define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)  /* For divider */

should become something like

	/* LATCH is used in the interval timer and ftape setup. */
	#define LATCH  ((CLOCK_TICK_RATE*jiffies_increment + HZ/2) / HZ)  /* For divider */

and you might be getting closer.

Of course, you need to make sure that LATCH is used only after 
jiffies_increment is set up. See "setup_pit_timer(void)" in
arch/i386/kernel/timers/timer_pit.c for more details.

		Linus

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/15/05, Linus Torvalds <torvalds@osdl.org> wrote:
> 
> On Fri, 15 Jul 2005, Jesper Juhl wrote:
> >
> > It's buggy, that I know. setting kernel_hz (the new boot parameter) to
> > 250 causes my system clock to run at something like 4-5 times normal
> > speed
> 
> 4 times normal. You don't actually make the timer interrupt happen at
> 250Hz, so the timer will be programmed to run at the full 1kHz.
> 
Right, that's the basic problem. I increase jiffies at a higher rate
but didn't slow the timer interrupt down at the same time.

> You also need to actually change the LATCH define (in
> include/linux/jiffies.h) to take this into account (there might be
> something else too).
> 
[...]
> and you might be getting closer.
> 
> Of course, you need to make sure that LATCH is used only after
> jiffies_increment is set up. See "setup_pit_timer(void)" in
> arch/i386/kernel/timers/timer_pit.c for more details.
> 

Thank you for all the pointers and hints. This is a new area of code
for me, so I'll need some time to poke around - the above helps a lot.
Unfortunately I won't have any time to work on this today, but I'll
see if I can get a working implementation together tomorrow.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/15/05, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> On 7/15/05, Linus Torvalds <torvalds@osdl.org> wrote:
> >
> > On Fri, 15 Jul 2005, Jesper Juhl wrote:
> > >
> > > It's buggy, that I know. setting kernel_hz (the new boot parameter) to
> > > 250 causes my system clock to run at something like 4-5 times normal
> > > speed
> >
> > 4 times normal. You don't actually make the timer interrupt happen at
> > 250Hz, so the timer will be programmed to run at the full 1kHz.
> >
> Right, that's the basic problem. I increase jiffies at a higher rate
> but didn't slow the timer interrupt down at the same time.
> 
> > You also need to actually change the LATCH define (in
> > include/linux/jiffies.h) to take this into account (there might be
> > something else too).
> >
> [...]
> > and you might be getting closer.
> >
> > Of course, you need to make sure that LATCH is used only after
> > jiffies_increment is set up. See "setup_pit_timer(void)" in
> > arch/i386/kernel/timers/timer_pit.c for more details.
> >
> 
> Thank you for all the pointers and hints. This is a new area of code
> for me, so I'll need some time to poke around - the above helps a lot.
> Unfortunately I won't have any time to work on this today, but I'll
> see if I can get a working implementation together tomorrow.
> 

Ok, I'm afraid I'm going to need another hint or two.

I've been looking at the timer code and getting thoroughly confused.
I've tried to find out where we actually program the interrupt
controller to say "this is the frequency I want you to interrupt me
at", but I can't seem to find it.
I'm aware that there are multiple possible time sources, and I've been
looking at the 8259 code, the ioapic code, the hpet code and various
other bits in arch/i386/kernel/ and arch/i386/kernel/timers/ , but I
seem to end up getting confused about all the different defines like 
CLOCK_TICK_RATE, ACTHZ, TICK_NSEC, TICK_USEC, etc...

Where do we actually program the tick rate we want?

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sun, 2005-07-17 at 04:13 +0200, Jesper Juhl wrote:
> Where do we actually program the tick rate we want?
> 

In arch/i386/kernel/timers/timer_pit.c:

    166 void setup_pit_timer(void)
    167 {
    168         unsigned long flags;
    169 
    170         spin_lock_irqsave(&i8253_lock, flags);
    171         outb_p(0x34,PIT_MODE);          /* binary, mode 2, LSB/MSB, ch 0 */
    172         udelay(10);
    173         outb_p(LATCH & 0xff , PIT_CH0); /* LSB */
    174         udelay(10);
    175         outb(LATCH >> 8 , PIT_CH0);     /* MSB */
    176         spin_unlock_irqrestore(&i8253_lock, flags);
    177 }
    178 

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Nish Aravamudan]

On 7/16/05, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> On 7/15/05, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> > On 7/15/05, Linus Torvalds <torvalds@osdl.org> wrote:
> > >
> > > On Fri, 15 Jul 2005, Jesper Juhl wrote:
> > > >
> > > > It's buggy, that I know. setting kernel_hz (the new boot parameter) to
> > > > 250 causes my system clock to run at something like 4-5 times normal
> > > > speed
> > >
> > > 4 times normal. You don't actually make the timer interrupt happen at
> > > 250Hz, so the timer will be programmed to run at the full 1kHz.
> > >
> > Right, that's the basic problem. I increase jiffies at a higher rate
> > but didn't slow the timer interrupt down at the same time.
> >
> > > You also need to actually change the LATCH define (in
> > > include/linux/jiffies.h) to take this into account (there might be
> > > something else too).
> > >
> > [...]
> > > and you might be getting closer.
> > >
> > > Of course, you need to make sure that LATCH is used only after
> > > jiffies_increment is set up. See "setup_pit_timer(void)" in
> > > arch/i386/kernel/timers/timer_pit.c for more details.
> > >
> >
> > Thank you for all the pointers and hints. This is a new area of code
> > for me, so I'll need some time to poke around - the above helps a lot.
> > Unfortunately I won't have any time to work on this today, but I'll
> > see if I can get a working implementation together tomorrow.
> >
> 
> Ok, I'm afraid I'm going to need another hint or two.
> 
> I've been looking at the timer code and getting thoroughly confused.
> I've tried to find out where we actually program the interrupt
> controller to say "this is the frequency I want you to interrupt me
> at", but I can't seem to find it.
> I'm aware that there are multiple possible time sources, and I've been
> looking at the 8259 code, the ioapic code, the hpet code and various
> other bits in arch/i386/kernel/ and arch/i386/kernel/timers/ , but I
> seem to end up getting confused about all the different defines like
> CLOCK_TICK_RATE, ACTHZ, TICK_NSEC, TICK_USEC, etc...
> 
> Where do we actually program the tick rate we want?

As you've seen, I think it depends on the timesource: for the PIT, it
would be arch/i386/kernel/timers/timer_pit.c::setup_pit_timer(). In
that function, you'll notice we outb() the LATCH value. I think there
are similar functions in the other timesources, e.g. the
arch/i386/kernel/apic.c::setup_APIC_timer().

Does that help some?

Thanks,
Nish

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Sat, 2005-07-16 at 19:35 -0700, Nish Aravamudan wrote: 
> As you've seen, I think it depends on the timesource: for the PIT, it
> would be arch/i386/kernel/timers/timer_pit.c::setup_pit_timer().

That one looks pretty straightforward.
arch/i386/kernel/timers/timer_tsc.c really looks like fun.  So many
corner cases...

BTW shouldn't this code from mark_offset_tsc():

402         if (pit_latch_buggy) {
403                 /* get center value of last 3 time lutch */
404                 if ((count2 >= count && count >= count1)
405                     || (count1 >= count && count >= count2)) {
406                         count2 = count1; count1 = count;
407                 } else if ((count1 >= count2 && count2 >= count)
408                            || (count >= count2 && count2 >= count1)) {
409                         countmp = count;count = count2;
410                         count2 = count1;count1 = countmp;
411                 } else {
412                         count2 = count1; count1 = count; count = count1;
413                 }
414         }

use an ifdef?  It only applies to cyrix_55x0, and mark_offset_tsc is a
pretty hot path.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[randy_dunlap]

On Sat, 16 Jul 2005 23:55:17 -0400 Lee Revell wrote:

> On Sat, 2005-07-16 at 19:35 -0700, Nish Aravamudan wrote: 
> > As you've seen, I think it depends on the timesource: for the PIT, it
> > would be arch/i386/kernel/timers/timer_pit.c::setup_pit_timer().
> 
> That one looks pretty straightforward.
> arch/i386/kernel/timers/timer_tsc.c really looks like fun.  So many
> corner cases...
> 
> BTW shouldn't this code from mark_offset_tsc():
> 
> 402         if (pit_latch_buggy) {
> 403                 /* get center value of last 3 time lutch */
> 404                 if ((count2 >= count && count >= count1)
> 405                     || (count1 >= count && count >= count2)) {
> 406                         count2 = count1; count1 = count;
> 407                 } else if ((count1 >= count2 && count2 >= count)
> 408                            || (count >= count2 && count2 >= count1)) {
> 409                         countmp = count;count = count2;
> 410                         count2 = count1;count1 = countmp;
> 411                 } else {
> 412                         count2 = count1; count1 = count; count = count1;
> 413                 }
> 414         }
> 
> use an ifdef?  It only applies to cyrix_55x0, and mark_offset_tsc is a
> pretty hot path.

I see your point, but several distros build kernels that run on
almost any x86-32 machine, so I think that it's there as is
for universal-kernel support.

---
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Sat, Jul 23, 2005 at 04:40:46PM -0700, randy_dunlap wrote:
> On Sat, 16 Jul 2005 23:55:17 -0400 Lee Revell wrote:
> 
> > On Sat, 2005-07-16 at 19:35 -0700, Nish Aravamudan wrote: 
> > > As you've seen, I think it depends on the timesource: for the PIT, it
> > > would be arch/i386/kernel/timers/timer_pit.c::setup_pit_timer().
> > 
> > That one looks pretty straightforward.
> > arch/i386/kernel/timers/timer_tsc.c really looks like fun.  So many
> > corner cases...
> > 
> > BTW shouldn't this code from mark_offset_tsc():
> > 
> > 402         if (pit_latch_buggy) {
> > 403                 /* get center value of last 3 time lutch */
> > 404                 if ((count2 >= count && count >= count1)
> > 405                     || (count1 >= count && count >= count2)) {
> > 406                         count2 = count1; count1 = count;
> > 407                 } else if ((count1 >= count2 && count2 >= count)
> > 408                            || (count >= count2 && count2 >= count1)) {
> > 409                         countmp = count;count = count2;
> > 410                         count2 = count1;count1 = countmp;
> > 411                 } else {
> > 412                         count2 = count1; count1 = count; count = count1;
> > 413                 }
> > 414         }
> > 
> > use an ifdef?  It only applies to cyrix_55x0, and mark_offset_tsc is a
> > pretty hot path.
> 
> I see your point, but several distros build kernels that run on
> almost any x86-32 machine, so I think that it's there as is
> for universal-kernel support.
 
The same latch bug is in stone age Intel Pentium chipsets and some
medieval SiS chipsets. VIA chipsets from the middle age have another
interesting bug in the PIT.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[randy_dunlap]

On Fri, 15 Jul 2005 05:46:44 +0200 Jesper Juhl wrote:

> +static int __init jiffies_increment_setup(char *str)
> +{
> +	printk(KERN_NOTICE "setting up jiffies_increment : ");
> +	if (str) {
> +		printk("kernel_hz = %s, ", str);
> +	} else {
> +		printk("kernel_hz is unset, ");
> +	}
> +	if (!strncmp("100", str, 3)) {

BTW, if someone enters "kernel_hz=1000", this check (above) for "100"
matches (detects) 100, not 1000.

> +		jiffies_increment = 10;
> +		printk("jiffies_increment set to 10, effective HZ will be 100\n");
> +	} else if (!strncmp("250", str, 3)) {
> +		jiffies_increment = 4;
> +		printk("jiffies_increment set to 4, effective HZ will be 250\n");
> +	} else {
> +		jiffies_increment = 1;
> +		printk("jiffies_increment set to 1, effective HZ will be 1000\n");
> +	}
> +
> +	return 1;
> +}

---
~Randy

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Jesper Juhl]

On 7/24/05, randy_dunlap <rdunlap@xenotime.net> wrote:
> On Fri, 15 Jul 2005 05:46:44 +0200 Jesper Juhl wrote:
> 
> > +static int __init jiffies_increment_setup(char *str)
> > +{
> > +     printk(KERN_NOTICE "setting up jiffies_increment : ");
> > +     if (str) {
> > +             printk("kernel_hz = %s, ", str);
> > +     } else {
> > +             printk("kernel_hz is unset, ");
> > +     }
> > +     if (!strncmp("100", str, 3)) {
> 
> BTW, if someone enters "kernel_hz=1000", this check (above) for "100"
> matches (detects) 100, not 1000.
> 
ouch. You are right - thanks. I'll be sure to fix that.
I haven't had time to look more at this little thing for the last few
days, but I'll get back to it soon. Thank you for the feedback.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

Jesper Juhl wrote:

>On 7/24/05, randy_dunlap <rdunlap@xenotime.net> wrote:
>  
>
>>On Fri, 15 Jul 2005 05:46:44 +0200 Jesper Juhl wrote:
>>
>>    
>>
>>>+static int __init jiffies_increment_setup(char *str)
>>>+{
>>>+     printk(KERN_NOTICE "setting up jiffies_increment : ");
>>>+     if (str) {
>>>+             printk("kernel_hz = %s, ", str);
>>>+     } else {
>>>+             printk("kernel_hz is unset, ");
>>>+     }
>>>+     if (!strncmp("100", str, 3)) {
>>>      
>>>
>>BTW, if someone enters "kernel_hz=1000", this check (above) for "100"
>>matches (detects) 100, not 1000.
>>
>>    
>>
>ouch. You are right - thanks. I'll be sure to fix that.
>I haven't had time to look more at this little thing for the last few
>days, but I'll get back to it soon. Thank you for the feedback.
>
>  
>
I have to admit that I like paranoid programming, and would rather see 
this look for "kernel_hz=" then convert the digits after to an integer 
and validate that. It would catch invalid values far better, allow other 
values to be either implemented as best as is possible if desired, and 
NOT ignore invalid values if they didn't match these predefined strings.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Russell King]

On Thu, Jul 14, 2005 at 05:24:39PM -0700, Linus Torvalds wrote:
> HOWEVER. I bet that somebody who really really cares (hint hint) could
> easily make HZ be 1000, and then dynamically tweak the divisor at bootup
> to be either 1000, 250, or 100, and then increment "jiffies" by 1, 4 or
> 10.

Wouldn't this be better suited to a VST like implementation, but
instead of using VST to dynamically adjust the timer divisor, it
operates in a "fixed" mode?

(I'm arguing this way because ARM has VST merged already, and all
there are no changes required to the core kernel code to achieve
this.)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Pavel Machek]

Hi!

> > Alan tested it and said that 250HZ does not save much power anyway.
> 
> Len Brown, a year ago: "The bottom line number to laptop users is battery
> lifetime.  Just today somebody complained to me that Windows gets twice the
> battery life that Linux does."
> 
> And "Maybe I can get Andy Grover over in the moble lab to get some time on
> that fancy power measurement setup they have...
> 
> "My expectation is if we want to beat the competition, we'll want the
> ability to go *under* 100Hz."
> 
> But then, power consumption of the display should preponderate, so it's not
> clear.
> 
> Len, any updates on the relationship between HZ and power consumption?

Last time I checked, HZ=100 to HZ=1000 difference was about 1W, about
twice as much as disk spinning up vs. disk spinned down.
								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Diego Calleja]

El Wed, 13 Jul 2005 15:13:44 -0400,
Lee Revell <rlrevell@joe-job.com> escribió:

> How about 500?  This might be good enough to solve the MIDI problem.

In 2.4 suse kernels you could set HZ at boot time. Since it doesn't seems
possible to find a HZ value that makes everybody happy, would be feasible
to do what suse did?

Some distros are already carrying programs which try to guess if a box
is a laptop or not, and then set some parameters (laptop-mode, cpufreq)
if they are. If HZ could be set at boot time they could use those heuristics
to add a extra boot flag in the boot loader without needing to recompile
the kernel

It'd be very uselful for distros, the same distro can be installed in a
multimedia-oriented box or a laptop, and no default is going to make
everybody happy...

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Benjamin LaHaise]

On Wed, Jul 13, 2005 at 12:10:48PM -0700, Linus Torvalds wrote:
> Long-term time drift is a known issue, and is unavoidable since you don't 
> even know the exact frequency of the crystal, since that is not only not 
> that exact in the first place, it depends on temperature etc. So long-term 
> time drift is something that we inevitably have to use things like NTP to 
> handle, if you want an exact clock.

diz gave #kernel a good diatribe a few weeks ago about the headaches 
associated with using the PIT as a clock source, with one of the more 
interesting tidbits being that chipsets will pull the frequency higher 
and lower at times in order to implement spread spectrum to comply with 
RF emissions.  The RTC doesn't suffer from this, but it only provides 
HZ values which are powers of two.  How bad would 256 Hz be?

		-ben
-- 
"Time is what keeps everything from happening all at once." -- John Wheeler

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Wed, Jul 13, 2005 at 03:35:40PM -0400, Benjamin LaHaise wrote:

> On Wed, Jul 13, 2005 at 12:10:48PM -0700, Linus Torvalds wrote:
> > Long-term time drift is a known issue, and is unavoidable since you don't 
> > even know the exact frequency of the crystal, since that is not only not 
> > that exact in the first place, it depends on temperature etc. So long-term 
> > time drift is something that we inevitably have to use things like NTP to 
> > handle, if you want an exact clock.
> 
> diz gave #kernel a good diatribe a few weeks ago about the headaches 
> associated with using the PIT as a clock source, with one of the more 
> interesting tidbits being that chipsets will pull the frequency higher 
> and lower at times in order to implement spread spectrum to comply with 
> RF emissions.  The RTC doesn't suffer from this, but it only provides 
> HZ values which are powers of two.  How bad would 256 Hz be?
 
The RTC historically used to have a lower quality (cheaper) crystal than
the 14.318 MHz crystal used for everything else. But with the spread
spectrum modulation of frequency, the PIT may finally be worse to
consider the RTC again.

Another BIG problem with RTC is that it doesn't allow reading its
internal counter like the PIT does, making TSC interpolation even harder.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Benjamin LaHaise]

On Wed, Jul 13, 2005 at 09:41:15PM +0200, Vojtech Pavlik wrote:
> The RTC historically used to have a lower quality (cheaper) crystal than
> the 14.318 MHz crystal used for everything else. But with the spread
> spectrum modulation of frequency, the PIT may finally be worse to
> consider the RTC again.

32.768kHz crystals are pretty much standard for use in digital clocks.  
Checking an electronics catalogue shows a fairly reasonable -60ppm to 
+30ppm rating over the operating temperature range (implying the device 
is tuned for 20C room temperature).

> Another BIG problem with RTC is that it doesn't allow reading its
> internal counter like the PIT does, making TSC interpolation even harder.

That's one thing I truely dislike about the current timer code.  If we 
could program the RTC interrupt to come into the system as an NMI (iirc 
oprofile already has code to do this), we could get much better TSC 
interpolation since we would be sampling the TSC at a much smaller, less 
variable offset, which can only be a good thing.

		-ben
-- 
"Time is what keeps everything from happening all at once." -- John Wheeler

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Wed, 13 Jul 2005, Benjamin LaHaise wrote:

> That's one thing I truely dislike about the current timer code.  If we 
> could program the RTC interrupt to come into the system as an NMI (iirc 
> oprofile already has code to do this), we could get much better TSC 
> interpolation since we would be sampling the TSC at a much smaller, less 
> variable offset, which can only be a good thing.

 And we'd get a lot more crashes on broken systems that do not handle NMIs 
in the SMM -- this is the very reason the NMI watchdog is disabled these 
days by default.  A whole lot of systems simply cannot handle NMIs 
happening randomly.

 Programming an I/O APIC to deliver the RTC interrupt (or any other that's 
edge-triggered) as an NMI is itself trivial (we can do this for the PIT 
for the NMI watchdog already).

  Maciej

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[George Anzinger]

Linus Torvalds wrote:
> 
> On Wed, 13 Jul 2005, Vojtech Pavlik wrote:
> 
>>No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
>>a counter that counts the ticks in nanoseconds (xtime ...), the first
>>will be exact, the second will be accumulating an error.
> 
> 
> It's not even that we have a counter like that, it's the simple fact that
> we have a standard interface to user space that is based on milli-, micro-
> and nanoseconds.
> 
> (For "poll()", "struct timeval" and "struct timespec" respectively).
> 
> It's totally pointless saying that we can do 864 Hz "exactly", when the
> fact is that all the timeouts we ever get from user space aren't in that 
> format. So the only thing that matters is how close to a millisecond we 
> can get, not how close to some random number.
> 
> So we do a lot of conversions from "struct timeval" to "jiffies", and if
> you don't take the error in that conversion into account, then you're
> ignoring what is likely a _bigger_ error.
> 
> Long-term time drift is a known issue, and is unavoidable since you don't 
> even know the exact frequency of the crystal, since that is not only not 
> that exact in the first place, it depends on temperature etc. So long-term 
> time drift is something that we inevitably have to use things like NTP to 
> handle, if you want an exact clock.
> 
> And in short-term things, the timeval/jiffie conversion is likely to be a 
> _bigger_ issue than the crystal frequency conversion.
> 
> So we should aim for a HZ value that makes it easy to convert to and from
> the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> good values for that reason. 864 is not.Linus Torvalds wrote:
> 
> On Wed, 13 Jul 2005, Vojtech Pavlik wrote:
> 
>>No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
>>a counter that counts the ticks in nanoseconds (xtime ...), the first
>>will be exact, the second will be accumulating an error.
> 
> 
> It's not even that we have a counter like that, it's the simple fact that
> we have a standard interface to user space that is based on milli-, micro-
> and nanoseconds.
> 
> (For "poll()", "struct timeval" and "struct timespec" respectively).
> 
> It's totally pointless saying that we can do 864 Hz "exactly", when the
> fact is that all the timeouts we ever get from user space aren't in that 
> format. So the only thing that matters is how close to a millisecond we 
> can get, not how close to some random number.
> 
> So we do a lot of conversions from "struct timeval" to "jiffies", and if
> you don't take the error in that conversion into account, then you're
> ignoring what is likely a _bigger_ error.
> 
> Long-term time drift is a known issue, and is unavoidable since you don't 
> even know the exact frequency of the crystal, since that is not only not 
> that exact in the first place, it depends on temperature etc. So long-term 
> time drift is something that we inevitably have to use things like NTP to 
> handle, if you want an exact clock.
> 
> And in short-term things, the timeval/jiffie conversion is likely to be a 
> _bigger_ issue than the crystal frequency conversion.
> 
> So we should aim for a HZ value that makes it easy to convert to and from
> the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> good values for that reason. 864 is not.

Uh, WAIT A NANOSECOND!  Look at what we are doing today in that department.  The 
key is not the ability to convert based on the value of HZ but on the implied 
value of jiffie given CLOCK_TICK_RATE.  Today the value we use for jiffie is 
999849 nanoseconds which is what the given CLOCK_TICK_RATE and HZ end up getting 
from the PIT.

By the time the user comes along we have TICK_NSEC and the current conversion 
routines which are not exactly simple but they are correct.

-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1688 bytes --]

On Thu, 14 Jul 2005 05:10, Linus Torvalds wrote:
> On Wed, 13 Jul 2005, Vojtech Pavlik wrote:
> > No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
> > a counter that counts the ticks in nanoseconds (xtime ...), the first
> > will be exact, the second will be accumulating an error.
>
> It's not even that we have a counter like that, it's the simple fact that
> we have a standard interface to user space that is based on milli-, micro-
> and nanoseconds.
>
> (For "poll()", "struct timeval" and "struct timespec" respectively).
>
> It's totally pointless saying that we can do 864 Hz "exactly", when the
> fact is that all the timeouts we ever get from user space aren't in that
> format. So the only thing that matters is how close to a millisecond we
> can get, not how close to some random number.

That may be the case but when I've measured the actual delay of schedule 
timeout when using nanosleep from userspace, the average at 1000Hz was 1.4ms 
+/- 1.5 sd . When we're expecting a sleep of "up to 1ms" we're getting 50% 
longer than the longest expected. Purely mathematically the accuracy of 
changing HZ from 1000 -> 864 will not bring with it any significant change to 
the accuracy. This can easily be measured as well to confirm. 

Using schedule timeout as an argument against it doesn't hold for me. 
Vojtech's comment of :
> "No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have a 
> counter that counts the ticks in nanoseconds (xtime ...), the first will be 
> exact, the second will be accumulating an error." 
is probably the most valid argument against such a funky number. 

Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[George Anzinger]

Con Kolivas wrote:
> On Thu, 14 Jul 2005 05:10, Linus Torvalds wrote:
> 
>>On Wed, 13 Jul 2005, Vojtech Pavlik wrote:
>>
>>>No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have
>>>a counter that counts the ticks in nanoseconds (xtime ...), the first
>>>will be exact, the second will be accumulating an error.
>>
>>It's not even that we have a counter like that, it's the simple fact that
>>we have a standard interface to user space that is based on milli-, micro-
>>and nanoseconds.
>>
>>(For "poll()", "struct timeval" and "struct timespec" respectively).
>>
>>It's totally pointless saying that we can do 864 Hz "exactly", when the
>>fact is that all the timeouts we ever get from user space aren't in that
>>format. So the only thing that matters is how close to a millisecond we
>>can get, not how close to some random number.
> 
> 
> That may be the case but when I've measured the actual delay of schedule 
> timeout when using nanosleep from userspace, the average at 1000Hz was 1.4ms 
> +/- 1.5 sd . When we're expecting a sleep of "up to 1ms" we're getting 50% 
> longer than the longest expected. Purely mathematically the accuracy of 
> changing HZ from 1000 -> 864 will not bring with it any significant change to 
> the accuracy. This can easily be measured as well to confirm. 
> 
> Using schedule timeout as an argument against it doesn't hold for me. 
> Vojtech's comment of :
> 
>>"No, but 1/1000Hz = 1000000ns, while 1/864Hz = 1157407.407ns. If you have a 
>>counter that counts the ticks in nanoseconds (xtime ...), the first will be 
>>exact, the second will be accumulating an error." 
> 
> is probably the most valid argument against such a funky number. 

No, that doesn't hold water either.  We already jigger jiffie to be _close_ to 
1/HZ and closer still to what we can get from the PIT as its true period (for 
example, today the jiffie is 999849 nanoseconds) and this too is only accurate 
to the nanosecond.  Here are the jiffie values for several HZ values using the 
formulas in the code which use the TICK_RATE as given by the hardware.  Note the 
error here is the difference between an asked for repeating timer of 1 second 
and what the system clock on the same system says, NOT what real time is in 
either case, just relative between the two.  In otherwords, if you set up an 
itimer to signal every second and looked at the long term drift between the 
signals it gives and the system clock you would see the itimer drifting by 
~914ppm (with HZ = 846).

HZ  	TICK RATE	jiffie(ns)	second(ns)	 error (ppbillion)
  100	 1193182	10000000	1000000000	       0
  200	 1193182	 5000098	1000019600	   19600
  250	 1193182	 4000250	1000062500	   62500
  500	 1193182	 1999688	1001843688	 1843688
1000	 1193182	  999848	1000847848	  847848
  846	 1193182	 1181717	1000914299	  914299

> 
> Cheers,
> Con

-- 
George Anzinger   george@mvista.com
HRT (High-res-timers):  http://sourceforge.net/projects/high-res-timers/

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Krzysztof Halasa]

Linus Torvalds <torvalds@osdl.org> writes:

> And in short-term things, the timeval/jiffie conversion is likely to be a 
> _bigger_ issue than the crystal frequency conversion.
>
> So we should aim for a HZ value that makes it easy to convert to and from
> the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> good values for that reason. 864 is not.

Probably only theoretical, and probably the hardware isn't up to it...
But what if we have:
- 64-bit jiffies done in hardware (a counter). 1 cycle = 1 microsecond
  or even a CPU clock cycle. Can *APIC or another HPET do that?
- 64-bit "match timer" (i.e., a register in the counter which fires IRQ
  when it matches the counter value)
- the CPU(s) sorting the timer list and programming "match timer" with
  software timer next to be executed. Upon firing the timer, a new "next
  to be executed" timer would be programmed into the counter's "match
  timer".

We would have no timer ticks when nobody requested them - the CPUs would
be allowed to sleep for, say, even 50 ms when no task is RUNNING.
-- 
Krzysztof Halasa

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Thu, Jul 14, 2005 at 12:25:40PM +0200, Krzysztof Halasa wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
> 
> > And in short-term things, the timeval/jiffie conversion is likely to be a 
> > _bigger_ issue than the crystal frequency conversion.
> >
> > So we should aim for a HZ value that makes it easy to convert to and from
> > the standard user-space interface formats. 100Hz, 250Hz and 1000Hz are all
> > good values for that reason. 864 is not.
> 
> Probably only theoretical, and probably the hardware isn't up to it...
> But what if we have:
> - 64-bit jiffies done in hardware (a counter). 1 cycle = 1 microsecond
>   or even a CPU clock cycle. Can *APIC or another HPET do that?

HPETs have a fixed frequency (usually 14.31818 MHz, but that depends
on the manufacturer).

> - 64-bit "match timer" (i.e., a register in the counter which fires IRQ
>   when it matches the counter value)

That's implemented in the HPET hardware.

> - the CPU(s) sorting the timer list and programming "match timer" with
>   software timer next to be executed. Upon firing the timer, a new "next
>   to be executed" timer would be programmed into the counter's "match
>   timer".
> 
> We would have no timer ticks when nobody requested them - the CPUs would
> be allowed to sleep for, say, even 50 ms when no task is RUNNING.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Krzysztof Halasa]

Vojtech Pavlik <vojtech@suse.cz> writes:

> HPETs have a fixed frequency (usually 14.31818 MHz, but that depends
> on the manufacturer).
>
>> - 64-bit "match timer" (i.e., a register in the counter which fires IRQ
>>   when it matches the counter value)
>
> That's implemented in the HPET hardware.

Interesting. So it could theoretically work. Unless the machine in
question lacks HPET.
-- 
Krzysztof Halasa

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Bill Davidsen]

David Lang wrote:

> On Wed, 13 Jul 2005, Bill Davidsen wrote:
>
>> How serious is the 1/HZ = sane problem, and more to the point how 
>> many programs get the HZ value with a system call as opposed to 
>> including a header or building it in? I know some of my older 
>> programs use header files, that was part of the planning for the 
>> future even before 2.5 started. At the time I didn't expect to have 
>> to use the system call.
>
>
> in binary 1/100 or 1/1000 are not sane values to start with so I don't 
> think that that this is likly to be that critical (remembering that 
> the kernel doesn't do floating point math) 


The kernel isn't the issue, it's programs which do timing and get values 
in ticks which they convert to time by dividing by HZ. I at least get 
that from a header, proper way would be with the syscall.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Vojtech Pavlik]

On Tue, Jul 12, 2005 at 10:39:00PM +1000, Con Kolivas wrote:
> On Tue, 12 Jul 2005 22:10, Vojtech Pavlik wrote:
> > The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> > and is divided by 12 to get PIT tick rate
> >
> > 	14.3181818 MHz / 12 = 1193182 Hz
> >
> > The reality is that the crystal is usually off by 50-100 ppm from the
> > standard value, depending on temperature.
> >
> >     HZ   ticks/jiffie  1 second      error (ppm)
> > ---------------------------------------------------
> >    100      11932      1.000015238      15.2
> >    200       5966      1.000015238      15.2
> >    250       4773      1.000057143      57.1
> >    300       3977      0.999931429     -68.6
> >    333       3583      0.999964114     -35.9
> >    500       2386      0.999847619    -152.4
> >   1000       1193      0.999847619    -152.4
> >
> > Some HZ values indeed fit the tick frequency better than others, up to
> > 333 the error is lost in the physical error of the crystal, for 500 and
> > 1000, it definitely is larger, and thus noticeable.
> >
> > Some (less round and nice) values of HZ would fit even better:
> >
> >     HZ   ticks/jiffie  1 second      error (ppm)
> > ---------------------------------------------------
> >     82      14551      1.000000152       0.2
> 
> 
> Most interesting... Would 838 Hz be a much better choice than 1000 then? 
> (apart from the ugliness).

No, 838 isn't significantly better. 864 and 627 would be better
candidates:

    HZ   ticks/jiffie  1 second      error (ppm)
---------------------------------------------------
   627       1903      0.999999314      -0.7
   838       1424      1.000109105     109.1
   864       1381      1.000001829       1.8

A good HZ value would make ntpd significantly happier, if the crystal is
of reasonable quality.

152ppm (1000Hz) is 13 seconds a day,
0.7 ppm (627Hz) is 22 seconds a year.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Martin J. Bligh]

>> > The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
>> > and is divided by 12 to get PIT tick rate
>> > 
>> > 	14.3181818 MHz / 12 = 1193182 Hz
>> > 
>> > The reality is that the crystal is usually off by 50-100 ppm from the
>> > standard value, depending on temperature.
>> > 
>> >     HZ   ticks/jiffie  1 second      error (ppm)
>> > ---------------------------------------------------
>> >    100      11932      1.000015238      15.2
>> >    200       5966      1.000015238      15.2
>> >    250       4773      1.000057143      57.1
>> >    300       3977      0.999931429     -68.6
>> >    333       3583      0.999964114     -35.9
>> >    500       2386      0.999847619    -152.4
>> >   1000       1193      0.999847619    -152.4
>> > 
>> > Some HZ values indeed fit the tick frequency better than others, up to
>> > 333 the error is lost in the physical error of the crystal, for 500 and
>> > 1000, it definitely is larger, and thus noticeable.
>> > 
>> > Some (less round and nice) values of HZ would fit even better:
>> > 
>> >     HZ   ticks/jiffie  1 second      error (ppm)
>> > ---------------------------------------------------
>> >     82      14551      1.000000152       0.2
>> 
>> 
>> Most interesting... Would 838 Hz be a much better choice than 1000 then? 
>> (apart from the ugliness).
> 
> No, 838 isn't significantly better. 864 and 627 would be better
> candidates:
> 
>     HZ   ticks/jiffie  1 second      error (ppm)
> ---------------------------------------------------
>    627       1903      0.999999314      -0.7
>    838       1424      1.000109105     109.1
>    864       1381      1.000001829       1.8
> 
> A good HZ value would make ntpd significantly happier, if the crystal is
> of reasonable quality.
> 
> 152ppm (1000Hz) is 13 seconds a day,
> 0.7 ppm (627Hz) is 22 seconds a year.

Does positive vs negative error make a difference to the timer subsystem?
Nish was telling me they had to add 1 extra tick to timer inaccuracies
because of the errors ... does changing the polarity of the error 
affect that (seems like it would ... but I got lost by now ;-))?

M.

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[john stultz]

On Tue, 2005-07-12 at 08:26 -0700, Martin J. Bligh wrote:
> >> > The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> >> > and is divided by 12 to get PIT tick rate
> >> > 
> >> > 	14.3181818 MHz / 12 = 1193182 Hz
> >> > 
> >> > The reality is that the crystal is usually off by 50-100 ppm from the
> >> > standard value, depending on temperature.
> >> > 
> >> >     HZ   ticks/jiffie  1 second      error (ppm)
> >> > ---------------------------------------------------
> >> >    100      11932      1.000015238      15.2
> >> >    200       5966      1.000015238      15.2
> >> >    250       4773      1.000057143      57.1
> >> >    300       3977      0.999931429     -68.6
> >> >    333       3583      0.999964114     -35.9
> >> >    500       2386      0.999847619    -152.4
> >> >   1000       1193      0.999847619    -152.4
> >> > 
> >> > Some HZ values indeed fit the tick frequency better than others, up to
> >> > 333 the error is lost in the physical error of the crystal, for 500 and
> >> > 1000, it definitely is larger, and thus noticeable.
> >> > 
> >> > Some (less round and nice) values of HZ would fit even better:
> >> > 
> >> >     HZ   ticks/jiffie  1 second      error (ppm)
> >> > ---------------------------------------------------
> >> >     82      14551      1.000000152       0.2
> >> 
> >> 
> >> Most interesting... Would 838 Hz be a much better choice than 1000 then? 
> >> (apart from the ugliness).
> > 
> > No, 838 isn't significantly better. 864 and 627 would be better
> > candidates:
> > 
> >     HZ   ticks/jiffie  1 second      error (ppm)
> > ---------------------------------------------------
> >    627       1903      0.999999314      -0.7
> >    838       1424      1.000109105     109.1
> >    864       1381      1.000001829       1.8
> > 
> > A good HZ value would make ntpd significantly happier, if the crystal is
> > of reasonable quality.
> > 
> > 152ppm (1000Hz) is 13 seconds a day,
> > 0.7 ppm (627Hz) is 22 seconds a year.
> 
> Does positive vs negative error make a difference to the timer subsystem?
> Nish was telling me they had to add 1 extra tick to timer inaccuracies
> because of the errors ... does changing the polarity of the error 
> affect that (seems like it would ... but I got lost by now ;-))?

It doesn't affect the soft-timer subsystem very much due to the
additional rounding described above, but it does affect anywhere jiffies
is used directly via HZ for time. Awhile back there were some issues we
had with proc output being confused in the transition via HZ, ACTHZ and
USER_HZ that were related.

This would probably be a good plug for Nish's time based (as opposed to
tick based) soft-timer work. By utilizing the timekeeping subsystem,
using absolute nanoseconds since boot instead of jiffies to expire soft-
timers we can avoid worrying about HZ/ACTHZ error in that subsystem.
Additionally it avoids any sort of accumulating error which could be
caused by lost ticks and allows for lower best-case latencies and
improved average latencies.

This helps pave the way for tickless/dynamic HZ systems by removing the
already broken assumption that ticks are regular and periodic.

Granted, the HZ/ACTHZ issue still exist, but the impact is more
isolated.

thanks
-john




 

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Nishanth Aravamudan]

On 12.07.2005 [10:50:23 -0700], john stultz wrote:
> On Tue, 2005-07-12 at 08:26 -0700, Martin J. Bligh wrote:
> > >> > The PIT crystal runs at 14.3181818 MHz (CGA dotclock, found on ISA, ...)
> > >> > and is divided by 12 to get PIT tick rate
> > >> > 
> > >> > 	14.3181818 MHz / 12 = 1193182 Hz
> > >> > 
> > >> > The reality is that the crystal is usually off by 50-100 ppm from the
> > >> > standard value, depending on temperature.
> > >> > 
> > >> >     HZ   ticks/jiffie  1 second      error (ppm)
> > >> > ---------------------------------------------------
> > >> >    100      11932      1.000015238      15.2
> > >> >    200       5966      1.000015238      15.2
> > >> >    250       4773      1.000057143      57.1
> > >> >    300       3977      0.999931429     -68.6
> > >> >    333       3583      0.999964114     -35.9
> > >> >    500       2386      0.999847619    -152.4
> > >> >   1000       1193      0.999847619    -152.4
> > >> > 
> > >> > Some HZ values indeed fit the tick frequency better than others, up to
> > >> > 333 the error is lost in the physical error of the crystal, for 500 and
> > >> > 1000, it definitely is larger, and thus noticeable.
> > >> > 
> > >> > Some (less round and nice) values of HZ would fit even better:
> > >> > 
> > >> >     HZ   ticks/jiffie  1 second      error (ppm)
> > >> > ---------------------------------------------------
> > >> >     82      14551      1.000000152       0.2
> > >> 
> > >> 
> > >> Most interesting... Would 838 Hz be a much better choice than 1000 then? 
> > >> (apart from the ugliness).
> > > 
> > > No, 838 isn't significantly better. 864 and 627 would be better
> > > candidates:
> > > 
> > >     HZ   ticks/jiffie  1 second      error (ppm)
> > > ---------------------------------------------------
> > >    627       1903      0.999999314      -0.7
> > >    838       1424      1.000109105     109.1
> > >    864       1381      1.000001829       1.8
> > > 
> > > A good HZ value would make ntpd significantly happier, if the crystal is
> > > of reasonable quality.
> > > 
> > > 152ppm (1000Hz) is 13 seconds a day,
> > > 0.7 ppm (627Hz) is 22 seconds a year.
> > 
> > Does positive vs negative error make a difference to the timer subsystem?
> > Nish was telling me they had to add 1 extra tick to timer inaccuracies
> > because of the errors ... does changing the polarity of the error 
> > affect that (seems like it would ... but I got lost by now ;-))?
> 
> It doesn't affect the soft-timer subsystem very much due to the
> additional rounding described above, but it does affect anywhere jiffies
> is used directly via HZ for time. Awhile back there were some issues we
> had with proc output being confused in the transition via HZ, ACTHZ and
> USER_HZ that were related.
> 
> This would probably be a good plug for Nish's time based (as opposed to
> tick based) soft-timer work. By utilizing the timekeeping subsystem,
> using absolute nanoseconds since boot instead of jiffies to expire soft-
> timers we can avoid worrying about HZ/ACTHZ error in that subsystem.
> Additionally it avoids any sort of accumulating error which could be
> caused by lost ticks and allows for lower best-case latencies and
> improved average latencies.

FWIW, I will be trying to get a new version of my patch which is
independent of John's timeofday rework (will use xtime &
wall_to_monotonic as a nanosecond "time" value) out before the end of
the week.

I think these arguments are still valid for the worst-case scenario with
my patch, as the hardware interrupt rate is still tied to HZ. But, at
least, we can discuss it in clearer terms and disjoin the soft-timer
issues from the hardware ones (I hope).

Thanks,
Nish

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[john stultz]

On Mon, 2005-07-11 at 17:38 -0700, George Anzinger wrote:
> Martin J. Bligh wrote:
> >>>Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
> >>>stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
> >>>ps).
> >>>
> >>
> >>Yes, that's called "progress" so no one complained.  Going back is
> >>called a "regression".  People don't like those as much.
> > 
> > 
> > That's a very subjective viewpoint. Realize that this is a balancing
> > act between latency and overhead ... and you're firmly only looking
> > at one side of the argument, instead of taking a compromise in the
> > middle ...
> > 
> > If I start arguing for 100HZ on the grounds that it's much more efficient,
> > will that make 250/300 look much better to you? ;-)
> 
> I would like to interject an addition data point, and I will NOT be subjective. 
>   The nature of the PIT is that it can _hit_ some frequencies better than 
> others.  We have had complaints about repeating timers not keeping good time. 
> These are not jitter issues, but drift issues.  The standard says we may not 
> return early from a timer so any timer will either be on time or late.  The 
> amount of lateness depends very much on the HZ value.  Here is what the values 
> are for the standard CLOCK_TICK_RATE:
> 
> HZ  	TICK RATE	jiffie(ns)	second(ns)	 error (ppbillion)
>   100	 1193180	10000000	1000000000	       0
>   200	 1193180	 5000098	1000019600	   19600
>   250	 1193180	 4000250	1000062500	   62500
>   500	 1193180	 1999703	1001851203	 1851203
> 1000	 1193180	  999848	1000847848	  847848
> 
> The jiffie values here are exactly what the kernel uses and are based on the 
> best one can do with the PIT hardware.
> 
> I am not suggesting any given default HZ, but rather an argumentation of the 
> help text that goes with it.  For those who want timers to repeat at one second 
> (or multiples there of) this is useful info.
> 
> For you enjoyment I have attached the program used to print this.  It allows you 
> to try additional values...

If I recall, 1001 was a decent choice and is relatively close the the
expected frequency. Also I think the error is positive instead of
negative, so it avoids the "jiffies are shorter then I expected!"
issues.

>From your program's output:
HZ      TICK RATE       jiffie(ns)      second(ns)       error (ppbillion)
1001     1193180          999013        1000012013         12013

thanks
-john

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Lee Revell]

On Mon, 2005-07-11 at 11:38 -0700, Martin J. Bligh wrote:
> >> Lots of people have switched from 2.4 to 2.6 (100 Hz to 1000 Hz) with no impact in
> >> stability, AFAIK. (I only remember some weird warning about HZ with debian woody's
> >> ps).
> >> 
> > 
> > Yes, that's called "progress" so no one complained.  Going back is
> > called a "regression".  People don't like those as much.
> 
> That's a very subjective viewpoint. Realize that this is a balancing
> act between latency and overhead ... and you're firmly only looking
> at one side of the argument, instead of taking a comprimise in the
> middle ...

I won't deny this for a second, I'm 100% arguing from the POV of one of
Alan's 'multimedia weenies'.  I do think that if you propose a big
change the burden of proof is on you to justify it, ideally with some
hard numbers.

Lee

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Alan Cox]

On Gwe, 2005-07-08 at 22:59, Andrew Morton wrote:
> Chris Wedgwood <cw@f00f.org> wrote:
> >
> > On Thu, Jun 23, 2005 at 11:28:47AM -0700, Linux Kernel Mailing List wrote:
>           ^^^^^^
> 
> It's been over two weeks and nobody has complained about anything.

Then your mail system is faulty because I did. 1000Hz is good for
multmedia, 
100Hz is good for power management/older boxes, 250Hz is too fast for
some laptop APM to avoid clock slew, too fast for power saving and too
slow for some multimedia weenies.

RE: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Brown, Len]

>Of course using APIC internal timers is generally the best idea on SMP,
>but they may have had reasons to avoid them (it's not an ISA interrupt,
so
>it could have been simply out of question in the initial design).

Best?  No.

Local APIC timers are based on a clock which on many processors will
STOP when the processor enters power saving idle states, such as C3.
So the LAPIC timer will not accurately reflect how much time
has passed across entry/exit from idle.

True, this has not been an issue on SMP, since there have not been
SMP systems shipping with C3, but as you know soon everything
interesting will be SMP...

-Len

RE: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Thu, 14 Jul 2005, Brown, Len wrote:

> >Of course using APIC internal timers is generally the best idea on SMP,
> >but they may have had reasons to avoid them (it's not an ISA interrupt,
> so
> >it could have been simply out of question in the initial design).
> 
> Best?  No.
> 
> Local APIC timers are based on a clock which on many processors will
> STOP when the processor enters power saving idle states, such as C3.
> So the LAPIC timer will not accurately reflect how much time
> has passed across entry/exit from idle.

 That's an APIC bug.  When Intel originally released the APIC (some 
thirteen years ago) they stated it should be used as a source of the timer 
interrupt instead of the 8254.  There is no excuse for changing the 
behaviour after so many years.  So if you are on a broken system, you may 
want to work around the bug, but it shouldn't impact good systems.

  Maciej

RE: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Brown, Len]

>That's an APIC bug.
>When Intel originally released the APIC (some
>thirteen years ago) they stated it should be used as a source of the
timer
>interrupt instead of the 8254.  There is no excuse for changing the
>behaviour after so many years.  So if you are on a broken system, you
may
>want to work around the bug, but it shouldn't impact good systems.


I'm perfectly happy having Linux optimize itself for the hardware
that it is running on.  However, the (harsh, I know) reality is that
systems with a reliable LAPIC timer in the face of C3 do not exist
today, and probably never will.  (don't shoot me, it wasn't my design
decision, I'm just the messenger:-)  Further, I expect that power saving
features, such as C3, will become more important and deployed more
widely in the future, rather than less widely.

So, the 13-year-old design advice will continue to apply to
13-year-old systems, but newer systems with C3 and HPET
should be using them.

-Len

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Chris Wedgwood]

On Fri, Jul 15, 2005 at 12:33:15PM -0400, Brown, Len wrote:

> So, the 13-year-old design advice will continue to apply to
> 13-year-old systems, but newer systems with C3 and HPET
> should be using them.

Last I looked HPET isn't everywhere yet (absent from nforce4
mainboards for example, but that might be a linux issue as I was told
window can see one).

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andi Kleen]

"Brown, Len" <len.brown@intel.com> writes:

> >That's an APIC bug.
> >When Intel originally released the APIC (some
> >thirteen years ago) they stated it should be used as a source of the
> timer
> >interrupt instead of the 8254.  There is no excuse for changing the
> >behaviour after so many years.  So if you are on a broken system, you
> may
> >want to work around the bug, but it shouldn't impact good systems.
> 
> 
> I'm perfectly happy having Linux optimize itself for the hardware
> that it is running on.  However, the (harsh, I know) reality is that
> systems with a reliable LAPIC timer in the face of C3 do not exist
> today, and probably never will.  (don't shoot me, it wasn't my design
> decision, I'm just the messenger:-)  Further, I expect that power saving
> features, such as C3, will become more important and deployed more
> widely in the future, rather than less widely.

At least on multi processor systems LAPIC has to work anyways (otherwise
you cannot schedule other CPUs), so it is fine to use there.

AFAIK there are no x86 CPUs right now that do both C3
and SMP. If they ever do then they will need to keep the
LAPIC ticking in C3.

This has nothing even to do with advanced power saving,
but is pretty much a hard requirement for Linux (and I would
be surprise if it wasn't one for other OS too). Without it
scheduling and local timers on APs will not work at all.

In theory it could be replaced with HPET if HPET had enough banks (one
for each CPU - most implementations today usually only have 2 or 4), but
that would severly limit scalability for lazy tick schemes because
they would depend on a common resource in the southbridge. Also the
max number of banks needed on a big system would be huge
(128? 256?) because you couldn't have more CPUs than that.

With PIC only it's absolutely impossible.

> 
> So, the 13-year-old design advice will continue to apply to
> 13-year-old systems, but newer systems with C3 and HPET
> should be using them.

Problem is that many systems even though they have HPET in the
hardware don't advertise it in ACPI because Windows doesn't use it
yet. AndLinux can't use it then neither because it doesn't know
where the registers are located (and guessing is too risky)  

This means they are stuck with the old PIC, which makes lazy ticks
very unpleasant. I think this is a big problem. We cannot wait
until Redmond brings their new release out with this.

-Andi

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Venkatesh Pallipadi]

On Fri, Jul 15, 2005 at 07:02:24PM +0200, Andi Kleen wrote:
> 
> At least on multi processor systems LAPIC has to work anyways (otherwise
> you cannot schedule other CPUs), so it is fine to use there.
> 
> AFAIK there are no x86 CPUs right now that do both C3
> and SMP. If they ever do then they will need to keep the
> LAPIC ticking in C3.
> 
> This has nothing even to do with advanced power saving,
> but is pretty much a hard requirement for Linux (and I would
> be surprise if it wasn't one for other OS too). Without it
> scheduling and local timers on APs will not work at all.
> 
> In theory it could be replaced with HPET if HPET had enough banks (one
> for each CPU - most implementations today usually only have 2 or 4), but
> that would severly limit scalability for lazy tick schemes because
> they would depend on a common resource in the southbridge. Also the
> max number of banks needed on a big system would be huge
> (128? 256?) because you couldn't have more CPUs than that.
> 
> With PIC only it's absolutely impossible.
> 

I wouldn't say it is totally impossible. There are ways in which Linux can work
without a reliable Local APIC timer. One option being - make one CPU that gets 
the external timer interrupt multicast an IPI to all the other CPUs that
wants to get periodic timer interrupt.

Thanks,
Venki

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Fri, 15 Jul 2005, Venkatesh Pallipadi wrote:

> I wouldn't say it is totally impossible. There are ways in which Linux can work
> without a reliable Local APIC timer. One option being - make one CPU that gets 
> the external timer interrupt multicast an IPI to all the other CPUs that
> wants to get periodic timer interrupt.

 That's like scratching your left ear with your right hand -- broadcasting 
that external timer interrupt in the first place is more straightforward.  
If you want to exclude CPUs from the list of receivers, just use the 
logical destination mode appropriately.

  Maciej

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andi Kleen]

>  That's like scratching your left ear with your right hand -- broadcasting 
> that external timer interrupt in the first place is more straightforward.  
> If you want to exclude CPUs from the list of receivers, just use the 
> logical destination mode appropriately.

The problem with that is that it would need regular synchronizations
of all CPUs to coordinate this.   Not good for scalability and I 
believe the fundamentally wrong way to do this.

-Andi

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Fri, 15 Jul 2005, Andi Kleen wrote:

> >  That's like scratching your left ear with your right hand -- broadcasting 
> > that external timer interrupt in the first place is more straightforward.  
> > If you want to exclude CPUs from the list of receivers, just use the 
> > logical destination mode appropriately.
> 
> The problem with that is that it would need regular synchronizations
> of all CPUs to coordinate this.   Not good for scalability and I 
> believe the fundamentally wrong way to do this.

 What to you mean by "regular synchronizations of all CPUs?"  And how is a 
broadcasted external timer interrupt different from a unicasted one 
redistributed further via an all-but-self IPI, except from removing an 
unnecessary burden from the CPU targeted by the unicast interrupt?

  Maciej

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Venkatesh Pallipadi]

On Fri, Jul 15, 2005 at 06:54:30PM +0100, Maciej W. Rozycki wrote:
> On Fri, 15 Jul 2005, Venkatesh Pallipadi wrote:
> 
> > I wouldn't say it is totally impossible. There are ways in which Linux can work
> > without a reliable Local APIC timer. One option being - make one CPU that gets 
> > the external timer interrupt multicast an IPI to all the other CPUs that
> > wants to get periodic timer interrupt.
> 
>  That's like scratching your left ear with your right hand -- broadcasting 
> that external timer interrupt in the first place is more straightforward.  
> If you want to exclude CPUs from the list of receivers, just use the 
> logical destination mode appropriately.
> 

Well.. I tried a patch to do the broadcast thing couple of months ago and 
failed to convince everyone :(.

Further, it doesn't work well if you want to exclude some CPUs from the list of recievers. Logical destination is simple only for less than 8 CPUs. Beyond that
with clustered or physical configuration it is difficult.

Thanks,
Venki

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Maciej W. Rozycki]

On Fri, 15 Jul 2005, Venkatesh Pallipadi wrote:

> Well.. I tried a patch to do the broadcast thing couple of months ago and 
> failed to convince everyone :(.

 I must have missed the patch -- but was the change unconditional or 
affecting only broken systems?  And how such systems were determined?

> Further, it doesn't work well if you want to exclude some CPUs from the 
> list of recievers. Logical destination is simple only for less than 8 
> CPUs. Beyond that with clustered or physical configuration it is 
> difficult.

 Well, I've thought the number of bits for LDR has been reexpanded at one 
point (it had been 32 originally with the 82489DX and then shrank to 8 
with the Pentium integrated APIC) -- it must have been something else...

 Anyway, for this you should just need a single bit as, quoting APIC 
documentation:

 "When the message addresses the destination using logical addressing 
scheme each Local Unit in the ICC bus compares the logical address in the 
interrupt message with its own Logical Destination Register.  If there is 
a bit match (i.e., if at least one of the corresponding pair of bits 
match) this local unit is selected for delivery."

Thus you could make, say, bit 31 of LDR the "timer bit", set it in all 
local APIC units and send timer interrupts in the Fixed delivery mode 
using a logical destination address with only bit 31 set.  You could then 
exclude processors from delivery by clearing bit 31 of LDR as needed.  It 
could probably be applied to addresses within clusters, too, though space 
there is a bit tight.

 None of such hassle should be needed in reality, though -- I presume the 
only broken systems are uniprocessor ones with the HT feature as real SMP 
systems need to have their APICs enabled all the time to be able to accept 
IPIs if nothing else.  As UP systems with HT have only two processors 
logically, there is no problem with using the logical destination mode as 
currently implemented.

  Maciej

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Andi Kleen]

> I wouldn't say it is totally impossible. There are ways in which Linux can work
> without a reliable Local APIC timer. One option being - make one CPU that gets 
> the external timer interrupt multicast an IPI to all the other CPUs that
> wants to get periodic timer interrupt.

That doesn't mix very well with variable ticks. And I believe
we really need them.

For no tick in idle you need a timer for each CPU that
can be programmed to a reasonably long interval to wake you
up after longer idleness. And all that
should work without bouncing cache lines around all the time
because that doesn't work on larger systems.

-Andi

Re: [PATCH] i386: Selectable Frequency of the Timer Interrupt

[Venkatesh Pallipadi]

On Fri, Jul 15, 2005 at 07:57:01PM +0200, Andi Kleen wrote:
> > I wouldn't say it is totally impossible. There are ways in which Linux can work
> > without a reliable Local APIC timer. One option being - make one CPU that gets 
> > the external timer interrupt multicast an IPI to all the other CPUs that
> > wants to get periodic timer interrupt.
> 
> That doesn't mix very well with variable ticks. And I believe
> we really need them.

It should work with variable ticks as we can easily add/remove CPUs from this
multicast destination.
  
> For no tick in idle you need a timer for each CPU that
> can be programmed to a reasonably long interval to wake you
> up after longer idleness. And all that
> should work without bouncing cache lines around all the time
> because that doesn't work on larger systems.

And each CPU timer itself does very little writes in terms of caches. The
two things that happen here are scheduler_tick and kstat_accounting. 
Or am I missing something? Did you have anything specific in mind wrt 
bouncing cachelines?

Thanks,
Venki