Hungry for hardware timers

Hungry for hardware timers

[Jaswinder Singh Rajput]

Hello,

I am investigating how many hardware timers are available for kernel,
system applications and user applications for x86 platforms.
Is there any help available for it so that systems developers can use
a hook up an interrupt and use any hardware timer.

In APIC timer, but each core is having only one timer and it is
already utilized by Linux :
... APIC TMICT: 00002078
... APIC TMCCT: 00000b5f
... APIC TDCR: 00000003

HPET have 3 timers :
[    0.328157] hpet: ID: 0x8086a201, PERIOD: 0x429b17f
[    0.328315] hpet: CFG: 0x3, STATUS: 0x0
[    0.328472] hpet: COUNTER_l: 0x6ff120, COUNTER_h: 0x0
[    0.329006] hpet: T0: CFG_l: 0x138, CFG_h: 0xf00000
[    0.329165] hpet: T0: CMP_l: 0x701baa, CMP_h: 0x0
[    0.329324] hpet: T0 ROUTE_l: 0x0, ROUTE_h: 0x0
[    0.329483] hpet: T1: CFG_l: 0x0, CFG_h: 0xf00000
[    0.330006] hpet: T1: CMP_l: 0xffffffff, CMP_h: 0x0
[    0.330166] hpet: T1 ROUTE_l: 0x0, ROUTE_h: 0x0
[    0.331005] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
[    0.331168] hpet: T2: CMP_l: 0xdf751c, CMP_h: 0x0
[    0.331328] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0

T0 and T2 is already used by Linux and T1 is used for RTC

System 8254 timer have 3 timers but it seems it is also used by Linux :
[ 4923.510233] 0: 83ae 1: 1102 2: 37f2
[ 4923.510251] 0: 8382 1: d10 2: 37dc

So I am curious why Linux kernel is consuming so many timers. And how
can I get spare hardware timers.

Thanks,
--
Jaswinder Singh.

Re: Hungry for hardware timers

[Clemens Ladisch]

Jaswinder Singh Rajput wrote:
> I am investigating how many hardware timers are available for kernel,
> system applications and user applications for x86 platforms.

Why would you want to have a separate timer for your application?

> Is there any help available for it so that systems developers can use
> a hook up an interrupt and use any hardware timer.

The kernel is supposed to abstract away the hardware; just use POSIX
timers.

> In APIC timer, but each core is having only one timer and it is
> already utilized by Linux :
> ... APIC TMICT: 00002078
> ... APIC TMCCT: 00000b5f
> ... APIC TDCR: 00000003

Having per-CPU timers allows the kernel to avoid synchronizing between
CPUs.

> HPET have 3 timers :
> [    0.328157] hpet: ID: 0x8086a201, PERIOD: 0x429b17f
> [    0.328315] hpet: CFG: 0x3, STATUS: 0x0
> [    0.328472] hpet: COUNTER_l: 0x6ff120, COUNTER_h: 0x0
> [    0.329006] hpet: T0: CFG_l: 0x138, CFG_h: 0xf00000
> [    0.329165] hpet: T0: CMP_l: 0x701baa, CMP_h: 0x0
> [    0.329324] hpet: T0 ROUTE_l: 0x0, ROUTE_h: 0x0
> [    0.329483] hpet: T1: CFG_l: 0x0, CFG_h: 0xf00000
> [    0.330006] hpet: T1: CMP_l: 0xffffffff, CMP_h: 0x0
> [    0.330166] hpet: T1 ROUTE_l: 0x0, ROUTE_h: 0x0
> [    0.331005] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
> [    0.331168] hpet: T2: CMP_l: 0xdf751c, CMP_h: 0x0
> [    0.331328] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
> 
> T0 and T2 is already used by Linux and T1 is used for RTC

T2 shouldn't be used.  What does /proc/interrupts say?

> System 8254 timer have 3 timers but it seems it is also used by Linux :
> [ 4923.510233] 0: 83ae 1: 1102 2: 37f2
> [ 4923.510251] 0: 8382 1: d10 2: 37dc

That thing is horribly slow; nobody would want to use this, if possible.


Regards,
Clemens

Re: Hungry for hardware timers

[Jaswinder Singh Rajput]

Hello,

On Tue, Sep 7, 2010 at 1:29 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
> Jaswinder Singh Rajput wrote:
>> I am investigating how many hardware timers are available for kernel,
>> system applications and user applications for x86 platforms.
>
> Why would you want to have a separate timer for your application?
>

I need a programmable periodic interrupt for an embedded project.

>> Is there any help available for it so that systems developers can use
>> a hook up an interrupt and use any hardware timer.
>
> The kernel is supposed to abstract away the hardware; just use POSIX
> timers.
>
>> In APIC timer, but each core is having only one timer and it is
>> already utilized by Linux :
>> ... APIC TMICT: 00002078
>> ... APIC TMCCT: 00000b5f
>> ... APIC TDCR: 00000003
>
> Having per-CPU timers allows the kernel to avoid synchronizing between
> CPUs.
>
>> HPET have 3 timers :
>> [    0.328157] hpet: ID: 0x8086a201, PERIOD: 0x429b17f
>> [    0.328315] hpet: CFG: 0x3, STATUS: 0x0
>> [    0.328472] hpet: COUNTER_l: 0x6ff120, COUNTER_h: 0x0
>> [    0.329006] hpet: T0: CFG_l: 0x138, CFG_h: 0xf00000
>> [    0.329165] hpet: T0: CMP_l: 0x701baa, CMP_h: 0x0
>> [    0.329324] hpet: T0 ROUTE_l: 0x0, ROUTE_h: 0x0
>> [    0.329483] hpet: T1: CFG_l: 0x0, CFG_h: 0xf00000
>> [    0.330006] hpet: T1: CMP_l: 0xffffffff, CMP_h: 0x0
>> [    0.330166] hpet: T1 ROUTE_l: 0x0, ROUTE_h: 0x0
>> [    0.331005] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
>> [    0.331168] hpet: T2: CMP_l: 0xdf751c, CMP_h: 0x0
>> [    0.331328] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
>>
>> T0 and T2 is already used by Linux and T1 is used for RTC
>
> T2 shouldn't be used.

[    0.323613] hpet: hpet_msi_capability_lookup(621):
..
[    0.325323] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
[    0.325483] hpet: T2: CMP_l: 0xffffffff, CMP_h: 0x0
[    0.325639] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
[    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.326279] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
[    0.328003] hpet: hpet_late_init(951):
..
[    0.331006] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
[    0.331167] hpet: T2: CMP_l: 0xdfe12a, CMP_h: 0x0
[    0.331327] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0

As you can see after hpet_late_init(951) T2 CMP_l is changed.


> What does /proc/interrupts say?
>

$ cat /proc/interrupts
           CPU0       CPU1
  0:     192286          0   IO-APIC-edge      timer
  1:       1039          0   IO-APIC-edge      i8042
  8:         50          0   IO-APIC-edge      rtc0
  9:        637       1903   IO-APIC-fasteoi   acpi
 12:        163        515   IO-APIC-edge      i8042
 16:      21285          0   IO-APIC-fasteoi   i915, ath9k,
ehci_hcd:usb1, uhci_hcd:usb2
 17:          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
 18:          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
 19:          0          0   IO-APIC-fasteoi   uhci_hcd:usb5
 44:       3511       9569   PCI-MSI-edge      ahci
 45:        515       1569   PCI-MSI-edge      hda_intel
 46:          2          0   PCI-MSI-edge      eth0
NMI:          0          0   Non-maskable interrupts
LOC:     164087     274764   Local timer interrupts
SPU:          0          0   Spurious interrupts
PMI:          0          0   Performance monitoring interrupts
PND:          0          0   Performance pending work
RES:      56164      64218   Rescheduling interrupts
CAL:        650        484   Function call interrupts
TLB:       1165        625   TLB shootdowns
TRM:          0          0   Thermal event interrupts
THR:          0          0   Threshold APIC interrupts
MCE:          0          0   Machine check exceptions
MCP:          2          2   Machine check polls
ERR:          1
MIS:          0
$

>> System 8254 timer have 3 timers but it seems it is also used by Linux :
>> [ 4923.510233] 0: 83ae 1: 1102 2: 37f2
>> [ 4923.510251] 0: 8382 1: d10 2: 37dc
>
> That thing is horribly slow; nobody would want to use this, if possible.
>

As you seen timers are running may be due to these code :
arch/x86/kernel/apm_32.c:       outb_pit(0x34, PIT_MODE);
 /* binary, mode 2, LSB/MSB, ch 0 */
arch/x86/kernel/i8253.c:                outb_pit(0x34, PIT_MODE);
arch/x86/kernel/i8253.c:                        outb_pit(0x30, PIT_MODE);
arch/x86/kernel/i8253.c:                outb_pit(0x38, PIT_MODE);
arch/x86/kernel/i8253.c:        outb_pit(0x00, PIT_MODE);       /*
latch the count ASAP */
arch/x86/kernel/i8253.c:                outb_pit(0x34, PIT_MODE);
arch/x86/kernel/vmiclock_32.c:  outb_pit(0x3a, PIT_MODE); /* binary,
mode 5, LSB/MSB, ch 0 */

It is also used secretly :
arch/x86/include/asm/mach_timer.h:      outb(0xb0, 0x43);
         /* binary, mode 0, LSB/MSB, Ch 2 */
arch/x86/kernel/acpi/realmode/wakemain.c:               outb(0xb6,
0x43);       /* Ctr 2, squarewave, load, binary */
arch/x86/kernel/tsc.c:  outb(0xb0, 0x43);
arch/x86/kernel/tsc.c:  outb(0xb0, 0x43);


By the way why I am getting this :

[  568.301571] CE: hpet increased min_delta_ns to 7500 nsec
[  568.301736] CE: hpet increased min_delta_ns to 11250 nsec
[  568.301888] CE: hpet increased min_delta_ns to 16875 nsec

Thanks,
--
Jaswinder Singh.

Re: Hungry for hardware timers

[Clemens Ladisch]

Jaswinder Singh Rajput wrote:
> On Tue, Sep 7, 2010 at 1:29 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
>> Jaswinder Singh Rajput wrote:
>>> I am investigating how many hardware timers are available for kernel,
>>> system applications and user applications for x86 platforms.
>>
>> Why would you want to have a separate timer for your application?
> 
> I need a programmable periodic interrupt for an embedded project.

And why do you not want to use POSIX timers for this?

>>> HPET have 3 timers :
>>> T0 and T2 is already used by Linux and T1 is used for RTC
>>
>> T2 shouldn't be used.
> 
> [    0.323613] hpet: hpet_msi_capability_lookup(621):
> ..
> [    0.325323] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
> [    0.325483] hpet: T2: CMP_l: 0xffffffff, CMP_h: 0x0
> [    0.325639] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
> [    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
> [    0.326279] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
> [    0.328003] hpet: hpet_late_init(951):
> ..
> [    0.331006] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
> [    0.331167] hpet: T2: CMP_l: 0xdfe12a, CMP_h: 0x0
> [    0.331327] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
> 
> As you can see after hpet_late_init(951) T2 CMP_l is changed.

Strange; nobody should be accessing this.
Do you need to use it before hpet_late_init?

>> What does /proc/interrupts say?
> 
> $ cat /proc/interrupts
>            CPU0       CPU1
>   0:     192286          0   IO-APIC-edge      timer
>   1:       1039          0   IO-APIC-edge      i8042
>   8:         50          0   IO-APIC-edge      rtc0
>   9:        637       1903   IO-APIC-fasteoi   acpi
>  12:        163        515   IO-APIC-edge      i8042
>  16:      21285          0   IO-APIC-fasteoi   i915, ath9k, ehci_hcd:usb1, uhci_hcd:usb2
>  17:          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>  18:          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>  19:          0          0   IO-APIC-fasteoi   uhci_hcd:usb5
>  44:       3511       9569   PCI-MSI-edge      ahci
>  45:        515       1569   PCI-MSI-edge      hda_intel
>  46:          2          0   PCI-MSI-edge      eth0

Nobody is using /dev/hpet, you should be able to grab it for T2.

> By the way why I am getting this :
> 
> [  568.301571] CE: hpet increased min_delta_ns to 7500 nsec
> [  568.301736] CE: hpet increased min_delta_ns to 11250 nsec
> [  568.301888] CE: hpet increased min_delta_ns to 16875 nsec

On my AMD machine, this happens only if I have C1E enabled, and only up
to 11250 ns.  It seems your BIOS has installed a SMI.


Regards,
Clemens

Re: Hungry for hardware timers

[Jaswinder Singh Rajput]

Hello,

On Tue, Sep 7, 2010 at 2:35 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
> Jaswinder Singh Rajput wrote:
>> On Tue, Sep 7, 2010 at 1:29 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
>>> Jaswinder Singh Rajput wrote:
>>>> I am investigating how many hardware timers are available for kernel,
>>>> system applications and user applications for x86 platforms.
>>>
>>> Why would you want to have a separate timer for your application?
>>
>> I need a programmable periodic interrupt for an embedded project.
>
> And why do you not want to use POSIX timers for this?
>

OK, I will try POSIX timers for the application but for system module
my preference is HPET T2.

>>>> HPET have 3 timers :
>>>> T0 and T2 is already used by Linux and T1 is used for RTC
>>>
>>> T2 shouldn't be used.
>>
>> [    0.323613] hpet: hpet_msi_capability_lookup(621):
>> ..
>> [    0.325323] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
>> [    0.325483] hpet: T2: CMP_l: 0xffffffff, CMP_h: 0x0
>> [    0.325639] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
>> [    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
>> [    0.326279] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
>> [    0.328003] hpet: hpet_late_init(951):
>> ..
>> [    0.331006] hpet: T2: CFG_l: 0x0, CFG_h: 0xf00800
>> [    0.331167] hpet: T2: CMP_l: 0xdfe12a, CMP_h: 0x0
>> [    0.331327] hpet: T2 ROUTE_l: 0x0, ROUTE_h: 0x0
>>
>> As you can see after hpet_late_init(951) T2 CMP_l is changed.
>
> Strange; nobody should be accessing this.

But it is changing in vanilla kernel.

> Do you need to use it before hpet_late_init?
>

No.

>>> What does /proc/interrupts say?
>>
>> $ cat /proc/interrupts
>>            CPU0       CPU1
>>   0:     192286          0   IO-APIC-edge      timer
>>   1:       1039          0   IO-APIC-edge      i8042
>>   8:         50          0   IO-APIC-edge      rtc0
>>   9:        637       1903   IO-APIC-fasteoi   acpi
>>  12:        163        515   IO-APIC-edge      i8042
>>  16:      21285          0   IO-APIC-fasteoi   i915, ath9k, ehci_hcd:usb1, uhci_hcd:usb2
>>  17:          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
>>  18:          0          0   IO-APIC-fasteoi   uhci_hcd:usb4
>>  19:          0          0   IO-APIC-fasteoi   uhci_hcd:usb5
>>  44:       3511       9569   PCI-MSI-edge      ahci
>>  45:        515       1569   PCI-MSI-edge      hda_intel
>>  46:          2          0   PCI-MSI-edge      eth0
>
> Nobody is using /dev/hpet, you should be able to grab it for T2.
>

OK. What will be IRQ # for T2.

If I access /dev/hpet by hpet_example it shows that it is using HPET
T0 as hi_hpet is 0 :

#./hpet_example info /dev/hpet
-hpet: executing info
hpet_info: hi_irqfreq 0x0 hi_flags 0x0 hi_hpet 0 hi_timer 2

As per dmesg([    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0)
so IRQs are like this :
HPET T0 : 2
HPET T1 : 8
HPET T2 : 0

Is it right, or I am missing something.

Thanks,
--
Jaswinder Singh.

Re: Hungry for hardware timers

[Clemens Ladisch]

Jaswinder Singh Rajput wrote:
> If I access /dev/hpet by hpet_example it shows that it is using HPET
> T0 as hi_hpet is 0 :
> 
> #./hpet_example info /dev/hpet
> -hpet: executing info
> hpet_info: hi_irqfreq 0x0 hi_flags 0x0 hi_hpet 0 hi_timer 2

This means HPET block 0, timer 2.  In practice, there is no machine with
more than one HPET block, so hi_hpet is rather useless.

> What will be IRQ # for T2.

Just check while hpet_example is running ...

> As per dmesg([    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0)
> so IRQs are like this :
> HPET T0 : 2
> HPET T1 : 8
> HPET T2 : 0

"0" means that the BIOS did not initialize it.  In this case, the driver
will initializes it when accessed, with the first supported IRQ above 15.
On your HPET's T2, this is IRQ 20.


Regards,
Clemens

Re: Hungry for hardware timers

[Jaswinder Singh Rajput]

Hello,

Sorry for delay.

On Tue, Sep 7, 2010 at 4:25 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
> Jaswinder Singh Rajput wrote:
>
>> What will be IRQ # for T2.
>
> Just check while hpet_example is running ...
>

$ cat /proc/interrupts | grep hpet
 20:         24          0   IO-APIC-fasteoi   hpet2

I am planning to use hpet2 from modules. But currently it seems
hpet_readl() and hpet_writel() are not accessible from outside the
kernel. Is there any plans to export them.

Thanks,
--
Jaswinder Singh.

Re: Hungry for hardware timers

[Clemens Ladisch]

Jaswinder Singh Rajput wrote:
> I am planning to use hpet2 from modules. But currently it seems
> hpet_readl() and hpet_writel() are not accessible from outside the
> kernel. Is there any plans to export them.

hpet_ioctl_common was intended to be exported, but isn't.

Better use hrtimers.


Regards,
Clemens

Re: Hungry for hardware timers

[Jaswinder Singh]

Hello,

On Tue, Sep 7, 2010 at 4:25 PM, Clemens Ladisch <clemens@ladisch.de> wrote:
> Jaswinder Singh Rajput wrote:
>> As per dmesg([    0.325958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0)
>> so IRQs are like this :
>> HPET T0 : 2
>> HPET T1 : 8
>> HPET T2 : 0
>
> "0" means that the BIOS did not initialize it.  In this case, the driver
> will initializes it when accessed, with the first supported IRQ above 15.
> On your HPET's T2, this is IRQ 20.
>

I have used /dev/hpet as periodic interrupt and it works great but
settings for HPET_T2_CMP will be like this :

10 micro-seconds :  110 /* 0.00001 sec  */
100 micro-seconds : 1230 /* 0.0001 sec   */
1 milli-second : 13800 /* 0.001 sec    */
10 milli-seconds : 142580 /* 0.01 sec     */
100 milli-seconds : 1431150 /* 0.1 sec      */
1 second : 14317330 /* 1 sec        */
10 seconds : 143179330 /* 10 sec       */
100 seconds : 1431799400 /* 100 sec      */

I do not have a oscilloscope so I am measuring it by using printks.
I am curious, Why it is irregular.

I am looking for one more timer interrupt handler, how can I get one
more interrupt handler for user.

Thanks,
--
Jaswinder Singh.