* [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
@ 2012-07-12 19:54 Stephan Kappertz
2012-07-12 20:30 ` Gilles Chanteperdrix
2012-07-13 9:00 ` Gilles Chanteperdrix
0 siblings, 2 replies; 8+ messages in thread
From: Stephan Kappertz @ 2012-07-12 19:54 UTC (permalink / raw)
To: xenomai
Hi,
I am trying to use Xenomai 2.6.1 on a beaglebone (AM3359 Cortex A8 SoC) running Linux 3.2.21. As the TI AM3359 is not supported in 3.2.21 mainline kernel, I am using the slightly patched kernel from linux-omap. The adeos patch applied without errors after minor fixups.
Building the kernel with or without Xenomai enabled succeeds.
Now the adeos patched kernel boots fine when disabling Xenomai & ipipe in the Kconfig configuration.
However, after enabling Xenomai & ipipe it hangs in an endless loop right after starting the Interrupt pipeline. Here's the bootlog:
Uncompressing Linux... done, booting the kernel.
Linux version 3.2.21 (stephan@ubuntu) (gcc version 4.6.2 (Buildroot 2011.11) ) #12 PREEMPT Thu Jul 12 21:08:26 CEST 22
CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: am335xevm
Ignoring tag cmdline (using the default kernel command line)
bootconsole [earlycon0] enabled
Memory policy: ECC disabled, Data cache writeback
AM335X ES1.0 (sgx neon )
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 65024
Kernel command line: console=ttyO0,115200n8 mem=256M earlyprintk=ttyO0 lpj=2494464
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 256MB = 256MB total
Memory: 250944k/250944k available, 11200k reserved, 0K highmem
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
vmalloc : 0xd0800000 - 0xff000000 ( 744 MB)
lowmem : 0xc0000000 - 0xd0000000 ( 256 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc03e3000 (3948 kB)
.init : 0xc03e3000 - 0xc083b000 (4448 kB)
.data : 0xc083c000 - 0xc087e440 ( 266 kB)
.bss : 0xc087e464 - 0xc08b3270 ( 212 kB)
NR_IRQS:396
IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
Total of 128 interrupts on 1 active controller
OMAP clockevent source: GPTIMER2 at 24000000 Hz
OMAP clocksource: GPTIMER1 at 32768 Hz
I-pipe, 0.032 MHz clocksource
sched_clock: 32 bits at 32kHz, resolution 30517ns, wraps every 131071999ms
Interrupt pipeline (release #1)
------------[ cut here ]------------
This is the last console output line. The cut here part comes from a WARN_ON_ONCE in kernel/softirq.c. This warning does not show up with ipipe disabled:
static void __local_bh_enable(unsigned int cnt)
{
WARN_ON_ONCE(in_irq());
WARN_ON_ONCE(!irqs_disabled()); <--- This warning asserts
if (softirq_count() == cnt)
trace_softirqs_on((unsigned long)__builtin_return_address(0));
sub_preempt_count(cnt);
}
Using a JTAG interface and gdb I see that the kernel didn't crash but the timer IRQ is constantly firing and the timer .ack handler is never called. When I CTRL^C break into the kernel, I always end up in one of the following functions:
handle_level_irq ()
omap3_intc_handle_irq ()
generic_handle_irq ()
irq_enter ()
irq_exit ()
And with less probability:
idle_cpu ()
irq_check_poll ()
rcu_enter_nohz ()
So far I found two things the AM3359 handles different compared to the Igepv2 board:
1) MACHINE_START has .handle_irq = omap3_intc_handle_irq while it is not defined in the ipipe-2.6 kernel repository
2) The AM3359 uses a different timer for system clock (not the default omap3 timer)
Any idea how I can dive into this issue? I have no clear understanding of the core-3.2 implementation so I'm not sure where to start.
Thanks a lot,
Stephan
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-12 19:54 [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC) Stephan Kappertz
@ 2012-07-12 20:30 ` Gilles Chanteperdrix
2012-07-12 20:47 ` Gilles Chanteperdrix
2012-07-13 9:00 ` Gilles Chanteperdrix
1 sibling, 1 reply; 8+ messages in thread
From: Gilles Chanteperdrix @ 2012-07-12 20:30 UTC (permalink / raw)
To: Stephan Kappertz; +Cc: xenomai
On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
> OMAP clockevent source: GPTIMER2 at 24000000 Hz
> OMAP clocksource: GPTIMER1 at 32768 Hz
> I-pipe, 0.032 MHz clocksource
This is wrong, normally, you should have something like:
[ 0.000000] OMAP clockevent source: GPTIMER2 at 13000000 Hz
[ 0.000000] OMAP clocksource: GPTIMER3 at 13000000 Hz
[ 0.000000] I-pipe, 13.000 MHz clocksource
This is ensured by this piece of code in arch/arm/mach-omap2/timer.c:
#ifndef CONFIG_IPIPE
OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
#else
OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
#endif
So, my guess is that you have some other patch changing this.
--
Gilles.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-12 20:30 ` Gilles Chanteperdrix
@ 2012-07-12 20:47 ` Gilles Chanteperdrix
2012-07-13 7:09 ` Stephan Kappertz
0 siblings, 1 reply; 8+ messages in thread
From: Gilles Chanteperdrix @ 2012-07-12 20:47 UTC (permalink / raw)
To: Stephan Kappertz; +Cc: xenomai
On 07/12/2012 10:30 PM, Gilles Chanteperdrix wrote:
> On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
>> OMAP clockevent source: GPTIMER2 at 24000000 Hz
>> OMAP clocksource: GPTIMER1 at 32768 Hz
>> I-pipe, 0.032 MHz clocksource
>
> This is wrong, normally, you should have something like:
> [ 0.000000] OMAP clockevent source: GPTIMER2 at 13000000 Hz
> [ 0.000000] OMAP clocksource: GPTIMER3 at 13000000 Hz
> [ 0.000000] I-pipe, 13.000 MHz clocksource
>
> This is ensured by this piece of code in arch/arm/mach-omap2/timer.c:
> #ifndef CONFIG_IPIPE
> OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
> #else
> OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
> #endif
>
> So, my guess is that you have some other patch changing this.
>
Sending your .config would help. Because you probably have
CONFIG_OMAP_32K_TIMER on, which should not be normally possible.
--
Gilles.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-12 20:47 ` Gilles Chanteperdrix
@ 2012-07-13 7:09 ` Stephan Kappertz
2012-07-13 7:32 ` Gilles Chanteperdrix
0 siblings, 1 reply; 8+ messages in thread
From: Stephan Kappertz @ 2012-07-13 7:09 UTC (permalink / raw)
To: xenomai
Hi Gilles,
the problem was not the .config file but the fact that CONFIG_OMAP_32K_TIMER was ignored for the AM3359 implementation.
So in arch/arm/mach_omap2/timer.c I changed
--------------------------------------------
#ifdef CONFIG_ARCH_OMAP3
#ifndef CONFIG_IPIPE
OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
#else
OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
#endif
OMAP_SYS_TIMER(3)
OMAP_SYS_TIMER_INIT(3_secure, OMAP3_SECURE_TIMER, OMAP3_CLKEV_SOURCE,
2, OMAP3_MPU_SOURCE)
OMAP_SYS_TIMER(3_secure)
OMAP_SYS_TIMER_INIT(3_am33xx, 2, OMAP4_MPU_SOURCE, 1, AM33XX_RTC32K_SOURCE) <--- ignoring CONFIG_OMAP_32K_TIMER here
OMAP_SYS_TIMER(3_am33xx)
#endif
--------------------------------------------
to
--------------------------------------------
#ifdef CONFIG_ARCH_OMAP3
#ifndef CONFIG_SOC_OMAPAM33XX
#ifndef CONFIG_IPIPE
OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
#else
OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
#endif
OMAP_SYS_TIMER(3)
OMAP_SYS_TIMER_INIT(3_secure, OMAP3_SECURE_TIMER, OMAP3_CLKEV_SOURCE,
2, OMAP3_MPU_SOURCE)
OMAP_SYS_TIMER(3_secure)
#else // #ifdef CONFIG_SOC_OMAPAM33XX
#ifndef CONFIG_IPIPE
OMAP_SYS_TIMER_INIT(3_am33xx, 2, AM33XX_CLKEV_SOURCE, 1, AM33XX_MPU_SOURCE)
#else
OMAP_SYS_TIMER_INIT(3_am33xx, 2, AM33XX_CLKEV_SOURCE, 3, AM33XX_MPU_SOURCE)
#endif
OMAP_SYS_TIMER(3_am33xx)
#endif
#endif
--------------------------------------------
With the AM33XX_CLKEV_SOURCE and AM33XX_MPU_SOURCE defined as
--------------------------------------------
#define OMAP4_MPU_SOURCE "sys_clkin_ck"
#define AM33XX_MPU_SOURCE OMAP4_MPU_SOURCE
#define AM33XX_RTC32K_SOURCE "clk_32768_ck"
#ifdef CONFIG_OMAP_32K_TIMER
#define AM33XX_CLKEV_SOURCE AM33XX_RTC32K_SOURCE
#define OMAP3_SECURE_TIMER 12
#else
#define AM33XX_CLKEV_SOURCE OMAP4_MPU_SOURCE
#define OMAP3_SECURE_TIMER 1
#endif
--------------------------------------------
That fixes the clock rate but otherwise does not change the fact that I'm stuck in a non ack'd interrupt at boot:
Uncompressing Linux... done, booting the kernel.
Linux version 3.2.21 (stephan@ubuntu) (gcc version 4.6.2 (Buildroot 2011.11) ) #14 PREEMPT Fri Jul 13 19:58:14 CEST 2012
CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c53c7d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: am335xevm
Ignoring tag cmdline (using the default kernel command line)
bootconsole [earlycon0] enabled
Memory policy: ECC disabled, Data cache writeback
AM335X ES1.0 (sgx neon )
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 65024
Kernel command line: console=ttyO0,115200n8 mem=256M earlyprintk=ttyO0 lpj=2494464
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 256MB = 256MB total
Memory: 250944k/250944k available, 11200k reserved, 0K highmem
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
vmalloc : 0xd0800000 - 0xff000000 ( 744 MB)
lowmem : 0xc0000000 - 0xd0000000 ( 256 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc03e3000 (3948 kB)
.init : 0xc03e3000 - 0xc083b000 (4448 kB)
.data : 0xc083c000 - 0xc087e440 ( 266 kB)
.bss : 0xc087e464 - 0xc08b3270 ( 212 kB)
NR_IRQS:396
IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
Total of 128 interrupts on 1 active controller
OMAP clockevent source: GPTIMER2 at 24000000 Hz
OMAP clocksource: GPTIMER3 at 24000000 Hz
I-pipe, 24.000 MHz clocksource
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
Interrupt pipeline (release #1)
------------[ cut here ]------------
- Stephan
Am 12.07.2012 um 22:47 schrieb Gilles Chanteperdrix:
> On 07/12/2012 10:30 PM, Gilles Chanteperdrix wrote:
>> On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
>>> OMAP clockevent source: GPTIMER2 at 24000000 Hz
>>> OMAP clocksource: GPTIMER1 at 32768 Hz
>>> I-pipe, 0.032 MHz clocksource
>>
>> This is wrong, normally, you should have something like:
>> [ 0.000000] OMAP clockevent source: GPTIMER2 at 13000000 Hz
>> [ 0.000000] OMAP clocksource: GPTIMER3 at 13000000 Hz
>> [ 0.000000] I-pipe, 13.000 MHz clocksource
>>
>> This is ensured by this piece of code in arch/arm/mach-omap2/timer.c:
>> #ifndef CONFIG_IPIPE
>> OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
>> #else
>> OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
>> #endif
>>
>> So, my guess is that you have some other patch changing this.
>>
>
> Sending your .config would help. Because you probably have
> CONFIG_OMAP_32K_TIMER on, which should not be normally possible.
>
> --
> Gilles.
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-13 7:09 ` Stephan Kappertz
@ 2012-07-13 7:32 ` Gilles Chanteperdrix
0 siblings, 0 replies; 8+ messages in thread
From: Gilles Chanteperdrix @ 2012-07-13 7:32 UTC (permalink / raw)
To: Stephan Kappertz; +Cc: xenomai
On 07/13/2012 09:09 AM, Stephan Kappertz wrote:
> Hi Gilles,
>
> the problem was not the .config file but the fact that CONFIG_OMAP_32K_TIMER was ignored for the AM3359 implementation.
>
> So in arch/arm/mach_omap2/timer.c I changed
>
> --------------------------------------------
> #ifdef CONFIG_ARCH_OMAP3
> #ifndef CONFIG_IPIPE
> OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
> #else
> OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
> #endif
> OMAP_SYS_TIMER(3)
> OMAP_SYS_TIMER_INIT(3_secure, OMAP3_SECURE_TIMER, OMAP3_CLKEV_SOURCE,
> 2, OMAP3_MPU_SOURCE)
> OMAP_SYS_TIMER(3_secure)
> OMAP_SYS_TIMER_INIT(3_am33xx, 2, OMAP4_MPU_SOURCE, 1, AM33XX_RTC32K_SOURCE) <--- ignoring CONFIG_OMAP_32K_TIMER here
> OMAP_SYS_TIMER(3_am33xx)
>
> #endif
> --------------------------------------------
>
> to
>
> --------------------------------------------
> #ifdef CONFIG_ARCH_OMAP3
>
> #ifndef CONFIG_SOC_OMAPAM33XX
>
> #ifndef CONFIG_IPIPE
> OMAP_SYS_TIMER_INIT(3, 1, OMAP3_CLKEV_SOURCE, 2, OMAP3_MPU_SOURCE)
> #else
> OMAP_SYS_TIMER_INIT(3, 2, OMAP3_CLKEV_SOURCE, 3, OMAP3_MPU_SOURCE)
> #endif
> OMAP_SYS_TIMER(3)
> OMAP_SYS_TIMER_INIT(3_secure, OMAP3_SECURE_TIMER, OMAP3_CLKEV_SOURCE,
> 2, OMAP3_MPU_SOURCE)
> OMAP_SYS_TIMER(3_secure)
>
> #else // #ifdef CONFIG_SOC_OMAPAM33XX
>
> #ifndef CONFIG_IPIPE
> OMAP_SYS_TIMER_INIT(3_am33xx, 2, AM33XX_CLKEV_SOURCE, 1, AM33XX_MPU_SOURCE)
> #else
> OMAP_SYS_TIMER_INIT(3_am33xx, 2, AM33XX_CLKEV_SOURCE, 3, AM33XX_MPU_SOURCE)
> #endif
> OMAP_SYS_TIMER(3_am33xx)
>
> #endif
>
> #endif
> --------------------------------------------
>
> With the AM33XX_CLKEV_SOURCE and AM33XX_MPU_SOURCE defined as
>
> --------------------------------------------
> #define OMAP4_MPU_SOURCE "sys_clkin_ck"
> #define AM33XX_MPU_SOURCE OMAP4_MPU_SOURCE
> #define AM33XX_RTC32K_SOURCE "clk_32768_ck"
>
> #ifdef CONFIG_OMAP_32K_TIMER
> #define AM33XX_CLKEV_SOURCE AM33XX_RTC32K_SOURCE
> #define OMAP3_SECURE_TIMER 12
> #else
> #define AM33XX_CLKEV_SOURCE OMAP4_MPU_SOURCE
> #define OMAP3_SECURE_TIMER 1
> #endif
> --------------------------------------------
>
> That fixes the clock rate but otherwise does not change the fact that I'm stuck in a non ack'd interrupt at boot:
The problem comes from the flow handler change. The simplest fix is to
use the default flow handler. Is there any place I can download the
patch to have a look at it?
--
Gilles.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-12 19:54 [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC) Stephan Kappertz
2012-07-12 20:30 ` Gilles Chanteperdrix
@ 2012-07-13 9:00 ` Gilles Chanteperdrix
2012-07-13 9:15 ` Gilles Chanteperdrix
1 sibling, 1 reply; 8+ messages in thread
From: Gilles Chanteperdrix @ 2012-07-13 9:00 UTC (permalink / raw)
To: Stephan Kappertz; +Cc: xenomai
On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
> So far I found two things the AM3359 handles different compared to the Igepv2 board:
>
> 1) MACHINE_START has .handle_irq = omap3_intc_handle_irq while it is not defined in the ipipe-2.6 kernel repository
I see now, that is the problem. You have to either disable
CONFIG_MULTI_IRQ_HANDLER, or modify omap3_intc_handle_irq to call
ipipe_grab_irq instead of handle_IRQ.
You will get lower irq latency by disabling CONFIG_MULTI_IRQ_HANDLER.
--
Gilles.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-13 9:00 ` Gilles Chanteperdrix
@ 2012-07-13 9:15 ` Gilles Chanteperdrix
2012-07-13 14:37 ` Stephan Kappertz
0 siblings, 1 reply; 8+ messages in thread
From: Gilles Chanteperdrix @ 2012-07-13 9:15 UTC (permalink / raw)
To: Stephan Kappertz; +Cc: xenomai
On 07/13/2012 11:00 AM, Gilles Chanteperdrix wrote:
> On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
>> So far I found two things the AM3359 handles different compared to the Igepv2 board:
>>
>> 1) MACHINE_START has .handle_irq = omap3_intc_handle_irq while it is not defined in the ipipe-2.6 kernel repository
>
> I see now, that is the problem. You have to either disable
> CONFIG_MULTI_IRQ_HANDLER, or modify omap3_intc_handle_irq to call
> ipipe_grab_irq instead of handle_IRQ.
ipipe_handle_multi_irq instead of ipipe_grab_irq will allow the code to
work both with and without CONFIG_IPIPE
--
Gilles.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC)
2012-07-13 9:15 ` Gilles Chanteperdrix
@ 2012-07-13 14:37 ` Stephan Kappertz
0 siblings, 0 replies; 8+ messages in thread
From: Stephan Kappertz @ 2012-07-13 14:37 UTC (permalink / raw)
To: Gilles Chanteperdrix; +Cc: xenomai
Thanks for looking into this!
I modified the patch to support both options you suggested and uploaded it again. Both versions do work beautifully now.
- Stephan
Am 13.07.2012 um 11:15 schrieb Gilles Chanteperdrix:
> On 07/13/2012 11:00 AM, Gilles Chanteperdrix wrote:
>> On 07/12/2012 09:54 PM, Stephan Kappertz wrote:
>>> So far I found two things the AM3359 handles different compared to the Igepv2 board:
>>>
>>> 1) MACHINE_START has .handle_irq = omap3_intc_handle_irq while it is not defined in the ipipe-2.6 kernel repository
>>
>> I see now, that is the problem. You have to either disable
>> CONFIG_MULTI_IRQ_HANDLER, or modify omap3_intc_handle_irq to call
>> ipipe_grab_irq instead of handle_IRQ.
>
> ipipe_handle_multi_irq instead of ipipe_grab_irq will allow the code to
> work both with and without CONFIG_IPIPE
>
> --
> Gilles.
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-07-13 14:37 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-07-12 19:54 [Xenomai] Xenomai 2.6.1 on beaglebone (TI AM3359 SoC) Stephan Kappertz
2012-07-12 20:30 ` Gilles Chanteperdrix
2012-07-12 20:47 ` Gilles Chanteperdrix
2012-07-13 7:09 ` Stephan Kappertz
2012-07-13 7:32 ` Gilles Chanteperdrix
2012-07-13 9:00 ` Gilles Chanteperdrix
2012-07-13 9:15 ` Gilles Chanteperdrix
2012-07-13 14:37 ` Stephan Kappertz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.