[Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[M. Koehrer]

Hi everybody,

I just noted another very interesting thing.
When I have the Xenomai support built directly into the kernel (no modules),
the Linux scheduler assigns all Linux tasks to CPU 0 (I have a SMP with a Dual Core CPU).
Using e.g. taskset I can of course force tasks to run on CPU 1 but it will not be done
automatically.

However, when I use modules for the Xenomai stuff, the Linux scheduler assigns
the tasks to both available CPUs...

This behaviour looks very strange to me...
My setup: Kernel 2.6.19.2, SMP, Pentium 4 D, Xenomai 2.3. + COW patches.
Any idea on this?!?

Thanks and regards

Mathias



-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

M. Koehrer wrote:
> Hi everybody,
> 
> I just noted another very interesting thing.
> When I have the Xenomai support built directly into the kernel (no modules),
> the Linux scheduler assigns all Linux tasks to CPU 0 (I have a SMP with a Dual Core CPU).
> Using e.g. taskset I can of course force tasks to run on CPU 1 but it will not be done
> automatically.
> 
> However, when I use modules for the Xenomai stuff, the Linux scheduler assigns
> the tasks to both available CPUs...
> 
> This behaviour looks very strange to me...
> My setup: Kernel 2.6.19.2, SMP, Pentium 4 D, Xenomai 2.3. + COW patches.
> Any idea on this?!?

Mathias, mind to re-check if this behaviour persists with latest SVN
(v2.3.x and/or trunk) and I-pipe 1.7-02? If yes, please file a short bug
report to

https://gna.org/bugs/?group=xenomai

so that it doesn't get lost (we plan to manage non-trivial bug-tracking
increasingly via that channel).

I consider this a non-critical issue. Still, it has to be understood and
probably fixed.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Dmitry Adamushko]

Hi,

I suppose, that's what happens.

__xeno_sys_init() (./ksrc/nucleus/module.c) which is module_init() and
when compiled in -> initcall()

calls xnarch_init() (./include/asm-i386/bits/init.h)

This one amongst other things does

...
#ifdef CONFIG_SMP
        /* Make sure the init sequence is kept on the same CPU when
           running as a module. */
        set_cpus_allowed(current, cpumask_of_cpu(0));
#endif /* CONFIG_SMP */

In fact, "running as a module" seems to be a bit
misleading here.  Then another macro should be used in addition,
MODULE or how it's called.

Now guess who is a "current" here? It's "init" (pid 0).

init() -> do_basic_setup() -> do_initcalls() -> ... ->
__xeno_sys_init() -> xnarch_init()

"init" is a first process to run and all the rest will inherit its
"cpus_allowed" field.

That's it.

-- 
Best regards,
Dmitry Adamushko

Re: Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[M. Koehrer]

Hi!

I am very busy this week, I can do a check next week.
However, it looks as Dmitry aleady got a clue to that issue... 

Regards

Mathias
> Hi,
> 
> I suppose, that's what happens.
> 
> __xeno_sys_init() (./ksrc/nucleus/module.c) which is module_init() and
> when compiled in -> initcall()
> 
> calls xnarch_init() (./include/asm-i386/bits/init.h)
> 
> This one amongst other things does
> 
> ...
> #ifdef CONFIG_SMP
>         /* Make sure the init sequence is kept on the same CPU when
>            running as a module. */
>         set_cpus_allowed(current, cpumask_of_cpu(0));
> #endif /* CONFIG_SMP */
> 
> In fact, "running as a module" seems to be a bit
> misleading here.  Then another macro should be used in addition,
> MODULE or how it's called.
> 
> Now guess who is a "current" here? It's "init" (pid 0).
> 
> init() -> do_basic_setup() -> do_initcalls() -> ... ->
> __xeno_sys_init() -> xnarch_init()
> 
> "init" is a first process to run and all the rest will inherit its
> "cpus_allowed" field.
> 
> That's it.
> 
> -- 
> Best regards,
> Dmitry Adamushko
> 

-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Philippe Gerum]

On Mon, 2007-02-19 at 12:01 +0100, Dmitry Adamushko wrote:
> Hi,
> 
> I suppose, that's what happens.
> 
> __xeno_sys_init() (./ksrc/nucleus/module.c) which is module_init() and
> when compiled in -> initcall()
> 
> calls xnarch_init() (./include/asm-i386/bits/init.h)
> 
> This one amongst other things does
> 
> ...
> #ifdef CONFIG_SMP
>         /* Make sure the init sequence is kept on the same CPU when
>            running as a module. */
>         set_cpus_allowed(current, cpumask_of_cpu(0));
> #endif /* CONFIG_SMP */
> 
> In fact, "running as a module" seems to be a bit
> misleading here.  Then another macro should be used in addition,
> MODULE or how it's called.
> 
> Now guess who is a "current" here? It's "init" (pid 0).
> 
> init() -> do_basic_setup() -> do_initcalls() -> ... ->
> __xeno_sys_init() -> xnarch_init()
> 
> "init" is a first process to run and all the rest will inherit its
> "cpus_allowed" field.
> 
> That's it.
> 

Eh, that's what I call a nice one Dmitry. This is the perfect example of
legacy code stabbing us in the back. We should indeed definitely check
for MODULE && SMP.

-- 
Philippe.

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Dmitry Adamushko]

On 19/02/07, Philippe Gerum <rpm@xenomai.org> wrote:
> On Mon, 2007-02-19 at 12:01 +0100, Dmitry Adamushko wrote:
> > Hi,
> >
> > I suppose, that's what happens.
> >
> > __xeno_sys_init() (./ksrc/nucleus/module.c) which is module_init() and
> > when compiled in -> initcall()
> >
> > calls xnarch_init() (./include/asm-i386/bits/init.h)
> >
> > This one amongst other things does
> >
> > ...
> > #ifdef CONFIG_SMP
> >         /* Make sure the init sequence is kept on the same CPU when
> >            running as a module. */
> >         set_cpus_allowed(current, cpumask_of_cpu(0));
> > #endif /* CONFIG_SMP */
> >
> > In fact, "running as a module" seems to be a bit
> > misleading here.  Then another macro should be used in addition,
> > MODULE or how it's called.
> >
> > Now guess who is a "current" here? It's "init" (pid 0).
> >
> > init() -> do_basic_setup() -> do_initcalls() -> ... ->
> > __xeno_sys_init() -> xnarch_init()
> >
> > "init" is a first process to run and all the rest will inherit its
> > "cpus_allowed" field.
> >
> > That's it.
> >
>
> Eh, that's what I call a nice one Dmitry. This is the perfect example of
> legacy code stabbing us in the back. We should indeed definitely check
> for MODULE && SMP.

Yep, as the very least. But even in this case, it's not very nice that
via cpus_allowed we affect - scheduling-wise - a process which loads
xeno modules and all its children as It's likely to be a shell.
I haven't looked for further details now but isn't it possible to
restore the actual state of "cpus_allowed" right after the init
sequence has been done?

>
> --
> Philippe.
>


-- 
Best regards,
Dmitry Adamushko

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Philippe Gerum]

On Mon, 2007-02-19 at 13:13 +0100, Dmitry Adamushko wrote:
> On 19/02/07, Philippe Gerum <rpm@xenomai.org> wrote:
> > On Mon, 2007-02-19 at 12:01 +0100, Dmitry Adamushko wrote:
> > > Hi,
> > >
> > > I suppose, that's what happens.
> > >
> > > __xeno_sys_init() (./ksrc/nucleus/module.c) which is module_init() and
> > > when compiled in -> initcall()
> > >
> > > calls xnarch_init() (./include/asm-i386/bits/init.h)
> > >
> > > This one amongst other things does
> > >
> > > ...
> > > #ifdef CONFIG_SMP
> > >         /* Make sure the init sequence is kept on the same CPU when
> > >            running as a module. */
> > >         set_cpus_allowed(current, cpumask_of_cpu(0));
> > > #endif /* CONFIG_SMP */
> > >
> > > In fact, "running as a module" seems to be a bit
> > > misleading here.  Then another macro should be used in addition,
> > > MODULE or how it's called.
> > >
> > > Now guess who is a "current" here? It's "init" (pid 0).
> > >
> > > init() -> do_basic_setup() -> do_initcalls() -> ... ->
> > > __xeno_sys_init() -> xnarch_init()
> > >
> > > "init" is a first process to run and all the rest will inherit its
> > > "cpus_allowed" field.
> > >
> > > That's it.
> > >
> >
> > Eh, that's what I call a nice one Dmitry. This is the perfect example of
> > legacy code stabbing us in the back. We should indeed definitely check
> > for MODULE && SMP.
> 
> Yep, as the very least. But even in this case, it's not very nice that
> via cpus_allowed we affect - scheduling-wise - a process which loads
> xeno modules and all its children as It's likely to be a shell.

It's insmod.

> I haven't looked for further details now but isn't it possible to
> restore the actual state of "cpus_allowed" right after the init
> sequence has been done?
> 

Not sure, unless we explicitely ask the I-pipe to notify us when
module_inits terminate.

> >
> > --
> > Philippe.
> >
> 
> 
-- 
Philippe.

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Dmitry Adamushko]

On 19/02/07, Philippe Gerum <rpm@xenomai.org> wrote:
> On Mon, 2007-02-19 at 13:13 +0100, Dmitry Adamushko wrote:
> > > > ...
> > > > #ifdef CONFIG_SMP
> > > >         /* Make sure the init sequence is kept on the same CPU when
> > > >            running as a module. */
> > > >         set_cpus_allowed(current, cpumask_of_cpu(0));
> > > > #endif /* CONFIG_SMP */
> > > >
> > > > In fact, "running as a module" seems to be a bit
> > > > misleading here.  Then another macro should be used in addition,
> > > > MODULE or how it's called.
> > > >
> > > > Now guess who is a "current" here? It's "init" (pid 0).
> > > >
> > > > init() -> do_basic_setup() -> do_initcalls() -> ... ->
> > > > __xeno_sys_init() -> xnarch_init()
> > > >
> > > > "init" is a first process to run and all the rest will inherit its
> > > > "cpus_allowed" field.
> > > >
> > > > That's it.
> > > >
> > >
> > > Eh, that's what I call a nice one Dmitry. This is the perfect example of
> > > legacy code stabbing us in the back. We should indeed definitely check
> > > for MODULE && SMP.
> >
> > Yep, as the very least. But even in this case, it's not very nice that
> > via cpus_allowed we affect - scheduling-wise - a process which loads
> > xeno modules and all its children as It's likely to be a shell.
>
> It's insmod.

Oh yep :) Don't think it's a problem then. SMP && MODULE should be ok.

> --
> Philippe.
>

-- 
Best regards,
Dmitry Adamushko

Re: [Xenomai-help] Standard linux assignment of processes to CPUs depends on Xenomai Config

[Philippe Gerum]

On Mon, 2007-02-19 at 13:46 +0100, Dmitry Adamushko wrote:
> On 19/02/07, Philippe Gerum <rpm@xenomai.org> wrote:
> > On Mon, 2007-02-19 at 13:13 +0100, Dmitry Adamushko wrote:
> > > > > ...
> > > > > #ifdef CONFIG_SMP
> > > > >         /* Make sure the init sequence is kept on the same CPU when
> > > > >            running as a module. */
> > > > >         set_cpus_allowed(current, cpumask_of_cpu(0));
> > > > > #endif /* CONFIG_SMP */
> > > > >
> > > > > In fact, "running as a module" seems to be a bit
> > > > > misleading here.  Then another macro should be used in addition,
> > > > > MODULE or how it's called.
> > > > >
> > > > > Now guess who is a "current" here? It's "init" (pid 0).
> > > > >
> > > > > init() -> do_basic_setup() -> do_initcalls() -> ... ->
> > > > > __xeno_sys_init() -> xnarch_init()
> > > > >
> > > > > "init" is a first process to run and all the rest will inherit its
> > > > > "cpus_allowed" field.
> > > > >
> > > > > That's it.
> > > > >
> > > >
> > > > Eh, that's what I call a nice one Dmitry. This is the perfect example of
> > > > legacy code stabbing us in the back. We should indeed definitely check
> > > > for MODULE && SMP.
> > >
> > > Yep, as the very least. But even in this case, it's not very nice that
> > > via cpus_allowed we affect - scheduling-wise - a process which loads
> > > xeno modules and all its children as It's likely to be a shell.
> >
> > It's insmod.
> 
> Oh yep :) Don't think it's a problem then. SMP && MODULE should be ok.
> 

Fixed, thanks.

> > --
> > Philippe.
> >
> 
-- 
Philippe.