Re: PREEMPT_RT : 2.6.16-rt12 and boot : BUG ?

["Serge Noiraud" <serge.noiraud@bull.net>]

Hi,

	many thanks for all these explanations.

mercredi 12 Avril 2006 03:58, Steven Rostedt wrote/a écrit :
> On Tue, 2006-04-11 at 18:15 +0200, Serge Noiraud wrote:
> > Hi,
...
> > 
> > I have several questions :
> > Is there a problem in my config file ?
> 
> Not that I know of. (perhaps debugging options are on. Ingo?)
> Or you have too many things as modules (that's our problem, not yours).
> 
> > Will this memory freed at end of kernel loading ?
> 
> No
> 
> > Why do we need such a size ?
> 
> It seems that the -rt kernel has increased the size of structures that
> are used in modules and are defined per cpu.
> 
> > What usage is this for ?
> 
> There are variables that are defined per CPU.  The reason for variables
> to be defined special for each CPU is that you want the variables in
> their own cache line such that modifying a variable that is specific for
> a CPU wont cause a write to a cache line that has a variable (say read
> only) to all CPUS. Because this would cause strain on the bus and slow
> things down as the write to the cache line is causing the other CPUs to
> update that line and become coherent.
> 
> But to make this easier for developers and to actually save space (don't
> want to waste the cache alignment just to space out variables), there is
> a lot of linker magic to do all the work for you.  So all a developer
> needs to do to declare a variable with DEFINE_PER_CPU(type, name) and
> friends and the linker takes care of the rest.
> 
> Currently this is implemented by creating a section at compile time to
> hold all these variables.  At compile time, only one set is made.  When
> the machine boots up, this section is copied (cache aligned) NR_CPUS
> times.  And to access these variables, a macro per_cpu(var, cpu) is used
> to find the variable in this index.  Note: since the size of
> PERCPU_ENOUGH_ROOM is used if it is bigger than the current compile time
> section, PERCPU_ENOUGH_ROOM must be a multiple of the cache size or
> there can be an overlap in the CPU cache lines. (hmm, this looks like a
> patch is needed.)
> 
> Now the problem you have is with modules.  Since the variables in the
> per_cpu() macro are looked up via an index and cpu, all these variables
> must be located in the same section.  Currently, to make this easier,
> (and this too probably should change), the per_cpu variables of a module
> are put in this same section.  So when a module is loaded, it finds a
> block in the per cpu area that is available and makes a copy of its per
> cpu variables into each section (per cpu).  But since this is static
> memory (the per cpu section cant grow) it must be allocated at boot up
> hoping that there's enough room for the modules that will be loaded in
> the future.  Don't worry about leaks, when a module is unloaded, it
> frees up the space in the per cpu area.
> 
> What you have seen, is that the -rt patch grew something that the
> modules were using per cpu.  So when it tried to allocate the space in
> the per cpu area, there wasn't enough room.  So your module failed to be
> loaded.
> 
> Hope this helps,
> 
> -- Steve


-- 
Serge Noiraud