Re: [RFC][PATCH 6/6] automatic tuning applied to some kernel components

[Nadia Derbey <Nadia.Derbey@bull.net>]

Andrew Morton wrote:
>>On Tue, 16 Jan 2007 07:15:22 +0100 Nadia.Derbey@bull.net wrote:
>>The following kernel components register a tunable structure and call the
>>auto-tuning routine:
>>  . file system
>>  . shared memory (per namespace)
>>  . semaphore (per namespace)
>>  . message queues (per namespace)
> 
> 
> This is the part of the patch series which really matters, and I just don't
> understand it :(
> 
> Why do we want to autotune these things?  What problem is this patch series
> solving?  Please describe this part of the work much, much more completely,
> so we can understand the need to add such a large amount of code to the
> kernel.

1) why these tunables?
The ipc tunables have been selected as "guinea-pig" tunables for the AKT 
framework because they are likely to be often used in data bases. This 
applies to file-max too.
Now, if the framework itself is accepted, the set of impacted tunables 
can easily be enhanced.

2) why autotuning:
There are at least 3 cases where it can be useful
. for workloads that are known to need a big amount of a given resource 
type (say shared memories), but we don't know what the maximum amount 
needed will be
. to solve the case of multiple applications running on a single system, 
and that need the same tunable to be adjusted to feet their needs
. to make a system correctly react to eventual peak loads for a given 
resource usage, i.e. make it tune up *and down* as needed.

In all these cases, the akt framework will enable the kernel to adapt to 
increasing / decreasing resource consumption:
1) avoid allocating "a priori" a big amount of memory that will be used 
only in extreme cases. This is the effect of doing an "echo <huge_value> 
 > /proc/sys/kernel/shmmni"
2) the system will come back to the default values as soon as the peak 
load is over.

> 
> It seems strange that the whole feature is Kconfigurable.  Please also
> explain the thinking behind that.

We wanted to make it configurable because it adds some overhead in terms of
1) generated kernel size
2) instructions added to the resource creation / removal code paths even 
if auto-tuning is not activated for th corresponding tunable -> 
performance impact.

> 
> I suspect the patches would be much simpler if you simply required that all
> these new tunables be of type `long'.  About seven eighths of the code
> would go away.  As would most of those eye-popping macros.
> 

Yes, agree with you: the idea here was to make the framework more 
generic. But I can change that.

Regards,
Nadia