Bloat report 2.6.3 -> 2.6.4

Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

I'm working on an automated scheme to measure the growth of the kernel
from release to release. The current methodology is to do comparisons
from x to y, with the default config for x in both cases.  This is
automated with bloat-o-meter from linux-tiny and my kpatchup script
(http://selenic.com/kpatchup) plus a couple glue scripts. I intend to
post these reports regularly with new releases so comments on
improving the content are appreciated.

Here's a current sample report [with some explanatory comments
interspersed].

2.6.3 -> 2.6.4

   text	   data	    bss	    dec	    hex	filename
3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3

[ Results of size <a> <b>. -c2.6.3 means both kernel images were built
with the 2.6.3 defconfig. The dec column tells us that the 2.6.4
kernel image has grown by almost 1% in one release. Output from
bloat-o-meter follows.]

add/remove: 241/88 grow/shrink: 380/265 up/down: 44772/20838

[ A quick summary of modifications. 241 new symbols in 2.6.4, 88
disappeared, 380 functions grew, 265 shrank, and total growth and
shrinkage of listed symbols]

[ Next we list the growth and shrinkage of given functions, sorted by
delta. Symbols that aren't present in a given version are reported as
- and treated as size zero. I'm currently only listing the top 15 gainers
and decliners in the report to keep things manageable, though
bloat-o-meter will generate a complete list. ]

function                                     old     new   delta
n_tty_receive_buf                           1991    4450   +2459
_csr1212_read_keyval                           -    1253   +1253
hpsb_allocate_and_register_addrspace           -    1139   +1139
iso_stream_schedule                            -     820    +820
do_shmat                                       -     767    +767
csr1212_append_new_cache                       -     519    +519
csr1212_generate_csr_image                     -     510    +510
iso_stream_init                                -     505    +505
csr1212_new_icon_descriptor_leaf               -     409    +409
early_serial_init                              -     398    +398
inet_confirm_addr                              -     374    +374
csr1212_parse_dir_entry                        -     350    +350
setup_early_printk                             -     349    +349
do_unblank_screen                              -     348    +348
csr1212_generate_tree_subdir                   -     342    +342
...
iso_stream_find                              495     199    -296
parport_parse_params                         321       -    -321
parport_setup                                749     419    -330
itd_stream_schedule                          342       -    -342
nodemgr_read_text_leaf                       348       -    -348
rpc_proc_read                                350       -    -350
svc_proc_read                                361       -    -361
read_businfo_block                           436       -    -436
nodemgr_scan_root_directory                  470       -    -470
acpi_boot_init                               684     157    -527
nodemgr_scan_unit_directory                  541       -    -541
ide_end_taskfile                             589       -    -589
sys_shmat                                    767       -    -767
ohci_init_config_rom                         955       -    -955
n_tty_receive_char                          2037       -   -2037

[ Note that renames like sys_shmat to do_shmat show up in the top and
the bottom. ]

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[Andrew Morton]

Matt Mackall <mpm@selenic.com> wrote:
>
> 2.6.3 -> 2.6.4
> 
>    text	   data	    bss	    dec	    hex	filename
> 3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
> 3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3
> 
> [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> with the 2.6.3 defconfig.

But defconfig was changed between 2.6.3 and 2.6.4.

Re: Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> Matt Mackall <mpm@selenic.com> wrote:
> >
> > 2.6.3 -> 2.6.4
> > 
> >    text	   data	    bss	    dec	    hex	filename
> > 3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
> > 3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3
> > 
> > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > with the 2.6.3 defconfig.
> 
> But defconfig was changed between 2.6.3 and 2.6.4.

Yes, and I'm attempting to compensate for that because defconfig
changes tend to overwhelm other stuff in the results. 

My strategy here doesn't work as well as I'd hoped. I'm taking the
defconfig from the previous kernel and then running yes "" | make
oldconfig, which sets any new symbols to their defaults. So this deals
with case where existing symbols change defaults, but doesn't address
new symbols at all.

And what's happening with some of the new symbols is that they're off
in defconfig but on in Kconfig. So I need to come up with a way to
take the old defconfig and merge in new symbols from the new
defconfig. Then throw it at make oldconfig to drop out any obsolete
symbols.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[Adrian Bunk]

On Fri, Mar 12, 2004 at 05:53:49PM -0600, Matt Mackall wrote:
> On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > Matt Mackall <mpm@selenic.com> wrote:
> > >
> > > 2.6.3 -> 2.6.4
> > > 
> > >    text	   data	    bss	    dec	    hex	filename
> > > 3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
> > > 3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3
> > > 
> > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > with the 2.6.3 defconfig.
> > 
> > But defconfig was changed between 2.6.3 and 2.6.4.
> 
> Yes, and I'm attempting to compensate for that because defconfig
> changes tend to overwhelm other stuff in the results. 
> 
> My strategy here doesn't work as well as I'd hoped. I'm taking the
> defconfig from the previous kernel and then running yes "" | make
> oldconfig, which sets any new symbols to their defaults. So this deals
> with case where existing symbols change defaults, but doesn't address
> new symbols at all.
> 
> And what's happening with some of the new symbols is that they're off
> in defconfig but on in Kconfig. So I need to come up with a way to
> take the old defconfig and merge in new symbols from the new
> defconfig. Then throw it at make oldconfig to drop out any obsolete
> symbols.

You have to realize that your "automated scheme to measure the growth of 
the kernel" doesn't work as you might have expected.

As an example, consider the following option gets added:

  config ACPI_NEW_OPTION
	bool "new option"
	depends on ACPI
	default y
	help
	  This option adds support for $new_feature.
	  It enlarges your kernel by about 100k.
	  It's save to say Y.


In your size metric, this option would be pure "bloat".


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

On Sat, Mar 13, 2004 at 06:08:40PM +0100, Adrian Bunk wrote:
> On Fri, Mar 12, 2004 at 05:53:49PM -0600, Matt Mackall wrote:
> > On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > > Matt Mackall <mpm@selenic.com> wrote:
> > > >
> > > > 2.6.3 -> 2.6.4
> > > > 
> > > >    text	   data	    bss	    dec	    hex	filename
> > > > 3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
> > > > 3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3
> > > > 
> > > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > > with the 2.6.3 defconfig.
> > > 
> > > But defconfig was changed between 2.6.3 and 2.6.4.
> > 
> > Yes, and I'm attempting to compensate for that because defconfig
> > changes tend to overwhelm other stuff in the results. 
> > 
> > My strategy here doesn't work as well as I'd hoped. I'm taking the
> > defconfig from the previous kernel and then running yes "" | make
> > oldconfig, which sets any new symbols to their defaults. So this deals
> > with case where existing symbols change defaults, but doesn't address
> > new symbols at all.
> > 
> > And what's happening with some of the new symbols is that they're off
> > in defconfig but on in Kconfig. So I need to come up with a way to
> > take the old defconfig and merge in new symbols from the new
> > defconfig. Then throw it at make oldconfig to drop out any obsolete
> > symbols.
> 
> You have to realize that your "automated scheme to measure the growth of 
> the kernel" doesn't work as you might have expected.
> 
> As an example, consider the following option gets added:
> 
>   config ACPI_NEW_OPTION
> 	bool "new option"
> 	depends on ACPI
> 	default y
> 	help
> 	  This option adds support for $new_feature.
> 	  It enlarges your kernel by about 100k.
> 	  It's save to say Y.
> 
> 
> In your size metric, this option would be pure "bloat".

I actually did explicitly note this problem in the part you clipped.
But I think it's fair to say that new features that are on by default
are in fact bloat in some sense.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[Adrian Bunk]

On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> 
> I actually did explicitly note this problem in the part you clipped.

I clipped nothing, I quoted your _complete_ mail.

> But I think it's fair to say that new features that are on by default
> are in fact bloat in some sense.

Perhaps in some sense, but not in any interesting sense.

For the average computer you can buy at your supermarket today it isn't 
very interesting whether the kernel is bigger by 1 MB or not.

People who need to care about the size of the kernel [1] use hand-tuned 
.config's that are far away from defconfig - and those people wouldn't 
enable unneeded features that are on by default.

You use a metric "size increase of a defconfig kernel [2]", and I simply 
claim that this metric doesn't measure anything useful for practical 
purposes.

cu
Adrian

[1] e.g. for small embedded systems, very old computers or
    boot _floppies_
[2] modulo some compensation for changed defconfig's

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Bloat report 2.6.3 -> 2.6.4

[Sam Ravnborg]

On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> But I think it's fair to say that new features that are on by default
> are in fact bloat in some sense.

You cannot do any metric based on defconfig, since per. definition defconfig is what
suits Linus's current i386 PC best.
If you really want to do a proper metric do something like the follwoing:

make allnoconfig
Use some sed/awk magic to enable the options you are interested in
- Networking
- procfs?
- ext2
- One net driver
- A bit more which make sense
make oldconfig

This should result in a reliable config for code metrics.
You just have to inspect that noone of your manually added options
are no longer recognized (grep for warnings in output).

If you get this working it would be nice to see the evolution starting from 2.6.1 and
onwards in line with John Cherrys compile stats.

	Sam

Re: Bloat report 2.6.3 -> 2.6.4

[Horst von Brand]

Matt Mackall <mpm@selenic.com> said:
> On Fri, Mar 12, 2004 at 03:22:06PM -0800, Andrew Morton wrote:
> > Matt Mackall <mpm@selenic.com> wrote:
> > >
> > > 2.6.3 -> 2.6.4
> > > 
> > >    text	   data	    bss	    dec	    hex	filename
> > > 3313135	 660247	 162472	4135854	 3f1bae	vmlinux-2.6.3-c2.6.3
> > > 3342019	 664154	 162344	4168517	 3f9b45	vmlinux-2.6.4-c2.6.3
> > > 
> > > [ Results of size <a> <b>. -c2.6.3 means both kernel images were built
> > > with the 2.6.3 defconfig.
> > 
> > But defconfig was changed between 2.6.3 and 2.6.4.
> 
> Yes, and I'm attempting to compensate for that because defconfig
> changes tend to overwhelm other stuff in the results. 

Use oldconfig?

Or just compare defconfig kernels (with all changes). Or all modules, ...

I.e., what are you trying to measure here?
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> > 
> > I actually did explicitly note this problem in the part you clipped.
> 
> I clipped nothing, I quoted your _complete_ mail.

Oh, sorry, indeed you did. And it was this bit:

 > And what's happening with some of the new symbols is that they're
 > off in defconfig but on in Kconfig. So I need to come up with a way
 > to take the old defconfig and merge in new symbols from the new
 > defconfig. Then throw it at make oldconfig to drop out any obsolete
 > symbols.

> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
> 
> Perhaps in some sense, but not in any interesting sense.
> 
> For the average computer you can buy at your supermarket today it isn't 
> very interesting whether the kernel is bigger by 1 MB or not.
>
> People who need to care about the size of the kernel [1] use hand-tuned 
> .config's that are far away from defconfig - and those people wouldn't 
> enable unneeded features that are on by default.

And my coverage of creep in other _commonly used_ parts of the kernel
would then be nil. Given that allyesconfig can't be expected to build
a kernel on any given day, defconfig is the least arbitrary and most
useful of arbitrary choices.

> You use a metric "size increase of a defconfig kernel [2]", and I simply 
> claim that this metric doesn't measure anything useful for practical 
> purposes.

defconfig is not an unreasonable approximation of features people use. 
If something is added to defconfig, odds are that people will start
using it. Not perfect, obviously, but I've yet to see anyone suggest
anything else that actually provides some coverage.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

On Sat, Mar 13, 2004 at 11:17:41PM +0100, Sam Ravnborg wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
> 
> You cannot do any metric based on defconfig, since per. definition defconfig is what
> suits Linus's current i386 PC best.
> If you really want to do a proper metric do something like the follwoing:
> 
> make allnoconfig
> Use some sed/awk magic to enable the options you are interested in
> - Networking
> - procfs?
> - ext2
> - One net driver
> - A bit more which make sense
> make oldconfig

And make oldconfig will go and turn on all the new stuff which is off
in defconfig and on in Kconfig.

But starting with a minimal config is not all that useful because it
doesn't show bloat creep in stuff people commonly use.

The point is not to distill everything down to one number, the point
is to get enough into the report that we can notice growth in all the
important areas. Obviously as things get renamed, moved around, etc.,
some closer inspection will be required to figure out what the impact
is. Changes in config workings are in the same bucket.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[Adrian Bunk]

On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
> On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
>...
> > > But I think it's fair to say that new features that are on by default
> > > are in fact bloat in some sense.
> > 
> > Perhaps in some sense, but not in any interesting sense.
> > 
> > For the average computer you can buy at your supermarket today it isn't 
> > very interesting whether the kernel is bigger by 1 MB or not.
> >
> > People who need to care about the size of the kernel [1] use hand-tuned 
> > .config's that are far away from defconfig - and those people wouldn't 
> > enable unneeded features that are on by default.
> 
> And my coverage of creep in other _commonly used_ parts of the kernel
> would then be nil. Given that allyesconfig can't be expected to build
> a kernel on any given day, defconfig is the least arbitrary and most
> useful of arbitrary choices.
> 
> > You use a metric "size increase of a defconfig kernel [2]", and I simply 
> > claim that this metric doesn't measure anything useful for practical 
> > purposes.
> 
> defconfig is not an unreasonable approximation of features people use. 

What exactly is your goal?

As already said:
  *** For the average user, the size of the kernel doesn't matter *** [1]
  *** People that care about size don't use defconfig ***

> If something is added to defconfig, odds are that people will start
> using it. Not perfect, obviously, but I've yet to see anyone suggest
> anything else that actually provides some coverage.

Did you ever consider that your approach of an "automated scheme" might 
be an approach of very limited value?

cu
Adrian

[1] OK, 10 MB more would matter, but we are more in the ranges of 
    perhaps a few hundreds kB

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Bloat report 2.6.3 -> 2.6.4

[Matt Mackall]

On Sun, Mar 14, 2004 at 01:32:20AM +0100, Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
> > On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
> >...
> > > > But I think it's fair to say that new features that are on by default
> > > > are in fact bloat in some sense.
> > > 
> > > Perhaps in some sense, but not in any interesting sense.
> > > 
> > > For the average computer you can buy at your supermarket today it isn't 
> > > very interesting whether the kernel is bigger by 1 MB or not.
> > >
> > > People who need to care about the size of the kernel [1] use hand-tuned 
> > > .config's that are far away from defconfig - and those people wouldn't 
> > > enable unneeded features that are on by default.
> > 
> > And my coverage of creep in other _commonly used_ parts of the kernel
> > would then be nil. Given that allyesconfig can't be expected to build
> > a kernel on any given day, defconfig is the least arbitrary and most
> > useful of arbitrary choices.
> > 
> > > You use a metric "size increase of a defconfig kernel [2]", and I simply 
> > > claim that this metric doesn't measure anything useful for practical 
> > > purposes.
> > 
> > defconfig is not an unreasonable approximation of features people use. 
> 
> What exactly is your goal?
> 
> As already said:
>   *** For the average user, the size of the kernel doesn't matter *** [1]
>   *** People that care about size don't use defconfig ***

Neither of these things matter. The important thing is that defconfig
encompasses a range of common options that are likely to be used, thus
people care about growth in those areas regardless of what subset or
superset they actually use. It is not possible to see growth in the
code for option FOO if option FOO is not enabled. As I pointed out in
the last message, allyesconfig would be ideal for my purposes and
fails both of the above quite dramatically.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: Bloat report 2.6.3 -> 2.6.4

[John Cherry]

On Sat, 2004-03-13 at 14:17, Sam Ravnborg wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
> > But I think it's fair to say that new features that are on by default
> > are in fact bloat in some sense.
> 
> You cannot do any metric based on defconfig, since per. definition defconfig is what
> suits Linus's current i386 PC best.
> If you really want to do a proper metric do something like the follwoing:
> 
> make allnoconfig
> Use some sed/awk magic to enable the options you are interested in
> - Networking
> - procfs?
> - ext2
> - One net driver
> - A bit more which make sense
> make oldconfig
> 
> This should result in a reliable config for code metrics.
> You just have to inspect that noone of your manually added options
> are no longer recognized (grep for warnings in output).
> 
> If you get this working it would be nice to see the evolution starting from 2.6.1 and
> onwards in line with John Cherrys compile stats.

I would be glad to add bloat stats to the existing compile statistics. 
However, as has been mentioned, the default config options (defconfig,
allyesconfig, and allmodconfig) change over time.  The value in the
bloat statistics would be up to the developer who could interpret these
stats relative to the current default configs.

> 
> 	Sam
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Bloat report 2.6.3 -> 2.6.4

[Mike Fedyk]

Matt Mackall wrote:
> On Sun, Mar 14, 2004 at 01:32:20AM +0100, Adrian Bunk wrote:
> 
>>On Sat, Mar 13, 2004 at 05:59:40PM -0600, Matt Mackall wrote:
>>
>>>On Sat, Mar 13, 2004 at 06:57:13PM +0100, Adrian Bunk wrote:
>>>...
>>>
>>>>>But I think it's fair to say that new features that are on by default
>>>>>are in fact bloat in some sense.
>>>>
>>>>Perhaps in some sense, but not in any interesting sense.
>>>>
>>>>For the average computer you can buy at your supermarket today it isn't 
>>>>very interesting whether the kernel is bigger by 1 MB or not.
>>>>
>>>>People who need to care about the size of the kernel [1] use hand-tuned 
>>>>.config's that are far away from defconfig - and those people wouldn't 
>>>>enable unneeded features that are on by default.
>>>
>>>And my coverage of creep in other _commonly used_ parts of the kernel
>>>would then be nil. Given that allyesconfig can't be expected to build
>>>a kernel on any given day, defconfig is the least arbitrary and most
>>>useful of arbitrary choices.
>>>
>>>
>>>>You use a metric "size increase of a defconfig kernel [2]", and I simply 
>>>>claim that this metric doesn't measure anything useful for practical 
>>>>purposes.
>>>
>>>defconfig is not an unreasonable approximation of features people use. 
>>
>>What exactly is your goal?
>>
>>As already said:
>>  *** For the average user, the size of the kernel doesn't matter *** [1]
>>  *** People that care about size don't use defconfig ***
> 
> 
> Neither of these things matter. The important thing is that defconfig
> encompasses a range of common options that are likely to be used, thus
> people care about growth in those areas regardless of what subset or
> superset they actually use. It is not possible to see growth in the
> code for option FOO if option FOO is not enabled. As I pointed out in
> the last message, allyesconfig would be ideal for my purposes and
> fails both of the above quite dramatically.

With CONFIG_BROKEN, in the kernel for a while, why doesn't allyesconfig 
work on a stock kernel?  Maybe there are some logic errors in kconfig...

Re: Bloat report 2.6.3 -> 2.6.4

[Tim Bird]

Adrian Bunk wrote:
> On Sat, Mar 13, 2004 at 11:33:32AM -0600, Matt Mackall wrote:
>>But I think it's fair to say that new features that are on by default
>>are in fact bloat in some sense.
> 
> Perhaps in some sense, but not in any interesting sense.
> 
> For the average computer you can buy at your supermarket today it isn't 
> very interesting whether the kernel is bigger by 1 MB or not.
> 
> People who need to care about the size of the kernel [1] use hand-tuned 
> .config's that are far away from defconfig - and those people wouldn't 
> enable unneeded features that are on by default.
> 
> You use a metric "size increase of a defconfig kernel [2]", and I simply 
> claim that this metric doesn't measure anything useful for practical 
> purposes.

Well maybe the bloat meter is helpful for identifying bloated
features that the kernel developers added to the default configs,
so embedded guys can know to avoid them, or, if they're
interesting, try to unbloat.

Even hand tuners can use some help.  The kernel is vast and
progress is fast.  (Didn't mean to make a ryhme... ;)

=============================
Tim Bird
Architecture Group Co-Chair
CE Linux Forum
Senior Staff Engineer
Sony Electronics
E-mail: Tim.Bird@am.sony.com
=============================