[ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

Hi all,

Probably the biggest kernel data bloat issue is in the ARM land, but
it also seems that it's becoming a Linux generic issue too, so I
guess it could be discussed in either context.

The kernel data bloat issue is slightly related to the devicetree talks
as well as I'm seeing a lot of the device tree bindings heading the wrong
way both at the binding level and the device driver level.

Basically the data bloat issue is there for the arch code and drivers
and may not show up initially until things have headed the wrong way for
too long.

After struggling with the data issues in the ARM land over past few
years, I think we have now pretty good idea how to spot the issues
early and avoid building databases into Linux kernel.

So some kind of discussion on data bloat and how to avoid it might be
interesting to a bunch of maintainers?

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Greg KH]

On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> Hi all,
> 
> Probably the biggest kernel data bloat issue is in the ARM land, but
> it also seems that it's becoming a Linux generic issue too, so I
> guess it could be discussed in either context.

Why is it specific to ARM?  What is so unique to ARM that causes it to
"bloat"?

And what exactly do you mean by "bloat"?

> Basically the data bloat issue is there for the arch code and drivers
> and may not show up initially until things have headed the wrong way for
> too long.

What do you mean by this?  You seem to be very vague here.

greg k-h

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Jason Cooper]

On Wed, Jul 31, 2013 at 05:33:51AM -0700, Greg KH wrote:
> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > Hi all,
> > 
> > Probably the biggest kernel data bloat issue is in the ARM land, but
> > it also seems that it's becoming a Linux generic issue too, so I
> > guess it could be discussed in either context.
> 
> Why is it specific to ARM?  What is so unique to ARM that causes it to
> "bloat"?
> 
> And what exactly do you mean by "bloat"?

Perhaps he's referring to the multiplatform effort?  You'd get a lot of
board code that wouldn't be used at each boot.  This should be declining
though...

thx,

Jason.

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Mel Gorman]

On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> Hi all,
> 
> Probably the biggest kernel data bloat issue is in the ARM land, but
> it also seems that it's becoming a Linux generic issue too, so I
> guess it could be discussed in either context.
> 

Would scripts/bloat-o-meter highlight where the growth problems are?

-- 
Mel Gorman
SUSE Labs

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Greg KH <greg@kroah.com> [130731 05:39]:
> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > Hi all,
> > 
> > Probably the biggest kernel data bloat issue is in the ARM land, but
> > it also seems that it's becoming a Linux generic issue too, so I
> > guess it could be discussed in either context.
> 
> Why is it specific to ARM?  What is so unique to ARM that causes it to
> "bloat"?

I think it has so far showed up on ARM because of no discoverable busses,
but chances are it will be more of a generic problem.

> And what exactly do you mean by "bloat"?

Stuffing data to kernel that should not be in the kernel at all. Or
if the data is needed by kernel, there should be only one set of the
data defined rather than multiple copies of the data built into the
kernel for each SoC or driver variant.

> > Basically the data bloat issue is there for the arch code and drivers
> > and may not show up initially until things have headed the wrong way for
> > too long.
> 
> What do you mean by this?  You seem to be very vague here.

People are unnecessarily defining registers in kernel for similar devices
over and over again for each new SoC at the arch level and now more and
more at the driver level.

One example of that are device tree based drivers that don't describe
the actual hardware, but instead have a binding that points to an index
of defined registers in the driver.

One way to avoid these kind of bloat issues is to allow drivers to load
data at multiple points: Only abtolutely minimal set of data should be
static, some should only come from the bootloader as a device tree or
ACPI tables, and some is best to be loaded after booting from /lib/firmware.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Jason Cooper <jason@lakedaemon.net> [130731 07:00]:
> On Wed, Jul 31, 2013 at 05:33:51AM -0700, Greg KH wrote:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > > 
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> > 
> > Why is it specific to ARM?  What is so unique to ARM that causes it to
> > "bloat"?
> > 
> > And what exactly do you mean by "bloat"?
> 
> Perhaps he's referring to the multiplatform effort?  You'd get a lot of
> board code that wouldn't be used at each boot.  This should be declining
> though...

Yes that too, device tree and ACPI help there a great deal as we don't
need to build in board specific data for each supported board, but
instead have just one copy of the board specific data come from the
bootloader.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

Hi,

On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> > 
> > What do you mean by this?  You seem to be very vague here.
> 
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
>
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.

-ECONFUSED... DT passes only the base address and the size of the
address space. If some versions of the IP have slightly different
register layout, that needs to be treated at the driver, right ?

> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.

why would we put data blobs in /lib/firmware ? I know we have discussed
this at some length before, but I still don't get the idea that, just
because data shouldn't be in the kernel, we would bloat /lib/firmware
with blobs which aren't really firmwares.

It would be like adding ACPI tables to /lib/firmware :-p

-- 
balbi

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Greg KH]

On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > > 
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> > 
> > Why is it specific to ARM?  What is so unique to ARM that causes it to
> > "bloat"?
> 
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
> 
> > And what exactly do you mean by "bloat"?
> 
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
> 
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> > 
> > What do you mean by this?  You seem to be very vague here.
> 
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
> 
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.

Ok, and exactly how much "larger" does something like this cost as a
real number, and as a percentage of the size of the kernel?

thanks,

greg k-h

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Mel Gorman <mgorman@suse.de> [130731 08:28]:
> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > Hi all,
> > 
> > Probably the biggest kernel data bloat issue is in the ARM land, but
> > it also seems that it's becoming a Linux generic issue too, so I
> > guess it could be discussed in either context.
> > 
> 
> Would scripts/bloat-o-meter highlight where the growth problems are?

Well to some extent yes, the board/SoC/driver specific options are
often behind Kconfig options. So if you want to limit the set of
supported SoCs and drivers for the kernel you can optimize it out.

The bloat-o-meter won't help for things like checking that a device
tree binding really describes the hardware, and is not just pointing
to a table of defined registers in the device driver.

A lot of the board specific, SoC specific, driver specific, debug
specifc and so on "data" should not be in the kernel to start with
and we can provide the same level of supported features in the kernel.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Felipe Balbi <balbi@ti.com> [130802 01:11]:
> Hi,
> 
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > > 
> > > What do you mean by this?  You seem to be very vague here.
> > 
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> >
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
> 
> -ECONFUSED... DT passes only the base address and the size of the
> address space. If some versions of the IP have slightly different
> register layout, that needs to be treated at the driver, right ?

The driver shoud know how to handle various _types_ of registers rather
than contain a data what these registers are. Then the DT should define
the _types_ of registers that the hardware has. Any register names etc
are just debug data that can be handled with userspace tools using debugfs.
 
> > One way to avoid these kind of bloat issues is to allow drivers to load
> > data at multiple points: Only abtolutely minimal set of data should be
> > static, some should only come from the bootloader as a device tree or
> > ACPI tables, and some is best to be loaded after booting from /lib/firmware.
> 
> why would we put data blobs in /lib/firmware ? I know we have discussed
> this at some length before, but I still don't get the idea that, just
> because data shouldn't be in the kernel, we would bloat /lib/firmware
> with blobs which aren't really firmwares.

Because the amount of data coming from the bootloader should be pretty
minimal. And it is for ACPI, and device tree will run into performance
issues trying stuff all the data there.

And a huge amount of the data we have in platform_data, defined registers,
device tree etc is only needed by the kernel for debugging, or is only
needed later on for things like PM.
 
> It would be like adding ACPI tables to /lib/firmware :-p

I don't think ACPI has this issue, the set of data coming from ACPI
is pretty minimal. But I bet a lot of the ACPI drivers could also load
the debug related data as additional kernel modules or from /lib/firmware
especially if it's somehow SoC or board specific.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Greg KH <greg@kroah.com> [130802 01:16]:
> On Fri, Aug 02, 2013 at 12:53:53AM -0700, Tony Lindgren wrote:
> > * Greg KH <greg@kroah.com> [130731 05:39]:
> > > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > > Hi all,
> > > > 
> > > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > > it also seems that it's becoming a Linux generic issue too, so I
> > > > guess it could be discussed in either context.
> > > 
> > > Why is it specific to ARM?  What is so unique to ARM that causes it to
> > > "bloat"?
> > 
> > I think it has so far showed up on ARM because of no discoverable busses,
> > but chances are it will be more of a generic problem.
> > 
> > > And what exactly do you mean by "bloat"?
> > 
> > Stuffing data to kernel that should not be in the kernel at all. Or
> > if the data is needed by kernel, there should be only one set of the
> > data defined rather than multiple copies of the data built into the
> > kernel for each SoC or driver variant.
> > 
> > > > Basically the data bloat issue is there for the arch code and drivers
> > > > and may not show up initially until things have headed the wrong way for
> > > > too long.
> > > 
> > > What do you mean by this?  You seem to be very vague here.
> > 
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> > 
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
> 
> Ok, and exactly how much "larger" does something like this cost as a
> real number, and as a percentage of the size of the kernel?

Well one example has been making omap4 SoC booting device tree only, and
that has reduced the built in kernel data for pinmux, board support and
platform init code by something like 6000 lines, with patches posted to
reduce the clock related build in kernel data by about additional 1700
lines. Sure some of that has moved to live under drivers, but mostly defined
in the .dts files.

But I'm afraid quite a bit of stuff in general is now just moved to
drivers without dealing with the data issues properly. So I'm hoping
we could establish some guidelines on doing things that might help other
maintainers to catch and solve similar issues.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Tony Lindgren <tony@atomide.com> [130802 01:00]:
> * Greg KH <greg@kroah.com> [130731 05:39]:
> > On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> > > Hi all,
> > > 
> > > Probably the biggest kernel data bloat issue is in the ARM land, but
> > > it also seems that it's becoming a Linux generic issue too, so I
> > > guess it could be discussed in either context.
> > 
> > Why is it specific to ARM?  What is so unique to ARM that causes it to
> > "bloat"?
> 
> I think it has so far showed up on ARM because of no discoverable busses,
> but chances are it will be more of a generic problem.
> 
> > And what exactly do you mean by "bloat"?
> 
> Stuffing data to kernel that should not be in the kernel at all. Or
> if the data is needed by kernel, there should be only one set of the
> data defined rather than multiple copies of the data built into the
> kernel for each SoC or driver variant.
> 
> > > Basically the data bloat issue is there for the arch code and drivers
> > > and may not show up initially until things have headed the wrong way for
> > > too long.
> > 
> > What do you mean by this?  You seem to be very vague here.
> 
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
> 
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.
> 
> One way to avoid these kind of bloat issues is to allow drivers to load
> data at multiple points: Only abtolutely minimal set of data should be
> static, some should only come from the bootloader as a device tree or
> ACPI tables, and some is best to be loaded after booting from /lib/firmware.

Oh and thinking about it a bit more, this issue is mostly with the
device drivers implementing frameworks, not the device drivers
using the frameworks. Things like clocks, regulators, muxes etc where
an almost similar instance is repeated tens or hundreds of times for
each SoC.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Russell King - ARM Linux]

On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> Oh and thinking about it a bit more, this issue is mostly with the
> device drivers implementing frameworks, not the device drivers
> using the frameworks. Things like clocks, regulators, muxes etc where
> an almost similar instance is repeated tens or hundreds of times for
> each SoC.

That is where it helps to have a strong maintainer for a subsystem who
has the guts to refuse to accept stuff which is similar to existing
implementations and insist that existing implementations are either
adapted or reused.

It's all very well someone coming along and writing a "generic" set of
implementations (like tglx did for the IRQ subsystem) but unless there's
a motivation for people to use the generic stuff (such as... you won't
get your code in if you don't use the provided generics unless you can
provide a very good reasoned argument) then people are just going to
write their own code time and time again.

It's just like how the clocksources have gone.  We now have multiple
implementations of how to read a counter which ticks at a specific
rate.  You wouldn't think that I wrote drivers/clocksource/mmio.c which
can handle all of these simple 32-bit/16-bit up/down counter cases.
Again, the problem is there is no strong reviewer there who looks over
every addition and says "no, use the generic stuff".

That's the basic problem here: the review, and people saying "no" to
new stuff doing the same as generic stuff.

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Mike Turquette]

Quoting Tony Lindgren (2013-08-02 00:53:53)
> People are unnecessarily defining registers in kernel for similar devices
> over and over again for each new SoC at the arch level and now more and
> more at the driver level.
> 
> One example of that are device tree based drivers that don't describe
> the actual hardware, but instead have a binding that points to an index
> of defined registers in the driver.

Apologies for possibly hijacking this thread, but this issue keeps me up
at night. People use DT for different things and have different ideas
about its Purpose In The World.

Tony wants to move data out of the kernel, which was the impetus behind
the OMAP DT clock patches that describes every single clock in a DT node
(around 250 clocks). This create very large dts, but reduces the clock
drivers to pure logic, no data.

Other folks are motivated only to get rid of board files and platform
data hacks. They keep all of the clock data in the clock driver and
instead use DT only as a way to hook up clocks to devices via a simple
mapping, thus describing how individual boards or SoC variants are set
up outside of the kernel source. dts remains small, essentially just an
array to map bindings but all of the clock data remains in the kernel.

Both approaches have their merits and drawbacks. Is it possible to
gather consensus on selecting a single approach? For the clock subsystem
I've accepted drivers and DT bindings which do either. I simply do not
have the DT experience or sensibilities to draw a line in the sand...
and maybe I should not draw a line in the sand and just let people pick
whichever approach they prefer (which maintains the status quo).

If the ARM Summit figures out all the answers then please let me know
what they are :-)

Thanks,
Mike

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Matt Sealey]

On Fri, Aug 2, 2013 at 3:13 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Mel Gorman <mgorman@suse.de> [130731 08:28]:
>> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
>> > Hi all,
>> >
>> > Probably the biggest kernel data bloat issue is in the ARM land, but
>> > it also seems that it's becoming a Linux generic issue too, so I
>> > guess it could be discussed in either context.
>> >
>>
>> Would scripts/bloat-o-meter highlight where the growth problems are?
>
> Well to some extent yes, the board/SoC/driver specific options are
> often behind Kconfig options. So if you want to limit the set of
> supported SoCs and drivers for the kernel you can optimize it out.
>
> The bloat-o-meter won't help for things like checking that a device
> tree binding really describes the hardware, and is not just pointing
> to a table of defined registers in the device driver.

Specifically naming and shaming, like arch/arm/mach-imx/clk-*.c kind
of bloat where names of clocks, their bit definitions etc. are
extremely specific to the chip but also can change per-board and
should be in the device tree? It would be possible to list and parse
all these clocks in about 1/3rd of the code of one of those files and
have it work on every one of the SoCs they cover, if they're
adequately described in the device tree (which gives an opportunity to
stop changing the clock binding for each SoC to add new "numbers" for
"clocks we forgot" and also allow registration of ONLY the
board-specific clocks, on a per-board basis (if you don't use the TVE
or SATA, why define a clock for it just to turn it off?).

There's not a lot else going on with regards to "too much data in the
kernel that shouldn't be in the kernel" to my mind that won't
eventually get taken out once bindings in the DT. Most other platforms
are pretty sane in this regard - what is left is legacy stuff for
boards that don't have DTs and methods are in place to boot similar
boards with DT-only. As above, Mike Turquette and the OMAP guys are
doing something very similar here.

What you'd suggest/need is a review by maintainers of each platform
they support for any static data they are maintaining that can either
be excised by moving it to the device tree as boot
configuration/description data, if that data is not basically constant
anyway, or just by getting rid of rare and unused boards. The PowerPC
guys had a great cull when they moved from arch-ppc to arch-powerpc
and decided no non-DT boards allowed. Anything that didn't get ported
got deleted when arch-ppc went away. The arm arch kernel tree doesn't
have the luxury of a hard expiration in place..

~

BTW Mike, the solution is big device trees with clock data. The kernel
*should* be pure logic, not description and data tables (data tables
are exactly why Device Trees exist!) except where they are a
fundamental constant. Clock trees are a *PER BOARD WORLD* even if they
use the same SoC.

I would - and have since the dawn of time - advocate doing it the way
you're doing it (250 clocks in the tree!!! GANBATTE!!!) simply because
the "logic" behind mapping clocks to numbers is obtuse and needs to be
reproduced in very limited ways - adding a clock that was previously
undefined but of a common, already-used structure (like a gate or mux
that had no previous driver to use it) means modifying the kernel AND
the device tree (and update the binding!). In this case, they are
doing it wrong and I keep seeing patches to add 2 or 3 new clocks per
merge window, and absolutely NONE of them are the JTAG clock, I keep
having to add that in to every kernel I test. It shouldn't be a
compile-time thing to boot, it should be a "which DT do I pick, the
one with JTAG or the one without?" issue. We should ONLY have to
change the device tree - individual clock type bindings would be
fixed, boards would have to define their clocks. Since any external
clocks and peripherals have to be validated by board designers anyway
(for stability), everyone will know exactly what every board will need
per peripheral and be able to define their own clock trees with fixed
parents if need be. Vendors can provide suitable reference trees that
match the full, clumsy configuration of an entire SoC for board
bringup and we should be enabling (as in DT status="okay" property)
the clocks that make sense on that board.

There's a major advantage as above - if you do not use the clock on
your board, you do not have to specify it. It may be left on by the
bootloader and drain power and cause noise and screw parenting for
muxes, but then that's an issue for the bootloader dev guys to
resolve.

The reason consensus doesn't flow that way is because "it is a lot of
work to do this in device tree" but someone wrote out those 5 or 6
board files already and so far the clock subsystem has been rewritten
like 4 times in the meantime. None of Freescale's BSPs (for example)
use the same clock subsystem as they update to new kernels.

I *will* change the i.MX stuff one day if I ever find the time. I
started already but I need to forward port it to a modern version and
get one of my boards fixed..

--
Matt

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Olof Johansson]

On Fri, Aug 2, 2013 at 2:31 PM, Matt Sealey <neko@bakuhatsu.net> wrote:
> On Fri, Aug 2, 2013 at 3:13 AM, Tony Lindgren <tony@atomide.com> wrote:
>> * Mel Gorman <mgorman@suse.de> [130731 08:28]:
>>> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
>>> > Hi all,
>>> >
>>> > Probably the biggest kernel data bloat issue is in the ARM land, but
>>> > it also seems that it's becoming a Linux generic issue too, so I
>>> > guess it could be discussed in either context.
>>> >
>>>
>>> Would scripts/bloat-o-meter highlight where the growth problems are?
>>
>> Well to some extent yes, the board/SoC/driver specific options are
>> often behind Kconfig options. So if you want to limit the set of
>> supported SoCs and drivers for the kernel you can optimize it out.
>>
>> The bloat-o-meter won't help for things like checking that a device
>> tree binding really describes the hardware, and is not just pointing
>> to a table of defined registers in the device driver.
>
> Specifically naming and shaming, like arch/arm/mach-imx/clk-*.c kind

[...]

TL;DR. Some friendly advice is to reduce the verbosity of your emails.
I lost patience a couple of paragraphs in.


I think we could come a long way on this if we could sort out probe
and init order enough that we could modularize some of this data (and
init code). I.e. if you could load some of these tables and drivers as
modules a lot of the bloat would no longer be an issue.

The problem, of course, is that things like clock, pinmux, etc tables
(and drivers) tend to be needed very early during boot and thus can
definitely not wait to the point where we usually run ramdisk
contents.

We could mandate that whatever drivers are needed on top also be
modules, which would take care of some of the dependency chain, but we
don't have a great way today to describe the needed probe/load order
of said modules. So it'd still need some new and additional
functionality to work well.

Pushing everything out into modules will also add some complexity for
embedded platforms who rather avoid ramdisks, and it will likely add
boot time which matters a lot to some users. On the other hand, it's
less likely to matter to the users who at the same time care about the
bloat of a multiplatform kernel, i.e. you tend to ether optimize for
boot time on particular well-defined hardware, or you care more about
having something that boots on most hardware but possibly with some
overhead in boot performance.


-Olof

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Russell King - ARM Linux <linux@arm.linux.org.uk> [130802 06:31]:
> On Fri, Aug 02, 2013 at 05:41:31AM -0700, Tony Lindgren wrote:
> > Oh and thinking about it a bit more, this issue is mostly with the
> > device drivers implementing frameworks, not the device drivers
> > using the frameworks. Things like clocks, regulators, muxes etc where
> > an almost similar instance is repeated tens or hundreds of times for
> > each SoC.
> 
> That is where it helps to have a strong maintainer for a subsystem who
> has the guts to refuse to accept stuff which is similar to existing
> implementations and insist that existing implementations are either
> adapted or reused.
> 
> It's all very well someone coming along and writing a "generic" set of
> implementations (like tglx did for the IRQ subsystem) but unless there's
> a motivation for people to use the generic stuff (such as... you won't
> get your code in if you don't use the provided generics unless you can
> provide a very good reasoned argument) then people are just going to
> write their own code time and time again.
> 
> It's just like how the clocksources have gone.  We now have multiple
> implementations of how to read a counter which ticks at a specific
> rate.  You wouldn't think that I wrote drivers/clocksource/mmio.c which
> can handle all of these simple 32-bit/16-bit up/down counter cases.
> Again, the problem is there is no strong reviewer there who looks over
> every addition and says "no, use the generic stuff".
> 
> That's the basic problem here: the review, and people saying "no" to
> new stuff doing the same as generic stuff.

Right, but to make all that easier I was thinking that we can possibly
provide some generic guidelines and tools for maintainers to avoid some
of these issues.

The data issues should be pretty easy to spot based on the size of the
driver, or looking at the object file with size command for the data to
text ratio etc.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Mike Turquette <mturquette@linaro.org> [130802 13:04]:
> Quoting Tony Lindgren (2013-08-02 00:53:53)
> > People are unnecessarily defining registers in kernel for similar devices
> > over and over again for each new SoC at the arch level and now more and
> > more at the driver level.
> > 
> > One example of that are device tree based drivers that don't describe
> > the actual hardware, but instead have a binding that points to an index
> > of defined registers in the driver.
> 
> Apologies for possibly hijacking this thread, but this issue keeps me up
> at night. People use DT for different things and have different ideas
> about its Purpose In The World.
> 
> Tony wants to move data out of the kernel, which was the impetus behind
> the OMAP DT clock patches that describes every single clock in a DT node
> (around 250 clocks). This create very large dts, but reduces the clock
> drivers to pure logic, no data.

Yes that should eventually allow "booting new hardware with old
kernels".. But ideally with clocks we should support any combination of
DT defined clocks and /lib/firmware defined clocks should be allowed to
avoid bloating the DT files. We should be able to boot to user space
with a pretty minimal set of clocks, and deal with the PM related settings
based on SoC specific data from /lib/firmware.
 
> Other folks are motivated only to get rid of board files and platform
> data hacks. They keep all of the clock data in the clock driver and
> instead use DT only as a way to hook up clocks to devices via a simple
> mapping, thus describing how individual boards or SoC variants are set
> up outside of the kernel source. dts remains small, essentially just an
> array to map bindings but all of the clock data remains in the kernel.

Right, that just moves the real problem to drivers and then it will
eventually suck up all your maintainer time with constant churn to patch
that data for various SoC revisions :)
 
> Both approaches have their merits and drawbacks. Is it possible to
> gather consensus on selecting a single approach? For the clock subsystem
> I've accepted drivers and DT bindings which do either. I simply do not
> have the DT experience or sensibilities to draw a line in the sand...
> and maybe I should not draw a line in the sand and just let people pick
> whichever approach they prefer (which maintains the status quo).
> 
> If the ARM Summit figures out all the answers then please let me know
> what they are :-)

Well I think it's been discussed many times earlier and it should be
clear that Linus wants this kind of data out of the kernel. So maybe
we could establish some kind of set of standards for the maintainers
to follow.

Regards,

Tony

Re: [Ksummit-2013-discuss] [ATTEND] [ARM ATTEND] kernel data bloat and how to avoid it

[Tony Lindgren]

* Olof Johansson <olof@lixom.net> [130802 22:37]:
> On Fri, Aug 2, 2013 at 2:31 PM, Matt Sealey <neko@bakuhatsu.net> wrote:
> > On Fri, Aug 2, 2013 at 3:13 AM, Tony Lindgren <tony@atomide.com> wrote:
> >> * Mel Gorman <mgorman@suse.de> [130731 08:28]:
> >>> On Wed, Jul 31, 2013 at 12:38:03AM -0700, Tony Lindgren wrote:
> >>> > Hi all,
> >>> >
> >>> > Probably the biggest kernel data bloat issue is in the ARM land, but
> >>> > it also seems that it's becoming a Linux generic issue too, so I
> >>> > guess it could be discussed in either context.
> >>> >
> >>>
> >>> Would scripts/bloat-o-meter highlight where the growth problems are?
> >>
> >> Well to some extent yes, the board/SoC/driver specific options are
> >> often behind Kconfig options. So if you want to limit the set of
> >> supported SoCs and drivers for the kernel you can optimize it out.
> >>
> >> The bloat-o-meter won't help for things like checking that a device
> >> tree binding really describes the hardware, and is not just pointing
> >> to a table of defined registers in the device driver.
> >
> > Specifically naming and shaming, like arch/arm/mach-imx/clk-*.c kind
> 
> [...]
> 
> TL;DR. Some friendly advice is to reduce the verbosity of your emails.
> I lost patience a couple of paragraphs in.
> 
> 
> I think we could come a long way on this if we could sort out probe
> and init order enough that we could modularize some of this data (and
> init code). I.e. if you could load some of these tables and drivers as
> modules a lot of the bloat would no longer be an issue.
> 
> The problem, of course, is that things like clock, pinmux, etc tables
> (and drivers) tend to be needed very early during boot and thus can
> definitely not wait to the point where we usually run ramdisk
> contents.

I think there's no need for that any longer. We've pretty much sorted
out all the issues so not much is needed early on.

Sure some clocks might be needed for the SoC specific clockevent.
But if those can be defined in the .dts files only those need to
be initialized early on.

For the pinmux, bootloader already must set up pins for SDRAM so
not much should be needed early on. But if some pins are needed, those
should be defined in the .dts files and the rest can be initialized
later on based on .dts, loadable modules, or /lib/firmware.
 
> We could mandate that whatever drivers are needed on top also be
> modules, which would take care of some of the dependency chain, but we
> don't have a great way today to describe the needed probe/load order
> of said modules. So it'd still need some new and additional
> functionality to work well.

Hmm I think we're just missing a concept of initializing the SoC
specific things in stages as most of it can be done at module init
time as long as the needed pieces can be initialized early.
 
> Pushing everything out into modules will also add some complexity for
> embedded platforms who rather avoid ramdisks, and it will likely add
> boot time which matters a lot to some users. On the other hand, it's
> less likely to matter to the users who at the same time care about the
> bloat of a multiplatform kernel, i.e. you tend to ether optimize for
> boot time on particular well-defined hardware, or you care more about
> having something that boots on most hardware but possibly with some
> overhead in boot performance.

The Linux generic frameworks should support initializing things
in multiple stages. Some of them do, but not all of them. Then
depending on the use case, SoC specific things can be initialized
with a suitable combination of .dts entries, loadable modules and
data from /lib/firmware.

Regards,

Tony